Types of User Research Methods

In the previous chapters, we’ve covered the fundamentals of UX research, gone over how to create a user research plan, and went over a foolproof framework for user research recruiting.

At this point, you’re probably brimming with all the essential knowledge and feeling ready to actually, y’know, do some user research.

But before we get into the individual methods (which we will, in depth), it’s worth taking time to review the different types of UX research methods at your disposal. Because there are a lot of different types of user research. 

Many methods are best suited to a particular scenario. Some only work well when combined with other, specifically complementary methods. And almost all deliver more valuable results when they are part of a comprehensive hybrid research strategy. 

That’s why, before you dive into the trenches, it’s helpful to take a step back and look at some of the broader categories that define different types of user research. 

This chapter provides the 30,000-foot view that gives you the overall lay of the land complete with the terms and topics that will help you decide which research method is best for your project. 

Let’s get to it.

In this chapter:

• Qualitative vs. quantitative methods
• Generative vs. evaluative methods
• Attitudinal vs. behavioral methods
• Remote vs. in-person user research 
• Moderated vs. unmoderated methods
• Which user research method should you use?

Qualitative vs. quantitative user research methods

The primary differences between qualitative and quantitative UX research have to do with how the data is collected and the nature of the data itself. 

Qualitative UX research typically involves collecting data through direct observation of a small group of people in order to assess behavior and answer the question: “Why?” Common qualitative research methods include interviews, focus groups, field studies, usability tests, and co-design sessions.

Quantitative UX research, on the other hand, usually involves collecting data from a much larger group of people in order to quantify a problem by answering the questions, “How much?” and “How many?” Common quantitative research methods include usability studies, surveys, click tests, card sorts, and A/B tests.

Which type of method you choose depends on the kind of question you’re asking. The qualitative and quantitative data produced by each type of research has unique strengths and weaknesses. 

Quantitative data provides very clear and unambiguous information about variables like how much, how many, and how often. The numerical nature of the data makes it easier to analyze, but it can also lack context. For example, while quantitative research can tell you fairly quickly and easily how often a set of users performs a particular task, that information is only valuable if you also have the context to judge whether that rate of task performance is good or bad. 

Qualitative data is more challenging to analyze and interpret. It is presented in the form of unstructured or semi-structured observational findings like comments, preferences, and motivations. While this kind of data usually contains the context, it doesn’t offer a black-and-white output, relying instead on researcher interpretation. It does, however, help identify root causes of behaviors, which then makes it easier to develop appropriate solutions. 

The two types of data may seem worlds apart, but they are actually highly complementary. When in doubt, mix methods for a more holistic view of the problem you’re trying to solve.

 As for whether it’s a good idea to skip the qualitative bit and lean into the statistics, well, Jakob Nielsen of Nielsen Norman Group had a few things to say about that:

“Quantitative studies must be done exactly right in every detail or the numbers will be deceptive. There are so many pitfalls that you're likely to land in one of them and get into trouble.

If you rely on numbers without insights, you don't have backup when things go wrong. You'll stumble down the wrong path, because that's where the numbers will lead.

Qualitative studies are less brittle and thus less likely to break under the strain of a few methodological weaknesses. Even if your study isn't perfect in every last detail, you'll still get mostly good results from a qualitative method that relies on understanding users and their observed behavior.”

Pro Tip: User Interviews is the fastest way to recruit participants for any kind of research. Talk to sales or sign up for a free account today.

Types of qualitative research 

The various qualitative research methods can be further categorized into 5 main research types: 

1. Ethnographic—observation of study participants in their natural environment to gain a better understanding of context, cultural insights, and participant’s day-to-day lives.
2. Narrative—in-depth research (typically interviews) with a small number of people to uncover narrative themes that illustrate how various factors can affect events and relationships.
3. Phenomenological—a variety of tactics (interviews, observation, documentation, etc.) used to describe and interpret a specific phenomenon or event, including participant perceptions and motivations.
4. Grounded theory—long-term, in-depth research to discover the social and psychological processes behind an event or situation, including its causes.
5. Case studies—a detailed accounting, either explanatory or exploratory, of an individual, organization, or event that reflects how things happen in the real world.

Note: That’s a good, concise overview of the differences between qualitative and quantitative research. If you’re looking for a deeper dive into the whole quant/qual thing, never fear—we’ve written a whole chapter on the subject.

Generative vs. evaluative user research methods

The primary differences between generative and evaluative UX research have to do with when the data is collected and why. Generative methods help you identify opportunities and ideas, while evaluative or evaluation research methods help you figure out if your existing solution is on the right track. 

Generative research (sometimes called foundational, exploratory, or (as in this Field Guide) discovery research) helps researchers gain a deep and highly detailed understanding of the audience, the market, and even internal project goals. It does a lot more than simply describe a user persona—it gets into the nitty-gritty of the real-world person who may eventually become a user. Generative research uses direct observation, deep inquiry, and careful analysis to develop a fully rounded, 360-degree understanding of the human beings in question—who they are, what their experiences are like (in relation to a product and to life in general), what they care about, what they believe in, how they think about the world, what drives their behavior and decisions.

The goal of discovery research is to unearth opportunities to innovate new solutions that will meet a specific and real need in the market. Without this, it’s all too easy to head off and develop a product no one wants. 

Evaluative research is used to evaluate people’s responses to a product or solution. It comes into play a little later in the product development process, but not that much later. Evaluative methods can deliver valuable insights as soon as you have an initial concept (even just a rough sketch or representative prototype), and should be used throughout the design and development process as a kind of reality check. Continuously putting design iterations in front of the relevant audience for feedback helps ensure that the final product delivers the experience people want while providing the solution they need.

Used together, generative and evaluative methods are like two sides of the same coin. One side helps you identify and define the problem you need to solve, while the other side makes sure you’re building the right solution to meet the need.

Discovery (generative) methods

Generative research methods include things like stakeholder interviews to uncover internal goals, field studies to provide real-life context, and in-depth user interviews.

Each of these methods uses a combination of first-hand observation, deep inquiry, and data analysis to ensure you’re building the right product at the right time for the right audience. Generative research methods often allow researchers to observe every aspect of a study participant’s engagement—verbal responses, actions, behavior, body language, and so forth. Researchers can gain a deeper understanding of the preferences and mental models by asking follow-up questions to uncover the “why” behind certain choices and behaviors. Any data—qualitative or quantitative—is analyzed to reveal relevant patterns in the responses. 

📖 Read more about Discovery Research Methods

Evaluative methods

There are a wide range of evaluative research methods, many of which are most valuable at specific stages of development. Their purpose is to validate whether or not your in-progress design is effectively solving the problem you identified during your generative research.

Evaluative methods employ many different approaches—from tree testing to qualitative usability testing, accessibility testing, A/B testing, and more—to determine what is and isn’t working in a design.  

📖 Read more about Evaluative Research Methods

Continuous research methods

Research doesn’t end once you’ve launched. In fact, it’s incredibly important to engage in ongoing listening and dialog with users using methods like NPS and other feedback surveys, product analytics, as well as customer support and feedback channels. Keeping the lines open is a critical part of sustaining your product’s value to users over the long term. 

Not only do continuous research methods give you your first look at how people experience your product in real life, they also help you adapt and evolve as things change. (And things will change—maintaining an ongoing conversation and really listening will help ensure that you don’t get left behind.)

📖 Read more about Continuous Research Methods

Attitudinal vs. behavioral user research methods

We know it’s shocking, but people often say one thing and do another. The disparity is usually unintentional, which is one of the reasons it’s so important to use UX research methods that collect both behavioral and attitudinal data. 

Attitudinal research methods rely on self-reported data. This means that they reflect people’s stated beliefs, perceptions, and expectations. This category includes things like interviews, surveys, focus groups, and card sorts—all of which ask study participants to tell researchers what they think.

The insights from attitudinal research methods are valuable, but they also need to be taken with a grain of salt. People sometimes fall short of telling you what they really think or are unable to fully articulate their perceptions. And even when they think they are telling you the whole truth and nothing but the truth, it can turn out that their expectations don’t match up with reality. (This is because human beings are notoriously bad at accurately predicting their own behavior.)

Despite these cautions, attitudinal research methods can still be helpful as a way of uncovering a participant’s mental model, which can then be used to shape a design to better meet user expectations.

Behavioral research methods are based on direct observation as a study participant interacts with a prototype or finished product. Researchers often prefer these methods since they are able to deliver more reliable insights based on real-world scenarios. Research methods that study a user’s actual actions include things like eye tracking, A/B tests, tree tests, first-click tests, and also user analytics.

There are also a number of research methods—user interviews and task analysis, for example—that can produce either attitudinal or behavioral data. In any situation, best practices include collecting a variety of both attitudinal and behavioral data. 

Remote vs. in-person user research 

The biggest and most obvious difference between in-person and remote research is that with remote research, the participant and the researcher are in different locations. 

Here is a quick run down of some of the advantages most often associated with each approach:

In-person research:

• Is easier to moderate because participants tend to be more attentive and can ask questions if they get stuck.
• Offer a more complete view of participant responses because researchers can combine verbal response and task-related actions with non-verbal cues.
• Provides more opportunities to ask follow-up questions or make additional in-the-moment inquiries about participant choices or thought processes.
• Gives researchers more control over the environment, including types of devices and software being used, internet connection, data security, noise, interruptions, and so forth.
• Creates better group dynamics, which can contribute to idea generation and useful dialog depending on the type of research being performed.
  

Remote research:

• Allows for much greater scheduling flexibility since there is no travel required.
• Is less expensive because there are no space-related costs and both recruitment and compensation costs are often lower than for in-person studies.
• Makes it easier to recruit a larger, more diverse group of participants.
• Comes with fewer recruitment constraints such as availability or geographic location.
• Can provide a more accurate reflection of real-life usage because participants can use their own equipment.
  

Remote research isn’t suited for all research methods. For example, ethnographic field studies are particularly hard to replicate remotely. 

But in many cases, the benefits of in-person research can be replicated in remote scenarios, giving researchers the best of both worlds: ultimate flexibility and cost-effectiveness without sacrificing the quality of responses. 

Moderated vs. unmoderated user research methods

The difference between moderated and unmoderated research methods is the role of the researcher. 

Unmoderated or automated research involves unobserved tests that a participant can complete at their own pace. The participant interacts with the product via an online platform or software that prompts them to answer specific questions or perform specific tasks. 

The nature of this kind of test makes it easier, less expensive, and faster to run than research facilitated by a professional moderator. The data from unmoderated research studies tends to be quantitative rather than qualitative, although some unmoderated tests include the option for participants to provide a recorded self-narration of their actions during the test. 

Examples of research methods that are often unmoderated include surveys, first-click tests, A/B tests, and user analytics.

In moderated research, a facilitator observes—either in person or remotely—as participants take part in the study. This real-time moderation allows researchers to adapt their script and process in response to participant actions and engagement, and also to ask follow-up questions that probe more deeply into why participants make certain choices. 

Because of the human element of introducing a facilitator, moderated tests can be more time consuming and more expensive. They can also require additional expertise and preparation since there is a specific skill set required to moderate a study effectively. And because the output usually includes a good deal of qualitative data, analyzing the results can also take a little more time.

Examples of research methods that are usually moderated include interviews, ethnographic field studies, focus groups, and task analysis.

Which user research method should you use?

The answer to this question is, of course, “it depends.”

The right user research method for any given study depends on a combination of factors, including which stage of product development you’re in, the nature of your research question, the types (and number) of participants you’re looking to recruit, your budget, and whether or not there’s a pandemic making in-person research particularly challenging!

There are several good frameworks out there to help you choose the right method, and we cover them in-depth in the upcoming chapter on how to choose a user research method.

💡Not sure which UX research method to choose? We created a free tool that can help.