Note to the reader:
This part of the field guide comes from our 2019 version of the UX Research Field Guide. Updated content for this chapter is coming soon!
Want to know when it's released?
Subscribe to our newsletter!
Getting started with user research can be intimidating. Even for experienced UX researchers, the variety of possible approaches to any given research project can feel daunting—with so many methods, which one should you choose? What is the best way to answer your research question? Is it the most efficient way that will keep your project within scope, under budget, and on time?
In the previous two chapters, we gave an overview of the different types of user research methods, and dove deeper into the important differences between qualitative and quantitative research. You may already have an idea, based on what you read and the pre-existing UXR knowledge you’re bringing to the table, which methods you want to use for your current project.
In this chapter, we’re going to help you solidify those choices, and introduce you to a few different frameworks for choosing the right user research method for any project that comes your way, or any research question that might crop up.
These frameworks—which can be adapted to your own research practice and the realities of your organization, customers, and unique constraints—will help you answer the question “which UX research method should I use?” efficiently and effectively every time.
Without further ado, here’s how to choose the right methods for your UX research study.
As ever, you’ll want to start with well-defined goals and clear research questions.
If you’re not already clear about the goals of your research, stop here. Do not pass go. Do not collect $200 in gift card incentives.
Circle back to the chapter on creating a user research plan for help identifying your research goals, which should be statements about what you’re trying to learn from your research.
To define your research goals, ask yourself:
You’ll want to further refine the answers to these questions through stakeholder research—your key stakeholders can clarify business goals and the kinds of decisions your research needs to enable to be successful.
Good research questions are specific, actionable, and practical. They also contain clues about who you need to recruit and which methods you’ll need to use.
Some example of good research questions:
In the likely event that you aren’t binge reading the UX Research Field Guide in a single sitting, it’s probably helpful to take a moment to revisit the different types of user research, along with the different questions they can help you answer.
User research methods can be divided along several different axes. They can be:
Qualitative UX research involves collecting data through direct observation of a small group of people in order to assess behavior and provide context. Qualitative methods produce unstructured or semi-structured observational findings like comments, preferences, and motivations.
Qualitative research helps you answer the questions:
Quantitative UX research, on the other hand, involves collecting data from a much larger group of people in order to quantify a problem and uncover patterns through statistical analysis.
Quantitative research helps you answer the questions:
Generative (discovery) UX research (also foundational, or exploratory research) uses direct observation, deep inquiry, and careful analysis to generate ideas (go figure) and discover opportunities for innovation that will meet a specific and real need in the market. Generative research is often (but not always) qualitative.
Discovery methods help you answer questions like:
Evaluative UX research is used to evaluate people’s responses to a product or solution. It is used throughout the product development cycle to test and validate the appeal, intuitiveness, and functionality of ideas, prototypes, and finished products.
Evaluative methods help you answer questions like:
Attitudinal UX research methods (like surveys and focus groups, for example) rely on self-reported data about people’s beliefs, perceptions, and expectations. Take what participants say with a grain of salt, especially if they’re referring to future behaviors—even the most self-aware user can’t predict the future.
Attitudinal research can help you answer questions like:
Behavioral UX research methods (like A/B tests and first click tests) are based on direct observations about how a study participant interacts with a prototype or finished product.
Behavioral research can help you answer questions like:
A user research framework is a systematic way of categorizing research methodologies and approaches to guide decisions about which method to use, when. Whether you’re a solo UX researcher or are working within a larger team, a reliable framework will provide consistency and can dramatically speed up research planning and decision-making.
They can be intricate, like this behemoth from UX researcher Lena Borodina:
Or they can be simpler, like Tomer Sharon's Lean User Research framework:
The best framework is the one that works for you. In order for it to be used effectively, a framework needs to be relevant to your team, your business goals, and your internal resources. If you don’t have the budget or people power for long ethnographic field studies, don’t include those types of studies in your framework. (Remember, frameworks can evolve.)
All of the frameworks in this chapter can and should be adapted to your own practice as necessary. We’ll cover how to map UX research methods to:
Let’s hop to it!
In the very first chapter, What Is User Research?, you learned about how (and why) to conduct user research at different stages of the product development cycle.
As you progress through the discovery, design, testing, implementation, and post-launch phases, the problems you need to solve will change. You’ll ask different questions, which will require different research methods to answer.
Most projects begin with some discovery, in which you’re trying to pinpoint the problem and get a clearer picture of who you’re solving it for.
The goals at this stage are to explore new directions, develop a detailed understanding of (potential) user needs and their context, and uncover opportunities. Both qualitative and quantitative approaches are used during this phase.
Conducting UX research during discovery helps you empathize and strategize.
Stakeholder Interviews—Understanding what your stakeholders ultimately value and need to know is a necessary component of effective user research. Part of a user researcher’s role is to align and articulate both business and user goals, and find where they intersect.
Ethnography—Ethnography is all about observing people and their habits in context (their natural habitats, if you want to get zoological about it). Field studies and online ethnographic studies can help you understand how people behave in groups and how they might interact with your product outside of a lab environment. What people do is so often different from what they report, and ethnographic studies can be a great way of getting around this issue.
Diary studies—When you need to understand long-term user behavioral patterns, diary studies are a great option. They provide insights into habits, changes over time, motivations, and long-term customer journeys. And they can be a solid alternative to ethnographic field studies when you’re on a limited budget.
Focus groups—Focus groups are not exactly the, ahem, darling of UX research circles these days. But they have their uses, especially when it comes to market research. Focus groups allow you to observe a lot of people relatively quickly. When you are very early in developing and marketing your product, this type of research can be useful in getting a broad view into the audience you seek to serve.
Generative user interviews—In-depth user interviews typically involve talking to participants one-on-one, asking them a set of non-leading questions, with an emphasis on past behaviors and perceptions. User interviews are valuable throughout the product development cycle, but are especially important in the early stages—the things you learn might confirm your hypotheses or lead to wildly different conclusions that reshape the direction of your inquiry.
Task analysis—Task analysis helps you better understand your user’s goals, and how they go about achieving them. This can be coupled with a variety of other methods, and is an important aspect of evaluating the overall user experience for most apps.
At this point, you have a good sense of your market, your users, and their needs. You’ve come up with an idea for a solution, are actively building (or have built) a prototype, and are ready for feedback.
This stage of product development is about evaluating your solutions and validating your decisions through both qualitative (formative) and quantitative means. The goals of testing are to validate conceptual fit, evaluate the usability of the design, and inform changes.
Eventually, you will reach a "go/no-go" decision point, when you transition into a period in which you are continually improving the design direction that you have chosen to reduce the risk of execution.
Conducting UX research during this phase helps you optimize and execute.
Qualitative usability testing—This method involves having participants think aloud as they interact with a prototype or product, allowing researchers to evaluate implicit and explicit cues and find patterns quickly.
A/B testing and multivariate testing—Whether you use A/B test, A/B/C test, single variant test, or multivariate test, the goal is to collect quantitative data on which version of your product best achieves the goal of the test—generally a hybrid of a business and user goal, for instance conversion rate on an ecommerce site.
First click testing—This is just what it sounds like. First click testing—in which you record the first click someone makes to accomplish their goal and analyze the results—is a popular way to assess the user’s ability to efficiently and effectively complete a task.
Card sorting and tree testing—Card sorting and tree testing are methods for testing your information architecture (IA), i.e. how you categorize and label content. Card sorts, which involve users grouping topics into categories, can provide great insight into your user’s mental models and how you should organize your content. Tree tests—in which users are tasked with finding information in a sitemap—are often used as a followup method to evaluate how intuitive your site navigation is.
Accessibility testing—The validation and testing stage should not be the first time accessibility crosses your mind; —accessible design begins with thoughtful planning and research design. By the time you get to this stage of product development, you should be using accessibility testing to stress test prototypes or products that have already been designed with accessibility in mind. In addition to running your product through accessibility checking tools, consider conducting qualitative usability studies with participants with disabilities.
You’ve launched a product… congratulations!
Now, keep researching it!
No, seriously—user research doesn’t (or shouldn’t) stop once you’ve launched a product or feature. “Set it and forget it” is not a smart product strategy. Ongoing listening methods help keep your product useful, impactful, and relevant over the long-haul.
The goals of post-launch research are to understand how well your product is performing, assess how well it continues to address customer needs, and identify opportunities for improvement. Methods are typically quantitative (summative) in nature, and allow you to measure product performance against itself or its competition.
Conducting UX research after launch helps you assess and iterate.
Feedback surveys—There’s a variety of survey types and tools that can help you meet your goals, from user intercept (in-app) surveys, to NPS surveys, to longer surveys that dig into specific aspects of the user experience. Like other methods at this stage, ongoing listening surveys typically require some cross-functional collaboration to do effectively. Talk to your product team to find out where and when an in-app survey would be most effective, partner with marketing to launch and monitor an NPS survey, and coordinate with customer development to segment and send email surveys to existing customers.
Analytics—Analytics can be a treasure trove of great quantitative data if you know how to analyze them and connect insight to action. Keeping a close track on key user flows, in-app behaviors, and business metrics will help you see changes over time.
Bug reports and support tickets—Bugs come up and need to be fixed. That’s just the reality. Make sure you have good bug reporting and systems to address them rapidly. Your dev team is probably on top of this, but you can use this data to understand where current or historical frustrations may have impacted the user experience in ways you wouldn’t have known otherwise. Similarly, FAQ and support desk reporting can help you understand where users are getting stuck. Taking proactive steps to clear up these moments in the product itself will improve the user experience and make your support team oh so happy.
Research is systematic inquiry. If it is not systematic, you’re not researching—you’re just being a busybody. To quote Roberta Dombrowski, our former VP of User Research at User Interviews:
"Research is the act of acquiring knowledge—we could literally conduct research on any topic under the sun. That endless potential scope is why it's so important that, as practitioners, the research we are leading and conducting is focused and aimed at enabling decisions for our organizations."
At User Interviews, we keep our research focused by using our own Decision Driven Research Framework, which is centered on the belief that the purpose of research is to enable decisions for your company, product, or service.
In this framework, the idea is to map your research methods to the types of decisions you want to enable. There are 4 types of decisions:
Below is a list of the most common methods we use when conducting research at User Interviews. For each method, we’ve created internal documentation about what the method is, the type of decisions it helps inform, and the output.
There are 5 stages to doing Decision Driven Research, the first three of which are all about creating a clear and focused plan for research. Here’s the process in full:
Many teams limit their research opportunities by consistently using the same familiar methods over and over again. And while it’s unlikely you’d ever need to use all the different UX research methods out there on a single project (what a nightmare), you should be prepared to mix up your playbook when the research question calls for it.
Nielsen Norman Group developed a helpful chart that illustrates where 20 popular methods fall along the following axes:
Attitudinal vs. behavioral—Attitudinal refers to what people say, while behavioral refers to what they do. Often there’s a pretty wide gap between the two. The most popular research methods blend elements of these two to capture both what people say and what they do.
Qualitative vs. quantitative—As we’ve been discussing, qualitative research data comes from observing behaviors and/or attitudes directly while quantitative research data is acquired indirectly via an instrument like a survey or web analytics tool.
Context of use—The final axis of NN/g’s three-dimensional framework looks at how study participants use the product in question (if indeed they do). Context of use can be defined as:
Using this chart, you can start to hone in on which methods will satisfy your research goals:
The folks at NN/g are careful to note that theirs, like all frameworks, is not set in stone. Few of the methods are fixed in a single place.
Most of the methods in the chart can move along one or more dimensions, and some do so even in the same study, usually to satisfy multiple goals. For example, field studies can focus on what people say (ethnographic interviews) or what they do (extended observations); desirability studies and card sorting have both qualitative and quantitative versions; and eye tracking can be scripted or unscripted.
User research rarely (if, indeed, ever) happens without constraints. You may be short on time, money, or people power. And that will impact which methods are available to you.
Say you want to learn about how participants interact with your product in a real-world context. Field studies would be an obvious choice—you could observe people in the field, using your product IRL. Great!
But field studies can take months.
If you don’t have the luxury of time, diary studies would be a solid option for learning about product use in-context. The best UX research method in this case is the one you can reasonably fit into your timeline and budget.
Unmoderated research involves unobserved tests that a participant can complete at their own pace using tools that prompt them to answer specific questions or perform specific tasks. The nature of this kind of test makes it easier, less expensive, and faster to run than research facilitated by a professional moderator.
Moderated research requires a moderator (go figure). Researchers observe as participants complete a study, which allows researchers to respond to participant actions on the fly and to ask follow-up questions that probe more deeply into why participants make certain choices.
Because of the human element of introducing a facilitator, moderated tests can be more time consuming and more expensive. They can also require additional expertise and preparation since there is a specific skill set required to moderate a study effectively. And because the output usually includes a good deal of qualitative data, analyzing the results can also take a little more time.
You shouldn't skip moderated research (which includes methods like interviews and qualitative usability testing). You really only need about 5 participants, anyway. But if you’re trying to collect data at scale, whether that’s qualitative or quantitative, remember that there’s only so much time in a day. Be realistic about how many facilitated research sessions your team can handle within a project’s timeline.
We don’t have a usability testing lab at User Interviews. Nor do we have a closet full of eye tracking devices or other biometric technology. In fact, we don’t have a lot of the things some user research teams rely on as part of their practice.
And that works for us! We’re a fully remote company, and we conduct remote UX research. We don’t need to monitor people’s heart rates or do research in clinically controlled environments to answer our research questions.
We also don’t pay for all-in-one research platforms. That’s less about cost, and more about the fact that we’d rather assemble our own toolkit with products that let us bring our own participants (recruited with User Interviews, naturally).
But that’s just us. The tools you need for user research might be different. Or not—many user research methods don’t require special software.
If your research question can be answered by simple means—do that. Research that’s hard to do doesn’t get done.
Well, that was kind of the whole point of this chapter. But if you’re still feeling stuck, we created a methods quiz that might help!
Editor’s note, December 2021: We’re in the process of updating the resources associated with each method recommendation in the UX Research Method Quiz.
Keep reading to learn more about each user research method in depth. First up: Discovery research methods!
