Note to the reader:
This part of the field guide comes from our 2019 version of the UX Research Field Guide. Updated content for this chapter is coming soon!
Want to know when it's released?
Subscribe to our newsletter!
The UX Research Field Guide is a comprehensive how-to guide to user research. By the time you finish reading, you’ll be a total pro at doing user research—from planning it to conducting sessions to analyzing and reporting your findings.
Okay, actually that’s a bit of a fib. To become a total pro, you need to actually put the learnings from this Field Guide into practice (aka, do some research). But this Field Guide does contain all the knowledge you need to design, conduct, and analyze user research.
Before we get into all of that, however, let’s take a step back and answer the essential question:
User research is the practice of researching users. Pretty straightforward, right?
Well, yes and no.
As we’ll see in this Field Guide, user research—or UX research, or user experience research—takes many different forms. The methods can be qualitative or quantitative (and indeed the most impactful research studies often combine both types of methods), and can range from observing people in their environment to tracking their eye movements in a lab.
Regardless of methodology, the big-picture goal of user research is the same: to understand and build empathy for users in order to make better decisions, build better product experiences, and create solutions people actually want.
And businesses invest in user research because creating solutions people want is a great way to make money.
User testing, also called usability testing, is a type of user research in which participants are asked to perform tasks, typically within a specific interface. These sessions can be unmoderated (the user is prompted by a testing tool, and is not observed in real time) or moderated by a facilitator.
Usability testing—which we will discuss in-depth in the aptly named Usability Testing chapter—can be either qualitative or quantitative in nature.
As we mentioned, user research comes in all shapes and sizes—from months-long diary studies to five-second tests.
There are many different kinds of UX research, depending on how you slice it. We give a more thorough primer on the different categories in a later chapter. For now, here’s what you need to know...
User research can be:
The primary difference between qualitative and quantitative UX research lies in the nature of the data.
Quantitative UX research produces quantifiable data, aka numbers. It typically involves collecting and analyzing data from a large number of people to unambiguously answer questions like “how much?”, “how many?”, and “how often?”
Examples of quantitative research methods:
Qualitative UX research, on the other hand, involves directly assessing behaviors and beliefs to produce data on user preferences, motivations, and pain points. Due to its often subjective nature, qualitative methods ultimately rely on researcher analysis and interpretation to uncover patterns and answer the question: “Why?”
Examples of qualitative research methods:
Mixed methods research is exactly what it sounds like—the practice of using multiple kinds of research methods to answer research questions. Quant and qual are complementary—used together, they provide nuanced, data-backed insights that can help teams make more confident, user-centric decisions.
Note: The term mixed methods is used when the methods you’re mixing are both quantitative and qualitative. If you’re blending multiple qualitative methods (and you often will), that’s called hybrid research.
The difference between generative and evaluative methods lies in when the data is collected—and why.
Generative UX research (also called discovery research or exploratory research) involves direct observation, deep inquiry, and careful analysis to develop a rounded understanding of users—who they are, what they care about, what drives their behavior and decisions, and so on. The data from this type of research helps generate ideas and discover opportunities for improvement.
Examples of generative research methods:
Evaluative UX research helps teams figure out if an existing (or in-development) solution is on the right track—is it meeting user expectations, is it addressing their needs, is it desirable? Although evaluative UX research comes after discovery work, don’t wait until you have a polished product to test your ideas—validate early and often.
Examples of evaluative research methods:
The difference between attitudinal and behavioral research methods lies in who is describing the user.
Attitudinal UX research methods rely on self-reported data—in these methods, study participants tell researchers what they think. The data reflects people’s stated beliefs, perceptions, and expectations. Because some people aren’t able to fully articulate their perceptions—and because human beings are notoriously bad at predicting their own behavior—attitudinal data requires careful interpretation, and cannot always be taken at face value.
Examples of attitudinal research methods:
Behavioral UX research methods involve observing user behavior, either in the field or during prototype and product testing. Behavioral research data is reported by the user researcher (or by a usability testing tool).
Examples of behavioral research methods:
Many research methods produce both attitudinal and behavioral data—when interviewing users, for example, researchers may also take note of body language and non-verbal cues.
The difference between moderated and unmoderated research methods lies in the role of the researcher.
In moderated research, the researcher is involved during the research session. Depending on the method, they may be observers or may take a more active role as facilitators. Real-time moderation allows researchers to adapt their script and process in response to participant actions and engagement, and also to ask follow-up questions that probe more deeply into why participants make certain choices.
Examples of research methods that are typically moderated:
In unmoderated research, researchers sit on the sidelines as participants complete unobserved tests using testing platforms or tools that play the role of the researcher, prompting them to answer specific questions or perform specific tasks. Because it’s hands-off and can be conducted asynchronously, unmoderated testing is often faster and less expensive on a per-participant basis.
Examples of research methods that are typically unmoderated:
User research is sometimes thought of as a process that happens pre-design and development, with a bit of market research thrown in at the end.
But really, research has a valuable role to play at every stage of product development. The goals of your research—the questions you’re trying to answer, the decisions you’re trying to facilitate—will change over the course of the product development cycle, as will the methods you use.
Most projects begin with some discovery, in which you’re trying to pinpoint the problem and get a clearer picture of who you’re solving it for.
Your goals at this stage should be aimed at developing a detailed understanding of potential users, their needs, and their context. To do this, researchers use generative research methods like internal stakeholder interviews, user interviews, diary studies, card sorts, or even ethnographic field studies.
You might also use methods like surveys (with open-ended questions), existing product data, competitive analysis, literature reviews, and other fact-finding methods to gain a more detailed understanding of the context.
A great product that doesn’t solve a market need is not a great product. Don’t rely on your instincts (trust us), don’t try to copy what’s worked in the past, and don’t hang your hat on a bunch of quant that doesn’t provide context for the ‘why’ behind the ‘what’. When in doubt, talk to people.
📖 Read more about Discovery Research Methods
Once you’re actively building, you’ll want to evaluate your solutions and validate your decisions. At this point, you have a good sense of your market, user needs and pain points, and will have a concept of what kind of solution you can offer. You’ve taken those ideas and built some wireframes, high fidelity mockups, or interactive prototypes.
Now, you need to understand if these designs help users solve their problems, how they interact with them, and where they get hung up.
Your goal at this stage will be to answer questions about conceptual fit and the usability of the product using evaluative research methods like usability tests, preference tests, A/B tests, tree tests, first click tests (so many tests!), and task analysis.
If knowing what product to build in the first place is the most important aspect of research, understanding whether or not you’re succeeding in building that product is certainly a close runner up.
The more variety you can present in your prototypes the better. Try a few out of the box ideas next to what feels “safe.” Truly, you never know how users will react unless you test.
And that’s why this stage is so very important. You may even be missing key features you haven't thought of. Once you have your analysis in, you can refine your prototypes and re-test until you’ve found a solution you (and your participants) are happy with.
📖 Read more about Evaluative Research Methods
User research doesn’t (or shouldn’t) stop once you’ve launched a product or feature.
For one thing, you’ll need to monitor user feedback to understand how well your product continues to meet customer needs. And what’s more, those user needs will likely change—and you’ll need to keep your finger on the pulse of that, too.
To do this, researchers conduct ongoing UX research. This often involves partnering with product, marketing, and support teams to implement and monitor things like intercept surveys, user analytics, NPS scores, and support tickets.
Continuous research might also involve interviewing or re-interviewing customers to provide additional context to quantitative ongoing listening methods.
You’ve been testing throughout the previous stages of product development. You have a good sense of who your users are, and the context they live in when using your product. You know which solutions will best solve their problems and empower them to complete tasks they value. You’ve built those solutions and they are live. You are done!
Except, you are not done.
Here’s why: First, if you’ve focused on qualitative data to date (good for you, we love qualitative data) you have not thoroughly tested your actual product in the wild with a multitude of quantitative data. You’ll want to make sure what you put out there is accomplishing the goals you set for it in the first place.
Second, things change. What worked well a year ago or more may no longer be the best solution. You may get hints of this through ad hoc user feedback, NPS scores, proactive surveys, or other channels. You’ll want to adapt.
Finally, you create a very good solution to the user’s problem—yet it’s very possible that you did not create the perfect solution.
Keep testing and optimizing until you hit a point of diminishing returns. Some features are less critical than others, and good enough may very well be good enough. Others are essential to the success of your product. In any case, “set it and forget it” is not a smart product strategy. Ongoing listening methods help keep your product useful, impactful, and relevant over the long-haul.
📖 Read more about Continuous Research Methods
This is the first question many stakeholders will ask. Since user research costs the company time and money, early champions for UXR often find themselves having to justify its value to others in the organization.
The fact is, research isn’t an add-on or a nice-to-have, it’s an essential part of the product development process. There are plenty of very good reasons to do user research—here are some of the big ones….
User researchers are human beings and human beings are flawed. Very, very flawed. In fact, user researchers often refer to a huge cognitive bias map to keep track of the various ways our brain can trick us into making decisions without enough information. One of these cognitive biases is called false consensus—that little voice in your head that says “it’s ok, everyone will interact with this thing the same way I do.”
But the truth is, no matter how smart your people are, no matter how much they all personally use your product, no matter how long they’ve all worked on this project, you need to talk to the people outside of your organization who will eventually buy or use your product.
Because you don’t know what you don’t know, and you can’t build what people want if you don’t know what they want.
User research can help you understand the motivations behind user behaviors, uncover problems that need solving, and develop relevant solutions with market appeal. It can also give your team invaluable insight into the customer experience, and identify opportunities to improve it.
Conducting UX research can teach you:
Whether you’re shipping a new feature, iterating on an existing interface, or launching a new product into the market, a dash of user research can help make it better.
User research is a risk mitigation strategy—it can help save companies from wasting heaps of precious time (and therefore money!) on misguided projects.
Dr. Susan Weinschenk, in partnership with Human Factors International, calculated that the cost of fixing a problem post-development was 100x that of fixing it beforehand, and developers spent 50% of their time on rework that could have been avoided.
Fixing a problem post-development at 100x cost is a big deal—it can even mean the problem doesn’t get fixed at all, if there’s no time or budget left to devote to that project.
Even though user research can have modest upfront costs, it’s much better (and cheaper!) than spending more time and money later on a problem that could have been avoided from the beginning.
And we should know—User Interviews actually began as MobileSuites, an app that helped people staying in hotels check in and access amenities straight from their phone. Our founders thought it was a really cool idea and they started building. There was just one problem... Once it actually made it out into the world, it wasn’t as valuable to customers as they thought it would be. So, with a failed idea and 100k left in the bank, they pivoted.
Our founders started testing new product ideas. In the process, they discovered that trying to find people to talk to about product ideas was actually pretty difficult. They started hearing this from other people, too—and realized there was an opportunity here.
So they set a benchmark to prove that this problem actually existed and people would pay them to solve it: If more than 50% of the people they talked to brought up “participant recruitment” or “scheduling” naturally as one of their top 3 user research pain points, our founders would be in business.
The key was waiting for participants to bring up these problems on their own—our founders asked what people’s biggest problems were, based on real past experiences, and let them tell us what they needed.
Sure enough, user researchers needed a better way to recruit participants. Fast forward 6 years, 50,000 projects launched and one $10 million Series A later and we can confidently say that UX research is good for business.
👋 Read more about Customer Experience Research in this guide
That’s what the UX Research Field Guide is all about!
We shared a framework for Minimum Viable Research in the Introduction, if you’re looking for the TL;DR “I’ve got to do user research like, right this second” version.
Otherwise, head to the next chapter for a closer look at the UX research process, step by step.
