Bias is Unavoidable in UX Research—Here’s What to Do About It

Bias is a four-letter word in UX research. 

As a researcher, you do your best to avoid bias, to de-center your perspective and see past your preexisting judgements and opinions. 

But as a researcher, you’re also human—and humans are not immune to unconscious bias. 

So instead of asking, “how do I eliminate bias in my research?”, the better question might be: “how do I reduce bias in my research?”

In this article, we’ll discuss:

• Why avoiding bias in UX research is (mostly) impossible
• How to identify bias and keep it to a minimum

🛑 Why avoiding bias in UX research is (mostly) impossible

Research cannot be 100% objective—that is, researchers cannot be 100% objective. 

As humans, we all carry personal biases, assumptions, and associations. These cognitive biases aren’t always inherently harmful; they allow our brains to reduce cognitive load for more efficient decision-making. 

“We aren't impartial. We never are. We can do our best to be fair.” –Zoe Glas in 🎙️Navigating Gender, Religion, and Politics in UX Research 

When UX researchers aren’t aware of their own biases (or in denial about the extent to which their personal biases can affect their work), this can negatively influence the study design, leading to unethical data collection methods, skewed analysis, and inaccurate conclusions. Researcher bias can be especially prevalent in qualitative research, because the data collected is more subjective and reliant on the judgment of the researcher. 

Case in point: 6 design failures that could’ve been avoided with inclusive UX research. 

Although there are steps you can take to limit the negative effects of bias in your research (and I’ll talk about those steps in the next section), most people struggle to identify and mitigate all of their biases all of the time. 

“That’s why the first rule of product design is simple—be open-minded. It’s essential to be aware of your own biases and be ready to fight them.” - Nick Babich in Adobe XD Ideas

Additionally, bias is one of the most common concerns with research democratization. People who do research (PwDRs, or people outside of the core research team, like product managers, marketers, and designers) usually aren’t as well-trained to identify and reduce bias as professional UX researchers are. 

This doesn’t make successful democratization unattainable. It just means that, in democratized models, UX research leaders with a professionally-tuned eye for spotting bias need to have oversight into the research, analysis, and synthesis process on other teams. 

⚠️ Fraudulent or poor-fit participants can introduce bias into your study. Learn more in Research Participant Credibility: 8 Ways to Spot Bad Fits or Dishonesty.

🔎 How to identify and limit bias in UX research 

You can't avoid bias entirely, but you can identify and reduce bias in UX research. Below, you’ll find practical strategies for keeping bias to a minimum:

1. Learn about the different types of bias. 
2. Practice self-awareness to identify your own biases.
3. Learn to spot bias in others, too.
4. Discuss your assumptions, goals, and hypotheses in your research plan.
5. Ask a “critical friend” (or two) to review your research plan.
6. Mix methods.
7. Recruit a representative group of participants.
8. Take a moment of “me time” before sessions.
9. You know you shouldn’t ask leading questions, right?
10. Put your poker face on. 
11. Invite people from other teams to shadow sessions and take their own notes.
12. Take notes on facts, not opinions.

1. Learn about the different types of biases.

You first have to know what biases exist in order to recognize and mitigate them. There are many different types of bias to be aware of in UX research, including:

• Confirmation bias: The tendency to interpret data in a way that supports your preexisting beliefs.
• False-consensus bias: The tendency to see your own opinions and behaviors as being common, while viewing other opinions and behaviors as being uncommon.
• Social desirability bias: The tendency to answer questions in a way that will make you look good to others, i.e. over-reporting "good" behavior and under-reporting "bad" behavior.
• Serial position bias: The tendency to remember the first and last items in a list better than those in the middle.
• Recency bias: The tendency to attribute greater importance to recent events than to historic ones. (A form of serial position bias.)
• Primacy bias: The tendency to place greater emphasis on your first impressions than any information you encounter later on. (A form of serial position bias.)
• Anchoring bias: The tendency to rely too much on pre-existing reference points or "anchors" when making decisions.
• Peak-end bias: The tendency to place greater emphasis on intense emotional moments (peaks) and the final moments (end) of an experience.
• Implicit bias: A negative judgment, prejudice, or stereotype-confirming attitude that influences your decisions but that you don't consciously recognize.
• Hindsight bias: The tendency to believe that past events were more easily predictable than they were.
• Clustering illusion bias: The tendency to find false patterns and trends in random information when no such patterns exist.
• Framing bias: The tendency to make decisions based on how the information is presented or "framed" instead of the information itself.
• Sunk-cost bias: The tendency to continue a behavior if you've already invested time, effort, and money into it, even if the current costs outweigh the benefits.
• Transparency bias: The tendency to overestimate how well other people understand your own internal thoughts and feelings, or to overestimate how well you understand other people's thoughts and feelings.
• Fundamental attribution bias: The tendency to attribute someone's behavior to their character or personality, while downplaying the influence of situational and environmental factors.
• Survivorship bias: The tendency to focus on successful cases while overlooking the failed ones. 
• Ambiguity effect bias: The tendency to choose options for which the probability of a certain outcome is known, while avoiding options for which the outcome is ambiguous or uncertain.
• Racial bias: The tendency to believe that people of a specific racial group have distinctive characteristics and behaviors, generally leading to negative emotional reactions and discrimination. 
• Cultural bias: The tendency to interpret words or behaviors according to one’s own cultural norms, instead of considering the words and behaviors within the context of the participants’ own culture. 

This isn’t an exhaustive list of all the biases that exist (to see the broad scope of biases, check out this cognitive bias index) but these are the ones you’re most likely to encounter in UX research. 

2. Practice self-awareness to identify your own biases.

News flash: Everyone is a little bit biased. 

That doesn’t mean we’re all bad people, or that none of us can behave in ways that uphold standards of ethics, fairness, and equity. It just means that in order to make fair, unbiased judgments, we need to recognize the internal biases we carry and try to work around them. 

Becoming more self-aware is no easy feat, but there are frameworks and exercises that can help you work on it.

For example, the American Academy of Family Physicians (AAFP) uses the acronym IMPLICIT to provide strategies for identifying your personal biases: 

• Introspection: Take an inventory of your personal biases. Write them out, take implicit association tests, or explore them through other means. 
• Mindfulness: Use mindfulness techniques like yoga or meditation to reduce stress and in-the-moment reactivity (which could lead to more biased decision-making). 
• Perspective-taking: Consider life from the point of view of the people you tend to stereotype. Talk to them, read books about their experiences, and consider how you would feel if you were put in their position. 
• Learn to slow down: Instead of jumping to conclusions, pause and reflect on your personal biases before interacting with others. 
• Individualization: Everyone has distinctive characteristics that set them apart from any stereotypical box they might be placed in. Think about what makes people unique as individuals, as well as the characteristics you might have in common. 
• Check your messaging: Sometimes well-intentioned statements like “I don’t see color” can actually hurt instead of help. Before you talk about bias, do your research to learn evidence-based messaging that creates a welcoming environment for all. 
• Institutionalize fairness: Bias can be perpetuated on a systemic level as well as a personal level. Advocate for equity-driving policies and programs at your company to address these systemic-level issues. 
• Take two: Identifying and addressing your own bias requires humility and life-long commitment. Continue to practice mindfulness and don’t be afraid to start over when you’ve found yourself slipping. 

As a researcher, you can add some of these steps, such as introspection, mindfulness, and perspective-taking, into your research process by listing your assumptions at the start of every project (see #4 for more on this). 

Another common strategy for developing awareness of your own biases is by practicing reflexivity. Learn more about reflexivity in qualitative research.

3. Learn to spot bias in others, too.

Once you’ve become familiar with identifying bias in yourself, learn to recognize it in your teammates and stakeholders too—and don’t be afraid to (tactfully) call it out when it comes up. 

Common signs of bias in UX research include: 

• Repeated references to the same anecdote without data to suggest that it’s a common issue. For example, a Customer Success Manager might repeatedly bring up a feature request from one customer, but one request isn’t representative of the entire client base and likely not enough data to base your product roadmap on. 
• Using the same methods with the same participants repeatedly. For example, when you run multiple usability tests with the same group of users, they’re likely to get familiar with your research style and begin to behave differently. 
• Presenting assumptions as facts or patterns without data to support them. For example, a stakeholder might try to push for a certain product development, claiming that users want it when no research has been done into users’ sentiments regarding that product. 
• Consistently extreme (overly positive or overly negative) feedback. For example, a coworker might always respond poorly to the research reports of a colleague they have bias against, regardless of the reports’ quality. 
• Placing blame on individuals, rather than environmental or contextual factors, when an issue comes up. For example, a researcher might fault an observer for not taking good enough notes when the session didn’t record due to a technical error. 
• Only focusing on data that supports your opinion. For example, you might focus on a handful of survey responses that support your assumptions, even if the majority of the survey responses suggests that you are wrong. 

4. Identify your assumptions, goals, and hypotheses in your research plan. 

By identifying your biases in the early stages of your research project, you can design the study to mitigate them and continue to address them throughout the user research process. 

For example, Jennifer Ibrahim has a process for identifying her assumptions at the start of every project:

 “I use the “5W1h” methodology to break down my assumptions about: 
- who I believe my target users are
- what problems my solution solves for them
- when they would use my product (during a commute, drive, swim, etc. )
- where they would use it (car, mobile phone, desktop, watch)
- how they would use it ( on an app, web interface, physical product etc.).”

5. Ask a “critical friend” (or two) to review your research plan. 

Critical friends are people who can offer different viewpoints, ask difficult questions, and encourage a more nuanced research approach and analysis. They can be other researchers, stakeholders, partners, or PwDRs from other teams.  

Identify (formally or informally) a critical friend or friends, and ask them to offer their perspectives on:

• The study topic (i.e. are we prioritizing the right questions? Why this, why now?)
• The study design and data collection methods (i.e. are the methods we use appropriate for answering the research question?)
• The participant audience (i.e. are we talking to everyone we need to talk to? Is this sample representative of our user base?)
• The timeline and context (i.e. are we reaching users during a time and in an environment where they’ll behave naturally?) 
• The research question, screener questions, and any interview questions you might ask in-session 
• The (planned) approach to analysis

By getting different perspectives on your research plan, you’re more likely to identify and reduce areas of bias throughout the course of the study. 

6. Mix methods. 

Mixed methods research can reveal more nuanced insights, marrying the “what” of quantitative research with the “why” of qualitative research. 

Only focusing on one method over the other can sometimes create bias. As Erika Hall says on the Awkward Silences podcast: 

“It is too easy to run a survey. That is why surveys are so dangerous. They are so easy to create and so easy to distribute, and the results are so easy to tally. And our poor human brains are such that information that is easier for us to process and comprehend feels more true. This is our cognitive bias. This ease makes survey results feel true and valid, no matter how false and misleading. And that ease is hard to argue with.

It’s much much harder to write a good survey than to conduct good qualitative user research. Given a decently representative research participant, you could sit down, shut up, turn on the recorder, and get good data just by letting them talk. … But if you write bad survey questions, you get bad data at scale with no chance of recovery.”

The impact of this “bad data at scale” can be mitigated, in part, by pairing quantitative methods like surveys with qualitative methods like user interviews. The broader and more varied your data, the more likely you’ll be to notice inconsistencies and break through assumptions for real insights. 

7. Recruit a representative group of participants. 

One of the most common bias-inducers in UX research is the recruitment audience. 

If you’re only sampling from a small segment of users; failing to recruit for diversity, disability, and accessibility; or recruiting customers when you should be recruiting external participants (or vice versa), then you’re probably introducing bias into your study inadvertently. 

Make sure you’re following recruitment best practices, like:

• Defining the right audience and sample size
• Creating effective, unbiased screener surveys (and double-screening when necessary!)
• Offering appropriate incentives
• Soliciting participant applications in the right channels
• Avoiding participant fatigue by recruiting from different audiences 
• Using effective tools for recruiting and panel management

📚 Related reading: 8 Uncomplicated Customer Recruitment Strategies for Product Managers, UX Designers, and Marketers

8. Take a moment of “me time” before sessions. 

‍

Moderating back-to-back interviews all day is not a sustainable practice for any researcher—and the stress of that researcher fatigue increases the likelihood that you’ll make biased decisions. In her YouX 2023 session, Wellness and UX: Going Beyond the User Experience, Dr. Christelle Ngnoumen said: 

“Stress is the biggest bias generator we have to contend with in our work.”

Participants are more likely to disengage and experience panel fatigue when they notice you’re also burnt out, especially if they feel like they are not being understood or heard with the right listening skills and attention. As you’re heading into research sessions, take a moment to gauge your stress levels and recenter yourself so as not to influence the tone and outcome of the session. 

You can recenter yourself with a short mindfulness meditation, a few deep breaths, a quick journaling exercise, stretching or yoga, or any other strategies that work for you. 

💜 ✨ Visit the 2023 Self-Care Playbook for UX Researchers to learn research-backed strategies for self-care and workplace wellness.

9. You know you shouldn’t ask leading questions, right? 

According to Lorie Whitaker of Rackspace, asking leading or biased questions is the biggest mistake people make when approaching user research:

“I’ve seen countless product managers ask the equivalent of ‘We’ve been working on this feature for months. It’s supposed to help you do your task. What do you think about it?’ I’ve also seen people assume they know their users because some research was done years ago and perhaps they go and visit a few customers each year and make faulty assumptions based on that and then, because time, money and resources have been invested, they refuse to challenge or validate their hypothesis.”

These kinds of leading questions encourage unnatural responses from participants who are instinctively inclined to validate your ideas, as Marie Prokopets of FYI explains:

“It’s worth repeating: you’ll get skewed results by asking questions like, “What do you think of this idea?” People want to reaffirm what you’re doing. But just because they think the idea is cool doesn’t mean it’s something they’ll put down money for, nor does it mean that there’s enough of a market for your solution.”

The lesson for researchers: Be careful of your wording. Watch out for different kinds of biased questions, like:

• Leading questions, which prompt the desired (biased) answer, e.g. “Do you prefer the old, clunky prototype website or our newest, cleaned-up version of the site?”
• Double-barreled questions, which ask about two separate topics but only allow for one response, e.g. “How would you rate our support articles and customer service response time?” 
• Absolute questions, which only allow the participant to pick from two extreme answers, e.g. “Were you delighted by your experience shopping with us today, yes or no?”

10. Put your poker face on. 

Your body language can introduce bias into live research sessions. If you’re pleased, surprised, or disappointed by any responses you get from participants, you must not show it.

Physical gestures and facial expressions can indicate your emotions to participants and make them behave unnaturally. For example, if you frown and shake your head when they move to click a particular button, they may choose to click a different button instead. 

11. Invite people from other teams to shadow sessions and take their own notes.

Designers, product managers, customer success managers, and other stakeholders can offer unique insight into the data you collect, allowing for more objective and well-rounded insights. 

Invite them to join research sessions with you to observe and make note of the things that stick out to them. Their added points of interest can help fill in your blind spots, making objective observations where you might’ve been clouded with bias. 

Plus, encouraging stakeholders to occasionally shadow research sessions can improve your company’s culture and attitudes regarding research—it’s a win-win. 

12. Take notes on facts, not opinions.

Note-taking is one of the most under-rated skills in UX research. 

Whether you're moderating a session or just observing, you have to take quick, clear notes that will help you speed up the analysis process later on. 

Unfortunately, note-taking is another area of research where bias is often inadvertently introduced. It’s important to only include facts and observations, rather than trying to interpret participant behavior in your notes. 

For example, if a participant frowns when they open up a product’s redesigned interface, you shouldn’t jump to the conclusion that they don’t like it. Simply write down what they did: “the participant opens the new UI, sighs, and frowns.” If you’re wondering why they responded in that way, you can always ask follow-up questions to get them to unpack their thoughts. 

✍️ Looking to step up your note-taking game? Achieve faster, easier note-taking for UX research with these skillful tips, templates, and examples.

Bias-less research with User Interviews

It’s impossible to eliminate bias entirely in UX research—but you can identify, minimize, and work with bias for better (more ethical) outcomes. 

And a great place to start is with your recruitment audience. 

User Interviews’s Recruit is the fastest, easiest way to recruit high-quality participants for UX research. Our participant scoring system evaluates researcher feedback and past participation to bring you only the highest-quality participants—and we never make you pay for no shows or sessions that didn’t work out.

It’s free to get started. Sign up today or visit our pricing page for more information.