Unmoderated User Research Isn’t What It Used To Be

In the past, if you asked a user researcher, “when should I do moderated vs. unmoderated research?”, you would’ve gotten a very straightforward response. 

“Use unmoderated research for short, low-stakes studies where you need a lot of testers but aren’t terribly concerned with who those testers are,” the researcher would have replied. “Reserve more in-depth, qualitative research for moderated studies.” 

This traditional view on unmoderated research made sense, because older testing tools weren’t optimized for more in-depth studies. Without a moderator to guide them through the task, participants were quick to drop off. 

But unmoderated has evolved. 

✨ Our modern take on unmoderated research: When you have access to a reliably high-quality participant pool, you can run longer, more in-depth unmoderated studies with lower drop-off rates. 

Let’s discuss:

• Why unmoderated research isn’t what it used to be, and
• What this means for UX researchers

3 reasons unmoderated methods have become a good option for qualitative research

Unmoderated research is no longer limited to answering a fixed, focused, straightforward set of “what” questions. With the right set of tools, you can use unmoderated to answer more dynamic, qualitative “why” questions while still ensuring a high level of participant engagement.

Three major shifts in the UX research industry have driven this change:

1. Unmoderated tooling has improved. 

Traditionally, unmoderated was limited because, as Kathryn Whitenton of Nielsen Norman Group explains:

“Early-prototype testing is difficult without a moderator to explain and help participants recover from errors or limitations of the prototype. Without a moderator, participants tend to be less engaged and behave less realistically in tasks that depend on imagination, decision making, or emotional responses.”

Today, though, unmoderated tools are easier to use, with more sophisticated features and a wider range of integrations. These developments have allowed researchers to attract more engaged participants, provide better guidance and instructions through complex tasks, and automate time-consuming tasks. 

For example, advanced video conferencing tools have allowed unmoderated studies to be almost as good as bringing someone into your lab: you can see the expression on their face and hear them talk through their behaviors and thought processes.

Plus, with a wider variety of tools on the market, researchers are better equipped to critically evaluate which tools are the best fit for their needs, and ultimately choose best-in-class tools that set them up for success. 

🧙✨ Psst—the venn diagram above stops at testing tools that were released in 2019. The unmoderated testing tool landscape has grown even more since then. Explore the 2022 UX Research Tools Map.

2. Platforms like User Interviews are providing more (and better) options for recruiting targeted, high-quality participants. 

Better, more qualified participants are more likely to follow through with complex unmoderated tests.

As recruiting and panel management platforms evolve, it’s becoming easier to find these engaged, reliable, high-quality participants for unmoderated studies. Some participants, like those from User Interviews’s panel, are better prepared to participate in longer, more in-depth studies than others. 

In fact, User Interviews is the fastest and easiest way to recruit participants for UX research. Whether you use Recruit to source from our growing pool of more than 2.4 million participants or Research Hub to bring along your own panel, we streamline every aspect of participant recruitment and panel management. With a growing suite of integrations and a custom API, we’re compatible with any testing tool or research method.

3. Research teams are facing greater demand with less bandwidth. 

User research is growing. In fact, according to UserZoom’s 2022 State of UX Report, 70% of respondents reported that the demand for UX research has increased in the past year, but almost half said that budgets hadn’t changed.

And according to the 2022 State of User Research Report, one-fifth of respondents said they were dissatisfied with their budget and resources for research.

With a greater need for research and an ever-narrowing budget, researchers need strategies for scaling their research practice while avoiding burnout and keeping budgets to a minimum. 

The modern approach to unmoderated studies offer researchers a way to collect in-depth, qualitative data without the same strain on their time and resources that a moderated study would require.

(Of course, unmoderated research can only get you so far—you also need a strong self-care routine and a recession-proofed user research plan.)

The implications for UX researchers

Unmoderated usability tests were the 4th most commonly used research method (35%) among researchers who responded to the 2022 State of User Research Report. Researchers love unmoderated studies because they can be run relatively quickly, cheaply, and with many users simultaneously—a massive boon in 2023’s economically-uncertain market.

With unmoderated research becoming increasingly available for a wider variety of methods, what will this mean for unmoderated usage among UX researchers?

The ability to run in-depth, qualitative unmoderated studies will provide researchers with: 

👀 The ability to gain more nuanced insights

Using a modern approach to unmoderated qualitative research can help you collect more in-depth data on a more frequent basis. Plus, pairing these qualitative studies with quantitative studies can help you gain a complete and nuanced understanding of your users.

As Matt Elbert, Senior Design Manager at Movista says:

“We do both types of usability testing: qualitative and quantitative. I think it's really important to have a mix of both. Quantitative testing gives us hard numbers, and those metrics are key in making data-driven design decisions. On the other hand, qualitative testing is incredibly useful because you get the voice of the user, and they're not just a "metric".

💰 The ability to do more on a tighter budget and timeline

Jeff Sauro of MeasuringU was right when he said: 

“One of the biggest barriers to conducting usability testing is the cost and time involved in testing.”

Unmoderated research was already a quicker option than moderated—but better tools, higher-quality participants, and smoother workflows mean that you can run unmoderated studies more efficiently than ever before. 

You can find other strategies for doing more research with less budget in “5 Ways to Recession-proof Your User Research Plan,” by Rachell Lee, including:

• Making smarter use of the tools, team, and resources you already have
• Insisting on participant quality, both from panels like User Interviews and your own customer panel
• Providing ‘just-right’ incentives for participants, and distributing them in an efficient manner
• Enabling other teams to conduct their own research (with proper guardrails and guidance, of course!)

🧰 The need for a specialized testing tool stack

Improved unmoderated tools have raised the bar for companies across industries. 

That means that all-in-one tools like UserTesting aren’t going to cut it for modern UX research teams. These types of “one-stop-shop” tools typically have weaker features, limited scalability, and higher costs than an integrated, best-in-class tool stack. 

Whether you need powerful tools to support enterprise-level user research or testing tools that are flexible and easy-to-use for smaller research projects, the flexibility of a layered tool stack is key to maximize the many benefits of unmoderated research.

👉 Compare the pros and cons of all-in-one tools vs. integrated tech stacks. 

Better unmoderated research starts here

Did you know that User Interviews supports more than moderated research?

Our smart workflows and upgraded features mean that User Interviews is built to adapt to whatever your study requires. For unmoderated research, our users’ top 5 favorite features for unmoderated research include:

• Flexible audience targeting, screening and verification
• “Smart flow,” a simple and adaptable participant experience
• Integrations with your favorite survey and testing tools
• Funnel views and bulk actions for easy progress tracking
• Re-recruiting for follow-up studies and mixed methods support

🔥 Start recruiting for usability tests, surveys, and other unmoderated methods today.