The State of User Research Report 2019

Editor's Note: We updated the title from "The State of User Research Report 2018" to "The State of User Research Report 2019" to reflect the year it was released and maintain consistency with later editions of the report.

Survey design and analysis by Erin May. Writing and graphics by Carrie Boyd.

User research is on the rise. More and more companies are putting user feedback and insights in the place the belong—front and center. This is a great thing, a trend we’re happy to be part of and help celebrate.

To kick off the year, we wanted to zoom out, and take stock of the state of user research as it stands right now. In our first annual State of User Research Report, we heard from 169 professionals who spend at least 10% of their workdays closely involved with user research. We probed into how they [perhaps you] spend their time, what they love, and don’t love, about practicing user research today, their hopes for tomorrow. The results have already been helpful for us internally, as we seek to build better products and experiences for research professionals, and we hope they’ll be enlightening for you as well.

Here are some highlights of our research, followed by more detailed analysis, and ending with an appendix containing charts galore.

TL;DR

1. More research please

75% of our respondents said their companies didn’t do enough research. The same percentage said they could do more to use the research they are doing to make smarter decisions. As the industry moves forward to conduct research more habitually, an important aspect of its success will be making sure teams are empowered to maximize the use of the research that’s happening.

2. Research helps teams understand customer needs and make better decisions

Our respondents’ favorite thing about research is the ability to make decisions with better evidence (40.7%). The top goals of their research programs overall? To understand customer need, rated at a 4.6 out of 5 in terms of importance, closely followed by validating early stage solutions, a 4.4 out of 5. We were happy to see understanding customer need come in #1.

3. Researchers think their programs are effective overall

On average, our respondents rated the effectiveness of their research at a 7.5 out of 10.  There’s a high correlation between how effective researchers view their efforts and how fulfilled they are in their roles. Those who reported they are more fulfilled at work (5 out of 7 or more) rated their effectiveness at a 7.9 while those who feel less fulfilled rate it at a 6.5. Which came first, fulfillment or effectiveness? We can’t say, but we dive into what makes teams feel effective and fulfilled below.

4. Validating early-stage solutions is a key use case for research

88.7% of respondents said they conducted research during the design and prototyping phase, once they already have something for users to look at or test. Nearly as many, 84.5% are conducting research before designing or prototyping anything. The best time to understand customer need—the top goal of research—is before designing anything. While the differences are very slight (88.7% versus 84.5%), we would hope to see pre-design, discovery research approach 100% over the coming years.

‍5. Researchers deploy many methods to answer questions

Our respondents typically use a mix of methods to conduct their research sessions, but two types rise above the rest. 95.6% of respondents conduct at least one user interview a month, and 91.1% conduct at least one moderated usability test. However large or small your team, anyone can get up and running with both of these methods with a small budget and limited amount of time.

6. Researchers most often rely on their own users as participants

While 30.2% of our respondents conducted more than half of their research with outside participants and 54.6% conducted more than half of their research with their own participants, 74.5%  used a mix of both kinds of participants. We always say, the best kind of research is the kind you’ll actually do, so taking advantage of the users right in front of you is a great way to do regular research. But, prospective users have a wealth of insights unbiased by familiarity with your product, and they’re easier to reach than ever.

7.  Research isn’t just for “researchers”

50.5% of our respondents said there were more people doing research at their company than there were dedicated researchers. On top of that, 35.7% of respondents said research wasn’t relegated to just one group or team. 35% of our respondents did not have research related titles, but are still doing lots of research. We love this topic as it’s a fun messy one, but in large, we are happy to see professionals across the spectrum participating in generating and using user insights!

8. Research budgets are a top frustration

The most popular frustration for our respondents was budget. 22.1% of respondents said their biggest frustration with research at work was not having enough budget or resources to get the job done. On top of that, 20.5% of researchers said their research budget was between $0-99 a month. You can do great research on a small budget, but if you want feedback outside your own user base and tools to collect and make sense out of the raw qualitative data, you won’t get far with $99 or less a month.

9.  Generally researchers are pretty fulfilled

When asked how fulfilled they felt at work—on average—our respondents rated their fulfillment at a 5.1 out of 7. A whopping 75.2% of our respondents rated their fulfillment at a 5 or higher. Factors like remote work, smaller companies, and prioritization of research correlated with stronger ratings here.

The long story

75% of user researchers say they don’t do enough research. On top of that, 76% of researchers aren’t satisfied with the way they use the research they do. We feel you researchers; we basically always want to do more research too. That’s why we’ve put together our first annual State of User Research Report. We wanted to shine a light on the frustrations, quirks, and habits of the user researchers we talk to every day. In this report, we’ll dive into our findings and walk you through how to get more research done right.

We surveyed 169 people who spend at least 10% of their professional time involved with user research, 65% of whom have research in their title. We created our survey and sent it out into the interwebs to gather feedback from people in 17 countries. 75.9% of our respondents were from the US. Our survey respondents ranged in age from 25-74, with 1,525+ combined years of experience. That means our respondents, if working continuously, would have been unearthing insights since shortly after the fall of the Roman Empire.

Why do we do UX research in the first place?

1. We like to be informed

Researchers’ favorite thing about research is the ability to make decisions based on better evidence (41%). The clearest business value of research is to build the right things, better—whether products, processes, or communications—and to continuously improve them. The only real way to do that consistently is with user research. User research helped us make the biggest business decision there is: what business should we build?

Not far behind is bringing the voice of the customer to the team (35.2%). UX, CX, “the experience economy”—all of these concepts are starting to find prominence and blur into one discipline centered around customer understanding, and, just as importantly, action based on that understanding. Bringing the voice of the customer to all teams is one of the most important steps in being a user centric organization. We love voice of customer programs, too, because they bring all sorts of data together into one profile. Passive feedback (NPS surveys, support tickets), active insights (user interviews, usability tests), and more come together to build deep understanding and empathy for customers over time. What’s better than that?

While only 17% ranked talking to people as their favorite part of the job—probably a good thing given the other options—anecdotally we hear all the time how much researchers love this aspect of the job personally. It’s important to love your work, and we think the opportunity to connect with people outside your bubble, whatever that might be, makes the work that much more rewarding.

2. We have specific goals we want to achieve

So if our respondents’ favorite thing about research is the ability to make better decisions based on evidence, what kinds of decisions are they hoping to make with the help of research? We asked our respondents what the main goals of their research program was. They rated each on a scale of 1-5, 1 being the least important to their goal and 5 being the most.

The top two goals of research outpaced the others. The most highly rated goal was  understanding customer need, which respondents rated at a 4.6. The second highest was validating early stage solutions, like prototypes and mockups, which was rated at a 4.4. This is no surprise, since the one of the main purposes of research at many organizations is to create better things that customers will actually want to use. With the emergence of concepts like design thinking, jobs to be done, and design sprints, research to understand what your customer truly needs is more popular than ever.

We also asked our respondents about the types of studies they are conducting. 57% of our respondents said that over 50% of their studies were unique or ad hoc, meaning they were created to answer a specific question, rather than being an ongoing or open-ended study. This means most research studies are created with a specific goal in mind, not to gather general insights. Effective research always needs a goal, though gathering general insights can be a worthy one when there’s a plan to use those insights.

We recommend augmenting your ad-hoc research with ongoing listening studies to keep up with what your user is feeling day-to-day. Making research a habit, rather than something you do when you need to find the answer to a question, can help your team get a head start on the research process.

Conducting smaller studies throughout your process is a good way to build a habitual research muscle;setting up a pipeline of continuous research is another good option. Studies have shown that you only need around 5 participants to gather meaningful insight from qualitative studies. So conducting a late-stage validation test, setting up a practice of ongoing listening, or just squeezing one more round of research into your plan can be beneficial, doable ways to consistent make smarter decisions and generate more user insights on any budget.

3. So how often does research actually help our respondents meet their goals?

Turns out, they were pretty confident about the effectiveness of their research. On average, they rated their effectiveness at a 7.5 out of 10.

When we broke down the data a bit more, we found the biggest difference is between people who are most fulfilled at work (5 out of 7 or more) and less fulfilled (4 out of 7 or less). Those who are more fulfilled give the effectiveness of their research a score of 7.9 compared to the less fulfilled group at 6.5. Correlation, not causation. Chicken/egg. We get it. But clearly there is a relationship between doing enough research to meet your team’s goals and being fulfilled at work, a huge measure of work happiness and predictor of retention.

In fact, the Gallup State of the American Workplace report found employees who were actively disengaged from their jobs were twice as likely to look for a new job than actively engaged employees. Whether because of budget, bureaucracy, or other issues, companies that want to keep their researching employees happy should make sure they’re set up to do the volume of research they need to reach their goals—understanding customers to make better decisions chief among them.

Additionally, we found that managers (8) view research efforts as a bit more effective than individual contributors (7.4). When we divided this data up by company type, we found that agencies ranked the highest among company types in thinking their research is effective (8), where enterprise companies ranked the lowest (7). There is no difference between folks with or without research in their title.

When do we do the UX research?

Most teams conduct research during the design and prototyping phase, which can be incredibly important to make sure you’re on the right track toward building the right solution. But, it’s even more important to start your research efforts before you imagine solutions—almost as many of you do that, so we aren’t freaking out about these results. Still, Nielsen Norman Group has some great rules of thumb on when to do research that are worth sharing here:

One of the questions we get the most is, “When should I do user research on my project?” There are three different answers:

Do user research at whatever stage you’re in right now. The earlier the research, the more impact the findings will have on your product, and by definition, the earliest you can do something on your current project (absent a time machine) is today.

Do user research at all the stages. As we show below, there’s something useful to learn in every single stage of any reasonable project plan, and each research step will increase the value of your product by more than the cost of the research.

Do most user research early in the project (when it’ll have the most impact), but conserve some budget for a smaller amount of supplementary research later in the project. This advice applies in the common case that you can’t get budget for all the research steps that would be useful.

If you’re reading this and you’re new to UX research, you can learn more about methods to use at each stage in our UX Research Field Guide.

• Discovery Methods
• Validation and Testing Methods
• Ongoing Listening and Passive Feedback Methods
  

What kinds of studies are researchers focused on?

There are tons of different ways to do research. It can be as informal as a conversation or as rigid as a survey. The type of research you do can will inevitably  from project to project, but many researchers have go-to methods for their needs.  While using lots of the methods isn’t an end goal in itself—if one or two methods does the job, it does the job—we love championing lesser used methods like diary studies and card sorts as very valuable for certain kinds of user questions. It’s good to have a rich toolkit, even if you don’t use every tool all the time.

‍
We found that the majority (95.6%) of respondents conducted at least one user or generative interview a month, making it the most popular way people in our study conducted research. We’re obviously pretty excited about that, since it’s our company name and everything. Interviews were closely followed by moderated usability tests, which 91.1% of respondents said they conducted monthly.

Of the 154 people that input an answer for each type of research, 18 super researchers said they conducted at least one study of each type on average each month. Among this group, their average monthly budget for research was $37,307. The average monthly budget for research across the board was $11,192. Not surprisingly, larger budgets mean more research, and more research variety.

‍
We were happy to see our respondents weren’t just relying on proactive, qualitative methods to generate user insights. There’s a world of insight on both the quantitative and passive feedback side of things, whether or not you or your team is responsible for creating/intercepting that data. A huge part of building an effective, sustainable research program is taking advantage of the research you already have, of all kinds, to build the best possible view or your users and customers. While 75% of you rely on web analytics, the numbers drop to 56% for sales and support driven feedback, and 55% for NPS or customer feedback metrics such as CSAT. We hope to see these numbers climb in the coming years. Qual and quant are friends, not enemies.

Who is participating in UX research studies?

When choosing who to conduct your research with, you typically have two options. You can either recruit from your own users, who are at least somewhat familiar with your product, or you can recruit from outside participants, who don’t use your product. Both have their pros and cons. Recruiting your own participants can help you decide how to build on top of existing features, improve customer retention, or even find bugs within your existing product. You also likely already have their contact info and can reach out to them directly. But recruiting strictly from your own users may not be the best choice if you’re testing out a new product idea, or want to test constantly without reaching out to the same core users repeatedly.

Recruiting outside users is great for testing out new ideas, determining product/market fit, and seeing how completely new users would approach your product or problem. Outside users do come with the hurdle of having to recruit them to your study (but no worries, we’ve got you covered with three free participants in your first study 😉).

We found that 30.2% of our respondents conduct more than half  of their research with outside participants, while 54.6% conduct more than half of their research with their own participants. 75%  of our respondents use a mix of both kinds of participants. We found whether conducting research with outside or their own participants, our respondents used the same methods and conducted research at similar frequencies.

What tools are you using to make user research faster, easier?

Our respondents use a variety of tools to make their research faster, easier, and overall less of a pain in the butt. We asked which tools they like for the most common research functions, like taking notes and recruiting.

Typically, to take notes, our respondents used word processors/note taking apps, video recordings, and/or transcriptions. The most common combination was a word processor/note taking app and a video recording, at 36.08%. This makes sense as transcription usually involves extra cost. However, 26.58% of our respondents used all three of these things for keeping track of their research notes.

Most people are using Excel and Google sheets to gather their feedback, with Airtable as a distant second. It will be interesting to see if this gap closes in the coming years, if other Airtable-like competitors chip away at the Google Sheet or Airtable market share, or if more people make the switch to other types of solutions, like productboard.

Regardless, our true second place belongs to the “other” methods, which 23.1% of respondents said they used alongside the 10 provided choices. Our other responses were usually physical solutions, like sticky notes and whiteboards. People also seemed to love more lightweight tools like Google Docs.
‍

User Interviews was the top way to recruit both outside and researchers’ own users among our respondents. No big deal. Maybe it’s because we offer three free participant credits when you sign up for Recruit, or because we make it easy to manage studies with your own participants. Maybe it’s because we sent this survey to our entire email list :). Either way, it feels good to be on top.

When it came to recruiting their own participants through other methods, our respondents can get pretty scrappy. Following User Interviews, the top three methods for recruiting your own participants were Mailchimp/email/marketing CRM, Calendly, and SurveyMonkey. Most of our respondents (69.4%) used multiple tools to create their own recruiting solution, often combining three or more tools to get the job done.

Recruiting users outside of your own user base was where most respondents decided to bring in the big guns, that is, a software that helps you manage the recruitment. The only options that users selected they had to fully manage themselves were social media (19.6%) and Craigslist (10.9%). Of the respondents who used social media, 74.1% combined it with another method, usually a software. For those using Craigslist, this number was even higher—93.4% of respondents combined it with another method.

Of the software, User Interviews was the crowd favorite, with usage by 52.2% of respondents. Other favorites included UserTesting (24.6%), Qualtrics (10.9%), and dscout (9.4%).

People are typically using presentations and shareable documents to catalog and store their findings. This comes as no surprise, since 62.4% of our respondents said they often hold a meeting to share their findings. A presentation makes it easy to walk stakeholders through learnings and get them on board with implementing changes based on your research.

There is, however, a bit of an art to presenting your research findings so people will enjoy, understand, and get excited about them. If you go too data-heavy, some stakeholders may tune out, assuming if they really need an answer they can come to you later. Going too far in the other direction, towards antecedentes or data that doesn’t have clear implications, can create the idea that your research really isn’t that important in the first place.

Our top tip for creating interesting presentations people will actually listen to? Use a narrative storytelling structure to draw people into your findings and to give them a clear place to go when they leave the meeting. You can make your story more interesting (and less biased) by including actual quotes, videos, or photos from your research sessions. This allows your stakeholders to hear about what the users are experiencing straight from the user themselves, without having to sit in on your sessions.

Our respondents had a variety of ways to collect survey data, but the most popular remains Google Forms, garnering 46.4% of our responses. Google Forms is free, but still has great features like skip logic. It was followed by SurveyMonkey, which offers both a free version and a powerful paid version that allows you to create advanced surveys and data filters. We reviewed a full list of survey tools in our Field Guide. Check it out if you’re on the hunt for a new one or just want to see what’s out there.  

Remote user testing sessions are increasingly popular. Among projects launched on the User Interviews platform, online or over the phone sessions take up on average 60.25% of sessions. In our survey, people preferred robust video chatting platforms to research-centric ones. The plurality of our respondents, 42.4%, said they conducted their sessions over Zoom. Second (39.7%) and third (23.2%) place were also taken up by video-chatting platforms, Google Hangouts and Skype, respectively. Many of the “other” responses (25.2%) were for other video-chatting platforms, like GoToMeeting and BlueJeans.

Where does research sit in organizational structures?

1. Research isn’t only for researchers

We asked our respondents how many dedicated researchers they had on their team and how many people regularly conducted research. 50.5% of them said they had more people regularly conducting research than dedicated researchers.

This means people aren’t letting the lack of “researcher” in their title hold them back from getting direct learnings from users themselves. Of course in some cases this is by necessity as not all organizations have a dedicated UX researcher at all. We are huge fans of all hands research, but it is also important to have some baseline standards and processes. Creating consistency across studies and preventing personal agendas from playing a role in how we gather research is especially important to teams that work this way. Building a continuous and prosperous research practice can be a lot of work, but getting more people involved in research can help the process to be smooth, fun, and efficient.

2. Research lives in multiple departments

When we asked our respondents where research sat within their organization, we got a wide variety of responses. For some teams, research is part of a design or product team. For some, it is spread across departments and job titles, or acts as a central research for everyone to access. Within our sample, research is most prevalent in design departments, with 44.6% of respondents saying research is part of their design team. We’re interested to see how this evolves over the coming years, as research grows in scope and adoption across teams, especially since the second most popular place for research was distributed across different pods/groups/teams (35.7%).

What can make research difficult?

1. Researchers would like to see research used more

75.2% of researchers are not fully satisfied with the way they use the research they do gather. Interestingly the same percentage, 75%, of our respondents wanted to do more research. As organizations begin doing more research (yay!), it’s as important that they find ways to make use of that research: analyze it, share it, get it seen and heard, and make it discoverable for as long as it useful to anyone it might be useful to. Easy ;)

2. Stakeholders don’t always know how to access research

Most stakeholders of research within an organization don’t know how to access research on their own. This puts research professionals in a position of constant sharing and finding, versus enabling and empowering, teaching folks to fish. We think there’s a big opportunity here, whether through formal Research Ops functions, or through someone taking up the task: research is more valuable and sustainable when people who will benefit from it can access it on-demand. This is especially true for discovery research, which can be more evergreen.

The 33 people who said 76%+ of their stakeholders knew where to find research came from companies of all different sizes, were at many different levels of their respective companies, and backgrounds. As a whole, they rated the effectiveness of their research at an 8. 72% said they hold meetings to discuss their findings (compared to average of 62.4%), 27% said they sometimes did, none said they never held meetings. The same percentages held for creating reports of their findings.

3. There’s not enough money for research

The average monthly budget for research is $11,192. The most common frustration our researchers had with their jobs was that they were not provided with enough budget or resources to get the job done (22.1%).  This was followed by a frustration with top-level buy in (20.8%).

Of the people we studied, the plurality said they had budgets of $0-99 a month for research (20.5%). This means most teams are provided with a small or non-existent budget to work with. Their companies ranged in size from 1-10k+ people. In the future, we will hopefully see an upturn in the budget allocated for research. In the meantime, our respondents are staying scrappy with a variety of free or low-cost tools and methods.

4. Most research is planned last-minute

Only 7.3% of our respondents said they had been planning their research session for months, and only 2.4% said they had started recruiting for their study a month ahead. The most common lead time for both planning and recruiting was just 1-2 weeks. This means research is moving quickly, going from idea to execution in the blink of an eye. Not a bad thing in many cases—moving fast and keeping research agile is good—but as research scales, planning a research roadmap alongside the product and org-wide “insights needed for various things we’re doing” roadmap is important too.

How do people find their way to on-the-job user research?

1. Researchers can come from any department

Though everyone in our study spent more than 10% of their time conducting research, they certainly weren’t all full-time researchers. In fact, only 34.9% of our respondents were completely dedicated to research. This is great for the practice of research as a whole, since it means that different roles, departments, and teams are embracing the power of research.

Regardless of who does research at your organization, the more people involved the better! At User Interviews, we’ve established a practice in which everyone is invited to conduct research. Turns out, we all have a pretty good time putting together studies, talking to our users, and bringing our findings back to the team. Though we’re still working on establishing the best cadence and practices as a growing team, we all can agree that having everyone take part in  research is a huge benefit.

2. Researchers come from a variety of backgrounds

There are many paths to get to a career in user research. You can start off as an account manager, writer, teacher, scientist, or really anything else. Most paths do have one thing in common though: an interest in understanding people.

As you can see, our respondents studied a wide variety of subjects as undergraduates. Everything from Art History to Industrial Design. And the 55.7% of our respondents with graduate degrees didn’t stick to research related subjects either.  

A majority of our respondents held graduate degrees. These degrees did not correlate with a higher salary or more fulfillment (4.9 out of 7 for graduate degrees and 5.6 out of 7 for undergraduate only).

Moral of the story? There are many different ways to get to a user research focused career. Jaclyn Perrone said it best on an episode of our podcast, Awkward Silences, “Do what you like and there will be things to do.”

3. Researchers like their jobs

Researchers are a lot of things—scrappy, collaborative, empathetic, curious—the list goes on. But do they feel fulfilled at work? Most people don’t, but, then again, our respondents aren’t most people :). According to the Gallup State of the Global Workplace report, only 15% of employees worldwide are actively engaged in their jobs. Gallup defines engaged as, “Employees who are highly involved in and enthusiastic about their work and workplace.”

We asked a slightly different question, how “fulfilled” our respondents felt at work. 75.2% of our respondents rated their fulfillment at a 5 out of 7 or above. When compared to Gallup’s overall engagement of 15%, that’s a pretty big leap. So talking to people, finding answers to important product questions, and bringing actionable results to our coworkers turns out to be pretty rewarding, especially when compared to the overall status quo.

Because we’re always interested in making research a more enjoyable process, we wondered if there’s anything that would make our respondents even more fulfilled at work. Our average rating for fulfillment was a 5.1 out of 7.

Remote work for the win!

People who rarely or never work remotely rated their fulfillment at a 4.9, while people who work remotely at least one day a week rated their fulfillment at a 5.3. We may be a little biased, but the numbers don’t lie.

Smaller is better.

People who work at a company with less than 200 employees rated their fulfillment at a 5.3, while people working at companies with more than 1,000 employees rated their fulfillment at a 4.9.

Prioritizing research reaps rewards.

We took a look at how our respondents described how their organizations thought about research. Meta right? People who thought their organizations rated the importance of research as an 8 out of 10 or higher also rated their fulfillment at a 5.6 out to 7. People who thought their organizations rated the importance of research as a 7 out of 10 or lower rated their fulfillment at a 4.4 out of 7.

It gets better with experience

Though our respondents with less experience, 0-4 years, rated their fulfillment at a 4.8, our respondents with more experience rated their fulfillment much higher. Our respondents with 5-10 years of experience rated their fulfillment at a 5.2, 10-20 years of experience rated fulfillment at a 5.3, and 21+ years rated their fulfillment at a 5.4.

How do researchers stay in the industry know?

Researchers like to read. Whether it’s online or actually curling up with a good book, our respondents seem to love staying up to date with the power of the written word. When we asked them how they stay in the loop with user research, the top three choices were reading based:Medium posts, blog posts, and books.

This was closely followed by talking to actual people, at things like meetups and events, over Slack channels, at conferences, or over webinars. This makes sense, as one of our respondents’ favorite things about doing research for work is talking to people.

Surprisingly, podcasts, trailed behind at #10. Maybe there’s just a gap in the market? Maybe you haven’t found the right UXR podcast for you? Maybe you’ve somehow missed our pod, Awkward Silences, which is chock-full of incredibly interesting conversations with people from all over the UXR world? Maybe. We’re here for you if that’s the case.

If you’re looking to broaden your UXR news horizons, check out this list of communities and resources.

Closing thoughts

You made it to the end. Thank you! Overall, we declare the state of the user research union strong. You are fulfilled, but hungry for more. More research, more impact from that research. More learning and conversation. More money for crying out loud.

We predict a year from now more organizations will be doing more research with larger budgets. A possible short-term challenge of this trend may be organizations struggling to take advantage of the insights, the raw qualitative data they are bringing into their organizations. We expect to see accessibility continue to grow as a focus for product teams, and that means more explicitly designing research studies to address it as well.

What do you hope to see? Let us know on Twitter @userinterviews. For press inquires, email erin@userinterviews.com

Appendix

Charts created with Venngage.