Explore our full report in more depth + subscribe to our newsletter to get free access to the full dataset of tools and takeaways across AI in UX research.
We knew AI was going to make its mark on user experience research and design when we launched our first AI in UX Research Report last year, but boy has it ever entrenched itself in the UX (and let’s be honest, cultural) zeitgeist right before our eyes in the past year.
So how has AI impacted UX Research and related disciplines within Design and Product in 2024? We analyzed thousands of collective datapoints across more than 200 survey participants, compared this year’s findings to our 2023 report, and asked new, pointed questions about the technology’s evolving role in the research landscape today and beyond.
Here, we offer a few of the key findings and takeaways from our analysis, which you can read in full using by downloading the full report and dataset.
Since our last report, the landscape of artificial intelligence has evolved dramatically. Sky-high valuations, new applications and use cases, and language model improvements meant that we needed to expand the way we categorized and investigated this technology.
We asked UX professionals about their use of specific kinds of AI technology, looking at three categories:
When ranking which they use most often, over 60% gave first place to all-purpose, followed by features within other tools, and finally research-specific.
All–purpose tools are often free, are widely available, and can be adapted to many uses. Despite this, nearly every research-specific tool in the list we provided (and there were over 30) had at least one active user. We expect this category to grow, especially as capabilities improve.
When our sample was asked to list their favorite uses for AI technology—regardless of the type—the majority of responses clustered around generation, ideation, and refinement.
UX professionals love to use AI to combat “blank page paralysis” at the start of projects, asking it to create briefs, scope timelines, and generate questions for surveys, screeners, and interviews. They also love AI toward the end of projects, for its power to distill, summarize, and sharpen their writing, whether that’s for reports, slide decks, or repositories.
These creative and collaborative use cases were reported by over 70% of our sample, which included folks in Product, Design, and Research. While you can take these findings as an indication that AI is helping many UX pros get better research done faster and insights shared more clearly to stakeholders and partners, the flip side is that we also see there’s still a lot of manual (read: human) review that goes into ensuring the efficacy of those outputs.
When asked how, if at all, they are using AI tools and technology across the phases of research (from design to insights repository), 90% deploy it during analysis and synthesis. The most popular tasks in this phase include summarizing notes, transcripts, and open-ended data, as well as analyzing that data for trends, themes, and clusters.
AI’s ability to sift, scan, sort, and sum up open-ended data — including voice, video, and text — is a boon for UX professionals, who must regularly choose between speed and quality when deploying mixed methods or qualitative methods. This, our sample reports, often lets them focus more of their time on the strategic implications of their findings, something that can often feel rushed (or missed entirely).
On the other end, 46% report not currently using AI at all for storage and repository needs. This was curious despite the development in AI tools whose purpose is exactly that (subscribe to our newsletter to learn more about them in our upcoming UX Tools Map).
Our sample of Product, Design, and Research professionals love the speed of AI, with nearly half (48%) mentioning it as a benefit of its use. Other benefits infer speed, including automating tasks (mentioned by 30%) and efficiency/productivity gains (another 37%).
However, UX pros also mentioned the need to review, revise, and sometimes retry AI technology to get things right. AI, they report, lacks nuance and context (29%), and is sometimes flat-out inaccurate (27%). As a result, 16% report the primary drawback is the time it takes to review both inputs—to get the AI to execute correctly—and outputs.
Future research should investigate the exact time-saving benefits given this tradeoff, but our data indicates that although AI helps speed and scale up many parts of a UX pros’ workflow, many are also building in padding to make sure it’s getting it (even close to) right.
Standalone all-purpose AI tools were far and away the top choice for our sample of UX professionals, whether they were in Product, Design, or Research. These tools' wide availability, (relative) ease of use, and freemium pricing models could explain their ranking.
As to which standalone AI tool our sample reported as their favorite, OpenAI’s ChatGPT was most utilized (73.6% mentioned it), followed by Anthropic’s Claude (12.6%), and then Microsoft’s Copilot (10.2%). Many participants reported using more than one, explaining that some models were better for one use case (e.g., writing) compared to another (analyzing).
“ChatGPT - Used for generating written content, summarizing complex papers, ideation, and answering research queries. It's especially helpful for brainstorming, outlining, and getting a second opinion on literature reviews.”
New this year was a list of research-specific AI tools and platforms. Although most of our sample (54.6%) reported not using any of this type of tools for their research, most of the tools had at least one participant using it.
Participants’ favorite use cases of AI for UX research organized around two groups: creation/refinement and organizing/categorizing. In the first group were applications like Question Generation/Refinement (40%), Planning/Ideation (24%), and Writing/Proofing (23%). For some, our findings show that AI helps start thoughts, iterate ideas, and home in on best options for parts of research.
In the second group were applications like Thematizing (26%) and Summaries (23%). Here, AI’s ability to digest, assess, and organize data (often open-ended) is a boon to research project work.
Knowing how UX professionals use AI tools and technology is only part of the story. When they use it is also instructive—alluding to the perceived strengths and weaknesses of AI.
We surveyed our participants about six phases of UX research and how they use it—if at all—within each phase. The phases were:
Within each phase, participants were asked to select any and all tasks they currently use AI tools and technology to support. If they did not use AI for that phase, they indicated so.
Although every phase had at least some reported AI usage, some were more popular than others. In particular, the Design (84% using) and Analysis and Synthesis (90% using) phases.
Within Design specifically, results showed a preference for AI supporting ideation and creation activities, such as interview/discussion guides (65%), survey or task questions (60%), and research plans/objectives (55%). Within Analysis and Synthesis, summarizing open-ended data such as transcripts or notes (73%) and thematic or clustering analysis (52%) were most frequently reported.
“Using AI tools for research is a game-changer! It saves time, reduces errors, and helps find valuable insights. AI quickly scans through vast amounts of data, pinpointing relevant info and connections I might miss.”
In both of these phases, UX professionals across Product, Design, and Research value AI’s ability to speed up processes, whether that’s generation in pre-fieldwork phases or the sometimes-tedious work of data management and processing. These benefits surface again when this group is asked to share the benefits of AI for their UX research practice in general.
On the other side of AI usage across research phases were Recruitment (40% not using) and Storage and Repository (46% not using).
As might be expected from other parts of this report, when participants were asked to describe the primary benefits of AI technology for UX research specifically, nearly half (49%) mention some form of speed and/or time savings. Another 38% reported efficiency and productivity gains, while another 30% the automation of tedious work.
For some, speed was a value in and of itself. For others, the speed, efficiency, and productivity increases had knock-on effects such as freeing up research bandwidth for tasks where having a human matters: rolling insights into strategic recommendations, connecting with stakeholders, and generally creating more stickiness for UX research.
“AI has helped me articulate themes and messaging that is otherwise difficult to develop or refine. AI can help do the due diligence legwork critical in the prep phase prior to conducting interviews, like stakeholder mapping and hypotheses.”
This quote hints at what many of these same participants would describe when asked to articulate the drawbacks of AI’s use for UX research. Compared to the benefits (where we found a total of 8 discrete categories), participants surfaced more kinds of drawbacks (a total of 14 categories). When reading the drawbacks, many read like an amendment to the benefits.
As such, the top drawback reported was that AI lacks context, often producing unnatural, vague, or confusing output (29%). Another 27% mentioned concerns about the (often invisible) processes powering models, reporting that AI was unreliable or untrustworthy. Another 27% believed AI can be flat-out inaccurate.
Still, other participants wrote about the time-consuming process of reviewing AI prompts and outputs for clarity; some described the lack of evidence, citations, and window into the “thought process” of AI models as limiting their use.
Taken together, this suggests a bind. UX professionals can be more efficient, productive, and speedy with the help of AI…if they also build in time to review, revise, and sometimes re-do its work.
“I'm still not sold on the reliability of the analysis that AI can provide. It feels like I need more time to re-translate what it suggests.”
More than a few Product, Design and Research folks in our sample described instances of overreliance by UX professionals and an overestimation of its ability to “do” effective user experience research. Both are harmful, they say, to the practice of research.
“AI outputs cannot be accepted as fact or truth, they MUST be vetted by a real researcher to ensure the outputs are true and based in fact. People who do not have a researcher brain may take outputs and run with them when they are not correct.”
It might be—once again—up to the UX community to both educate themselves on the ethical and rigorous use of AI tools and technology, and to educate stakeholders, partners, and leadership on the ways AI can help—and hinder—a UX research workflow. As with research democratization, which we still believe is a net benefit, guardrails should be in-place, education prioritized, and systems developed.
The landscape of artificial technology will surely change one year from now, but the picture of its impact, uses, and limitations within UX research is coming into focus.
The survey was created using SurveyMonkey and included 19 questions (4 screening, 3 open-ended, and 12 closed-ended). It was promoted via social media (e.g., LinkedIn, Slack), our Fresh Views newsletter, targeted emails, in-app messaging within the User Interviews platform, and partner (e.g., other UX research tools) distribution.
From Oct. 2 to Oct. 15, 1063 responses were collected. 812 participants were eliminated because they did not qualify for inclusion based on their responses. An additional 44 participants were eliminated for not completing at least two-thirds of the survey. Finally, one person was removed for suspicious responses (i.e., marking every “Other” response option with the text “Good”). The total number of qualified responses was 206.
The sample was primarily from “UX/Product Design” (44.7%) and “Research” (27.2%) departments; another 7.3% were from “Product Management.” Other departments represented include “Insights or Innovation” and “Marketing” (both 3.9%).
Finally, over half (52.9%) of the sample reported their current job involved conducting or supporting research between 75-100% of the time. Another 37.4% reported it was between 25-74% of their time. Only 9.7% reported doing so was 10% or less of their job.
Taken together, this sample is full of UX Designers, Researchers, and Product folks who primarily spend more than 25% of their time conducting or supporting research projects.
Thanks to Maria Kamynina for data analysis, Nick Lioudis for copy, Holly Holden for design, and all the participants who graciously shared their experiences with AI.
Check out our latest State of User Research Report to understand how the landscape has evolved since 2019.
