Never Ask Users If They Like Your Idea; Do This Instead

“If I had asked people what they wanted, they would have said faster horses.” - Henry Ford

If you conduct user research, I bet you've heard this one too many times to count. Sometimes the person asking the question intends to discount user research altogether, and other times they really want to know how you will get value from talking to users. If the users themselves often don't know what they really want, what should you ask them?

After conducting hundreds of user interviews, I'm beyond convinced that we can get tremendous value from talking to users at every stage of a project. I encounter the most skepticism around talking to users before we have a concept or prototype to get feedback on, but that's my favorite kind of research to do. Here's why:

I've seen time and again that the best products solve real problems that real people face. When you are asking users for feedback on an idea, you are testing how well the idea solves the problem that you set out to solve. You typically screen participants to talk with people who have that problem. This design research is critical and must be done, and by now a good amount of the tech industry, particularly in the user experience field, believes in its value.

But usability testing is not enough to drive the high growth that most of us are looking for. That's because when you focus on this kind of research, you don't often get a chance to see if you are solving the most valuable problems. High growth comes when you find unmet needs and design new solutions to those problems. I find that most organizations are still not applying the latest scientific learnings to the way that they do this earlier stage generative or market research, if they do it all.

Too often, this early stage research results in something like this recent story:

"We have a good problem," she told me. "Everybody wants what we are going to build."

"That's great," I said, encouraging her to tell me more. "How do you know?"

"Everyone we talk to tells us they like our idea," she told me. "Now we just need to decide what our MVP is. How do we figure that out?"

She had come to me for help with design research, believing that she had already validated her idea. But her words made me cautious, so I treaded carefully. "Great question," I say. "Do you have an idea of which user pains will be easiest to solve?"

"They all have the same pains," she tells me.

I dig deeper. "Do they all feel the pain equally? Do some of them have specific needs that will make it harder for them to adopt your solution?"

"I'm not sure," she says.

So we make a plan to find out.

Here are the steps I used to make a plan with her. I use this same framework whenever I'm making a new user interview plan. To illustrate the steps, because my business is helping people do better product research, I'll share the plan I've made to interview users about how they interview users. Bonus: meta is fun.

1. Define and prioritize research goals

You always want to start with why. Research is usually done to inform decisions that have to be made. What are those decisions? Which ones are riskiest? Which ones need to be made now, and which ones can be put off until later? Which ones will be expensive to change if you later learn new information?

In my case, I'm broadly looking to better understand what's challenging to people who commission or conduct user interviews, so that I can tailor my services to solve their problems. I would like to find and convert more research clients for my business. When visitors are first learning about my services, they will form opinions that may be hard to change. Thus, my top priority is to learn about how they find and choose researchers. Specifically, I'd like this research to inform the creation of a compelling service detail page (and validate the need for one).

Once you have a clear idea of the research goals and which decisions the research will inform, you can get more specific about your research plan.

2. List specific questions you want answered

At this point, you want to brainstorm lots and lots of questions. This is sometimes called question-storming. You likely won't get answers to all of them, but this exercise helps you uncover the big questions and get in the right frame of mind for focusing on the study participants.

Think about what evidence you wish you had in order to make those decisions confidently. What do you want to know about your users? What is the context around the decisions they make and the actions they take? What are the differences among them? Which ones make a bigger difference on how likely they are to become your customer? List as many questions as you can think of, and then prioritize them so that you can stay on the highest value content in your interviews.

In my case, I created a list of over 20 questions, and then prioritized them to focus on these three:

1. Who commissions research?
2. What job are they hiring the research to do?
3. How do they know if the research is successful?

Now you'll be better able to steer each interview to the most valuable information. Next, you'll need to decide who to recruit.

3. Determine who to talk with

How do you know when you’ve talked to enough people? My rule of thumb is that when you start to see patterns or hear the same things again, you're ready to make decisions based on the research. But how long does that take? If you've planned your research well, it could be as few as 3 to 5 people. On the other hand, if you've talked to 10 people and you're still not seeing patterns, then you should take a look at how you are deciding who to talk to. Think about what differences in your potential users might make a difference in the way they use your product or service. Focus in on the differences that are likely to affect the decision area you are working in for this research effort. The more targeted you can get, the better.

If you're reading this and thinking, but my product needs to appeal to everyone, keep in mind that you'll need to get from here to there. In today's world of personalized digital experiences, products win by solving a particular problem for a targeted set of users who have similar unmet needs. Only after a company has gained traction can they begin to scale to more and more user segments until, if they are lucky, we may one day look on them as something that everybody wants.

In my case, I hypothesize that the significant differences will be around their existing expertise and resources. I'm most interested in helping people level up their research skills and encouraging them to talk with customers themselves, so I want to set up a screener that finds people who want to get started with user research or to get more value out of their research efforts.

Once you've got your target participant profile, you can set up your research recruitment. While the recruitment is underway, move on to crafting your interview guide.

4. Build your interview guide part 1: don't ask the questions directly

Direct questions often get answers that are subject to cognitive bias. The respondent will answer with their idealized scenario, rather than their reality. If you aren't sure what I'm talking about, just ask someone who wants to get healthier how likely they are to go to the gym next week. Then, ask them how many times they went to the gym last week. Get different answers? This is because we aren't very good at predicting our own future behavior. (If you want to go deeper on this topic, check out the Ladder of Evidence by Teresa Torres.

So as an interviewer, what can you do? Ask instead for specific stories. Go back to your list of research questions, and think about what stories from the participant's past would give insight into how they've acted and behaved in your area of interest. If you're looking to improve an existing product, ask them to tell you about the last time they encountered the particular use case you are interested in. If you're exploring a new product, think about related areas.

For my example, I'd want to find stories about times when they hired research help. If they had never done so before, I'd ask for stories about times when they sought help in an area that they wanted to get stronger in. I put my interview guide in a table format so that I can have prompts and reminders of what's important right next to where I take notes during the live interview. It looks like this:

5. Build your interview guide part 2: plan a conversational flow to build rapport

Ever listen to a user interview where the interviewer sounded like they were doing a telephone survey? I have, and it was painful. The interviewer asks questions, and the respondent gives terse answers. At its worst, it can sound more like an interrogation than a conversation.

I find that the trick to avoiding this situation is to focus on building rapport before asking the deeper questions. I always start by introducing myself and asking if it's ok to record the session. Then I ask them about themselves. Most people, especially those who agree to participate in research, love to talk about themselves! Since I do a lot of B2B work, my standard opening question is to ask someone to tell me about their job and how long they've been doing it. For B2C, I ask them some basic personal interest questions to get them flowing.

However you start, the key is to make sure the first questions are easy, not thought-provoking. When questions make us think, we often start to worry about whether we are getting the "right" answer. You want to get the participant to feel like they are having a conversation with an old friend, not a challenging teacher.

So build out your interview guide with questions and prompts that will help you build up this rapport. Start with context questions, like descriptions of their organization or their family. Add transitions before changes in topic, so they aren't wondering what you are getting at and trying to guess how to answer. Then lead into asking them for specific stories. Plan your wrap up as well: tell them that the conversation was really helpful for you. At this point, you may want to tell them a bit about the project you are working on, if you haven’t already, and ask if they'd like to stay in touch for future research or updates.

6. Don't let the perfect be the enemy of the good

Once you've got an interview guide drafted, get started! With modern tools like User Interviews, conducting research has become so cheap that you don't need to spend hours and hours preparing to get the most value out of every call. If you have a lot of internal stakeholders, give them a chance to share their concerns and questions, but emphasize that you are starting a continuous process. The more you interview, the less planning you need to do for each one. Build up a habit and work on improving your technique and flow over time.

You'll be on your way to using customer interviews to truly empathize with your user, driving innovation in your product.

Want to contribute to User Interviews content? Here’s how.