You can’t build great products without user research. That’s something we really believe at User Interviews, an idea ingrained in our DNA as they say. But user research means a lot of different things to different companies—not a bad thing—and for all the buzz around topics like design thinking, empathy, and agile research, there are at least as many questions as answers when it comes to doing research “right.”
So, we’re on a mission to figure out how to do it right. Or at least how to do it right for OUR organization. And we have a hypothesis, one we will evolve and test and validate of course, that this approach we arrive at might benefit a lot of other companies too. Because at the end of the day, we have a lot of the same questions, maybe we have some of the same solutions, too.
So far, we are a couple weeks into our “official” efforts to “get really good at doing user research.” Part of that effort involves reading and learning, and sharing what we’re reading and learning internally, even more than normal. As part of that, we came across this excellent article from Joe Munko, Partner Director, Research & Insight, Windows & Devices Group at Microsoft. It’s about creating timeless research, not disposable research. That appealed to me, so I reached out to Joe and wouldn’t you know it, he wrote back—I don’t care what anyone says, user researchers are the best people. We talked, I took notes, he had some cool stuff to say about HOW to create timeless research, and I’m going to share that with you here.
What exactly are we trying to accomplish with doing research right and what do we even mean by that?
There’s a lot of debate these days on doing research right versus not doing it at all. Just enough research versus methodological rigor. Democratizing user interviews versus keeping your customers away from anyone who isn’t a trained moderator. Let’s just say, as with all things, there’s a continuum of worst to best, and of course context is everything.
For our purposes, we’re interested in building a research practice that:
- Yields clear insights. It should actually help us know what to do next.
- Is repeatable. We don’t want to spend most of our finite research time thinking about process and mechanics.
- Is habitual. It should get lots of people in our organization talking to customers/users regularly.
- Builds long-term customer knowledge. Each study should have a clear learning objective, but that need not be limited to validating a particular feature.
- Is fun, not too hard, and efficient. We want it to be something enjoyable. We don’t want it to feel like a burden. We don’t want to waste time.
- Is special. We hope to, over the next year or so, develop some actually new, provocative, interesting ways of making research work in a real company that others might want to steal from us—that would, if we’re being real, be the ultimate success.
So with that in mind, Joe and I covered a lot of these topics a little bit, but we focused on the aspects of building long-term knowledge through what he calls timeless research. I was thrilled to dive into this topic with someone who has actually been working to make it happen for years. In my year or so of being fully enmeshed in the world of user research, the piece on making the most use out of the research you actually do feels like one of the most important, and least solved. What follows are some of the main insights from our conversation, insights we plan to let inform our evolving approach to user research at User Interviews.
“You don’t learn it until you apply it.”
If the goal is to make research actually useful, to turn the unknown or assumed into the known and understood, part of what we’re talking about is the nature of learning itself. Joe uses the example of a child who touches a hot stove. If they touch the stove again, the learning hasn’t been applied, it hasn’t actually been learned. On the other hand, if they associate touching the stove with getting hurt, and don’t do it again, they have learned the key lesson. They applied it.
User research works the same way. Insights sitting in a Google doc somewhere will never be applied. So, how do you apply the research you do?
“It starts with people.”
User research is all about people at every turn. From building an overall understanding of human motivations, behavioral economics, and the like over time, to asking questions that elicit true answers, to sharing insights in a way that makes an impact on those you seek to influence (i.e. your internal stakeholders), this whole discipline is all about people.
When it comes to building timeless research, Munko shares “We’re so inclined to speed, we don’t have people whose goal it actually is to learn.” People work in systems, and many of our systems, especially in an agile environment, prioritize speed above all else. But if you say, actually our goal in research is to learn about our users, and to learn more over time, so we can build products that serve them well, that our competitors could never dream of building because they haven’t invested that time to actually learn—then speed to cobble together a method that meets the constraints of the product that needs to ship next week doesn’t feel like the right goal anymore.
So the first transformation that needs to happen is that people need permission and motivation to design research that will last beyond the next product release, and then to start with “what do we already know,” instead of starting from scratch each time. “The fastest method is to use what you already have. That will beat an a/b test anytime,” Joe says.
Build research to last
Of course, you won’t have anything to reuse if you don’t build your research to last in the first place. So, how do you build research to last? Part of the cause of disposable research is this construct of new product = new research. A better way is:
- Start with what you already know
- Form hypothesis using this information
- Build a study for new information needed to validate this hypothesis
- Incorporate new and old research into findings
- Incorporate new research insights into your “canon” of research, so you can use it again
How do you get to a place where you can build that canon of research over time?
Document it
It’s very hard to share what’s in your head, at scale, over time. Get notes, research artifacts, and especially findings and insights out of your head and into a document.
Document it well
The minimum step is to put it somewhere. The more important step is to put it somewhere accessible. For this, you’ll need some tools. At Microsoft, Munko uses a combination of Powerpoint, Sharepoint, Word and the like, and of course they all work perfectly 😉, but they aren’t purpose-built for atomized research.
If I’m a researcher, or a person in a company who is trying to solve a customer problem by understanding customers, and I need to understand what tools people use when they’re using our product’s calendar and how they feel about it, I’m not looking for the “calendar study” that doesn’t exist. I’m looking for a summary of feedback we’ve gotten on these topics through active (say user interviews or qualitative usability testing) and passive (say support tickets and NPS survey data) feedback and insights.
I don’t need a report, I need insights spread across a variety of inputs, brought together. “How do I ladder what I learned into how to apply it?” as Munko says. Many organizations, including Munko’s team at Microsoft, are working at creating tools to better support this opportunity. Theirs is called HITS, or Human Insight Tools & Services.
“HITS is a platform that helps teams easily discover, curate, and democratize human insights, breaking them free from organizational and memory silos. HITS modernizes the way we work by atomizing insights so researchers can reuse and remix them, making connections that contribute to a dynamic and living body of knowledge."
For our efforts, we’re experimenting with Productboard. It’s a step up from Airtable, which we were using, because it’s easier to both submit and find information you’re looking for. More on this later as we get to using it and refining the process. A few things we’re thinking about though are: automating where possible and creating the most useful metadata, plus the easiest ways to input that metadata. Munko says of creating a data model in the beginning of their system “You’d be surprised how much time and energy we’ve spent on it,” and that “words matter” and “which words you choose is super important.”
The metadata we are currently capturing include:
- Freeform tags we can apply to insights
- Features we can add insights to
- Who is the insight from (customer/user)?
- Who collected the insight (internal person)?
- Date captured
- Channel we collected feedback from (email, Zendesk, Google doc = user interview, etc)
We’re starting to think about what should be tags versus features, and how to store general insights about user context and motivation that don’t map clearly to current or planned features. This feels important for creating timeless, reusable research, but hey, you can’t figure everything out at once, yes?
Use it
Only through using the research canon you’re building can you test its efficacy in doing its job: helping you find insight and apply it efficiently. As you use it, seek to improve the system based on what works and what does not. “The more mature your product gets, the more you’ll be asking the same questions,” Munko notes. Setting up a system early should provide compounding benefit over time, saving you time by not reinventing the wheel, and building the right products the first time. That’s what we’re banking on, and we’ll be back soon to report on our progress and learnings.
Want to contribute to User Interviews content? Here’s how.
.png)
