Your Validation Test Didn't Validate Your Solution, Now What?

“You learn more in failure than you ever do in success.” -Jay-Z

Having a goal is one of the key components of any research project, but there are useful goals and then there are, you know, not useful goals. To be right about everything is not possible or the point.

You probably know that in theory, but then you get to what’s called the validation phase of research, and you’ve done so much work already, and it can be really tempting to just want to prove yourself right and move on.

Here’s the thing about validation research: it’s not about validating your product in isolation as good or bad; it’s about testing whether or not your product would provide a valid solution for the specific problem you’re trying to solve—which you’ve already already in the discovery phase—right now. Some even argue that the word validation should be avoided altogether.

In reality, you’ll likely get more useful insight from research that does not fully validate your concept, and that after all is the overarching goal of research: useful insight.

Tip: Avoid telling your test participants that they’re participating in “validation research” as it can imply that their goal is to prove that things work well rather than identify what could be improved.

So what is validation research actually meant to do and how do you avoid the trap of rushing through it?

Validation research provides useful insight into how well your prototype, wireframe, or high fidelity mockup meets the needs of the user as you’ve defined them.

Over on the InVision blog, Sarah Doody explains the three questions that you’re really trying to answer in usability interviews for validation:

1. Can people discover your product’s features on their own?
2. Did the outcome of a task match what was described?
3. Are there gaps in the experience?

All of these questions are oriented around whether the thing you’ve created does what it intends to do and whether it does it well.

Assuming your research does not validate your product entirely, what do you do next?

Don’t fret. All insight is good insight, and if you didn’t suss out any negatives at all, especially on your first go at a solution, there’s maybe something wrong with your research process?  

Let’s say your users have no problem finding one of your features but they don’t use it the way you had intended.

This is a great opportunity to consider ways to:

• Add cues into the product to guide users toward the intended use of your feature
• Adapt the feature itself to make the intended task more clear and intuitive
• Dig deeper into newly discovered use cases for your feature that you may not have envisioned
• Explore whether your intended use is really useful/desirable to users

How to Use Your Validation Insight

Every “invalidation” is an opportunity to make tangible improvements and avoid backtracking later on when you’ve invested so much more into building your product.

Heather McCloskey of UserVoice advocates using validation research to continually test your assumptions and avoid pouring resources into ideas that aren’t really necessary:

Don’t stop validating after you’ve determined your idea and MVP are worth putting effort into. As you learn more about your customers and begin developing and fine-tuning your product, it’s wise to make validation (or validated learning) a continuous process throughout your product’s lifecycle & development process… You may discover that existing solutions are just as efficient and effective as your new idea. You may discover that what seemed like a big, hairy scary issue is really an easily tamed pussycat. Or you just might have come up with the next big thing.

Sometimes your validation research will lead to obvious minor tweaks, but what if your insights send you right back to the drawing board with little to salvage? That will happen too. For this reason, many teams will create a few rough low fidelity versions of solutions early on to see which hits closest on the mark, then continuing to iterate in that direction, as opposed to having one succeed or fail option and then starting over.

The benefit of asking the right questions, gathering insights, and applying them in the early stages of the product development cycle (and beyond) is that you can save so much wasted time and resources down the line.

Final Thoughts

Your validation research (and any research, really) is only useful if it gets put to use, so how you apply your findings is every bit as important as how you do the research itself.

If your research doesn’t validate a particular feature as you’ve designed it, avoid jumping to conclusions about why too early on. Instead, use it as fuel for further inquiry, and a move ahead with what you do know.

Validation research is part of the journey, not the destination. Framing it this way may take some pressure off everyone involved, and help you maximize the critical feedback you will almost inevitably receive.

Want to contribute to User Interviews content? Here’s how.