Many businesses have been forced to change their model to adapt to government restrictions due to coronavirus. Consumer behavior has also changed. What does that mean for your customers? How can you meet their needs during this time of uncertainty?
Beyond using research as a chance to hone your response to the pandemic, it's good practice to perform ongoing customer research regardless of world events. Even in normal times, conducting research remotely has some advantages, such as the ability to test users in their natural environment, pay lower participation incentives, and get research done faster.
Whether you’ve done remote research before or are just picking it up, our hope is that you’ll find useful processes and best practices in this guide. We’ll cover:
- What remote usability testing is (for the new folks) and why it’s not the same thing as user testing
- When to use moderated vs. unmoderated research sessions
- Research methods and questions
- Remote usability and user testing tools
- Participant recruitment
- Remote testing advice (from researchers who do a lot of remote testing).
Try the #1 rated research recruitment tool today.
What Is Remote Usability Testing?
Usability testing lets you study the ways someone might use your service or product or how they progress through the designs you’ve created. You can conduct usability research either remotely or in person, but we’re focusing on remote testing in this post.
- Usability testing is a type of user testing (though some people use the terms synonymously). Usability testing is more focused on the technical details: Can users find what they’re looking for? Do they know which buttons to press to complete tasks they set out to do? Does the site function as it was designed to?
- User testing is an umbrella term for research that helps you gauge the user’s experience, determine whether your product meets the users’ needs, evaluate new designs and concepts, and so forth.
Note: We provide advice for both remote user testing and usability testing in this post. Most advice for user testing is applicable to usability testing, but we also provide usability-specific advice where appropriate.
When you're preparing for remote usability testing, technical issues matter a lot. Consider:
- Whether there are any security concerns you need to address ahead of time
- If materials are accessible outside of your business’ firewall
- The level of instruction you’ll need to provide to participants
- How participants can get access to any additional tech they’ll need.
For the user research you’re planning, you’ll need to choose between moderated and unmoderated remote testing. In choosing between them, consider the research questions you want to answer (Can you get answers to them without being present for the test?), your timeline (Just how quickly do you need results?), and your budget (What testing practices align with your funding?).
Moderated Remote Usability Testing
Moderated remote usability testing is useful if you want to be able to ask follow-up questions, guide the participant through a test or series of tests, or simply want to be available in case something goes wrong.
When conducting moderated remote usability tests, try to create an open, welcoming environment so participants are comfortable talking about their experiences. Do your best to avoid leading questions (questions that prompt the participant to respond the way you want them to instead of how they would without any biasing).
Here are a few pieces of advice on conducting moderated research sessions from a researcher who does them frequently:
Unmoderated Remote Usability Testing
Remote unmoderated usability testing is better when you don't need to be present during the test. For example, if you just want to make sure the user experience for a specific process is intuitive, you can have them record their session. Of course, that means you’ll need to leave extra thorough instructions since you won't be there to guide participants through the test.
We recommend sending test completion instructions to participants immediately after they agree to take the test. That way, they’ll be able to take it when it is most convenient for them. In addition, you'll want to test those instructions ahead of time — test the test, as they say. Have a team member, family member, or friend take the test to ensure no one has misunderstood what you're trying to get them to do.
You won't be able to gauge body language, so if you’re doing qualitative research, remind participants to talk about what they're thinking and feeling during the test. You can screen for articulation ahead of time to make sure they’re able to express their thoughts in a clear manner.
How to Conduct Remote Research
In this section, we’ll discuss user testing methods and give you a resource for questions you can ask. Our goal here is to walk you through the main tasks associated with conducting remote research:
- Planning the type of test you'll run
- Deciding what type of tools you use
- Finding participants
- Running the research.
All of the methods listed can be accomplished remotely with the tools linked or listed in the below section on tools.
Usability Testing Methods
- Performance Test: A usability test can be as simple as having a user record themselves attempting to accomplish a specific task while using your product or service. For example, you might have them capture their screen while trying out your onboarding flow. If you want qualitative performance testing, how they feel about and respond to the experience will be the focus of your work. If you’re looking for quantitative data, you’ll need tools to measure how long task completion takes, how many clicks users make before they finish, and so forth.This test is best used on features that are nearing completion or to evaluate the performance of existing structures.
- Expectancy Test: If you’re mid-way through design, an expectancy test can serve as a pulse-check. Show participants a design or prototype and ask, “What do you think this does?” If the actual function and their expectations don’t align, you’ll know to make some changes.
- Visual Affordance Test: Give users an image of the page you’re testing, then ask them what they think is clickable and what they think is not clickable. This helps you evaluate whether your design aligns with users’ expectations.
User Testing Methods
- Card Sorts: Because card sorts help you determine information architecture, you could make the case that they are usability tests. But no matter how you categorize them, card sorts are a helpful way to create intuitive site navigation for your users. And they’re relatively easy to set up for remote, online testing.
- Interviews: Sometimes, all you need is a conversation to get great insights into what your customers care about. These still require careful planning to make the best of your time, but they require very little technology to produce meaningful results. Researchers employ generative interviews early in the ideation or product development process to learn what the target user cares about, what their problems are, what they’re currently doing to solve those problems, and so forth. That said, interviews can also be used in conjunction with other user testing methods to get more information from participants.
- Diary Studies:An excellent way to learn how someone interacts with your product or with some aspect of their environment over time. It’s best to get electronic submissions from participants (whether that means having them type up their thoughts or using their smartphones to record themselves).
- Prototype Testing: You don’t have to be in the same room as someone to get their reactions to a prototype. We recommend making web and mobile app prototypes high fidelity and creating more clickable areas than only what’s needed for the paths you’re testing (so that it’s a better representation of the real experience).
For more user testing methods, visit our UX research field guide.
Other Remote Research Methods
- Surveys: Surveys aren't really a usability test, but they are still tremendously informative if done well. They’re usually conducted remotely, so all the tools you need to get started are readily available. If you run a user experience survey the way Els Aerts does, then it’s an excellent starting point before diving into user testing.
- Site Analytics: A good precursor to research. If Google Analytics shows a sudden drop-off in traffic to a lucrative e-commerce page, for example, you’ll want to know why. Or you might use heat maps to see which areas users spend the most time on, then conduct research to improve their experience.
70+ sample user testing questions for your next study.
Remote Usability Testing Tools
We took a thorough look at user testing tools previously, and you can also find our take on the best video conferencing tools here. Sonya Badigian, a researcher for an entirely remote team, shared some of her favorite usability tools with us during our podcast, Awkward Silences:
- Video Conferencing Tools: Zoom, Skype, GoToMeeting, BlueJeans
- Usability Testing Tools: Lookback, Userlytics, UsabilityHub, Optimal Workshop, Loop11, PlaybookUX
- Wireframing and Prototyping Tools: Invision, Balsamiq, Axure, Sketch, Marvel, Mockplus
The tools you choose depend on what kind of research you want to do, so we recommend choosing tools based on the specific functionality you need.
Participant Recruitment
Once you know exactly what you want to test and how you plan to test it, it's easier to choose participants who will provide the insights you’re looking for.
In this stage, we recommend choosing participants based on their behavior, not just their demographics. In this post on recruiting usability test participants, we shared more details on how to craft your “pitch” (what you use to ask potential participants to take your screener survey), the screener survey questions, and your instructions.
You’ll also need to decide on incentives (we have a calculator for that), go find the participants (which we can help with), and book them (we do this automatically if you work with us).
And of course, don’t forget to decide how many people to include in your research. For a simple usability test, around 5 is often all you need.
Try User Interviews free today.
7 Pieces of Remote Testing Advice
Aside from the things you normally do in your in-person research, here’s our top advice to have smoother, more effective remote testing sessions.
- Set clear expectations for how long the test will take: Conducting a moderated test? Tell participants how long they’ll be chatting with you, and check that they still have time for the full conversation. Conducting an unmoderated test? Tell participants in advance how long they are expected to spend, at least roughly, on the exercises you set before them.
- Practice with someone you know: It's a good idea to test the instructions with someone you know before you send them to participants. Try as we might, there are times when the instructions we think are simple mean something different to someone else. If you test the instructions with someone ahead of time, you may be able to avoid further issues. It's also a good idea to revisit your instructions if the first few research sessions result in unhelpful information.
- Break up complex tasks into smaller tasks: If you want participants to complete multiple tasks, make sure they are written up separately. For example, if you want them to accomplish setting up an account, taking an action, and inviting collaborators, ensure those are all listed as separate steps. It's easy for testers to miss instructions that are crammed together.
- Make a reminder to hit ‘Record’: If you’re doing moderated testing, it's so important to record your conversation. Make a note in an obnoxiously bright color where you know you’ll see it so that you remember to click ‘Record’.
- Make video optional unless you really need it: Making video optional means you can get a wider variety of participants and a higher-quality audio recording. If seeing someone's face is important, then don't skip the video. But if video is not crucial to your research, it's okay to pass on the face-to-face time.
- Have a backup plan: Try as you might, no matter how well you prepare, you may still find that something fails during the research session. For every piece of technology you're using, have a back-up plan. For example, if you're planning to have participants join a Zoom call via laptop, ensure they have the call-in information in case their internet goes down or their mic stops working.
- Make sure participants know they won't offend you: If you are not directly involved in making the product you’re testing, make like Sonya Badigian: Explain that your feelings won't be hurt by negative feedback. Even if you are involved in the design, it's good to reassure participants that you want their honest feedback and are just as pleased with negative feedback as with positive feedback.
Conclusion
If you’re looking for more information on how to conduct better remote UX research, we recently spoke to Behzod Sirjani, Head of Research and Analytics Operations at Slack. He covered topics such as the benefits of remote research, things to consider during the COVID-19 pandemic, best practices for remote research, tool recommendations, and more.
.png)
.webp)
