What's A User Feedback with a Severity Scale?

Published:

September 5, 2024

Last Updated:

September 5, 2024

Feedback is a gift, right?

We tend to think so, but can also concede that all feedback is not necessarily created equal. When communicating user feedback within your organization, it’s important to indicate how severe the issue is—otherwise, it’s all too easy for this important data to come across as a lot of noise.

Enter the user feedback severity scale, a framework that provides additional context, supports internal teams in effectively prioritizing what comes inbound, and ensures that the most severe issues for customers can be addressed well ahead of minor or (gasp) major problems.

We use just such a scale here at User Interviews (which you’ll find below). It’s likely that you’ll need to do some workshopping to create a severity scale that works for your own organization.

So, what factors need to be taken into account when building your own severity scale?

Below, we’ll cover:

The importance of a severity scale
A sample user feedback scale
Creating a severity scale at your organization
When and how to report severity

Why is a user feedback severity scale important?

Many organizations monitor the quantity of user experience or product issues using a tagging system. The more an issue comes up among customers, the higher level of importance it gets assigned.

However, the frequency of a given issue is only part of the story. Imagine 10 people report a minor issue (such as a typo), compared to only four reporting a severe issue (one that may lead to more drastic negative outcomes, like customer churn). Considering severity and frequency together is a more strategic way to prioritize issues.

With a severity scale, you can add a new layer to your tagging system, allowing for better prioritization, which in turn might reveal new product or feature opportunities.

How to create a severity scale

Creating a severity scale is a collaborative process. There are multiple stakeholders with a lot on the line when it comes to product roadmap prioritization and user experience improvements. Some stakeholders that should be involved in this conversation include:

Customer Success
Operations
Product
Research
Sales

💡 Learn more about cross-team collaboration for research success.

Sample user feedback severity scale

Here’s an example of how we’ve categorized feedback severity at User Interviews (courtesy of our Lead UX Researcher Morgan Mullen)

Create your own severity scale with our free template

You may notice we’ve omitted purely positive feedback from the severity scale. That’s not to say it’s unimportant — we love it when customers sing our praises. In these cases, you can just note it as “positive feedback” rather than giving a rating, as this severity scale use case focuses solely on constructive feedback.

💚 Love research templates? Get our severity scale template and head to our library and download all of them for free

Assessing levels of severity at scale

So, what makes a piece of feedback a 0 vs a 3? Here are a few examples of what that might look like:

Delighter

“It would be so cool if you included XYZ integration!”
“Is there a way to make this feature more customizable so that I can create some more buzz around the organization?”

Minor

“The export function doesn’t appear to be available for certain pieces of data — when can you add the ability to do that?”
“Is there an automated option for XYZ process your team can explore?”

Moderate

“The functionality of X feature is making it difficult to use the product.”
“This functionality doesn’t really align with my workflow — is there a way to have that match those processes?”

Major

“I’m concerned we’re collecting personal information that might affect our GDPR compliance – can you ensure that is not the case?”
“Collaboration in XYZ features is not efficient and a huge pain point for us — can you add this to your product roadmap?”
“Is there a way to better manage permissions and access so I can ensure sensitive data isn’t being seen by everyone on my team?”

Keep in mind that multiple customers may encounter the same issue, but the level of severity may differ from customer to customer. And of course, everyone’s product experience will be different—but by evaluating feedback from multiple customers, you should be able to gauge which issues are collectively more or less severe.

🛑 Disclaimer: Severity scales are not going to be a perfect science. There is always some level of subjectivity in any form of qualitative coding. And, like anything, it will take a bit of practice to ramp up in applying the scale. Do your best based on the definitions in the scale and examples provided above. When in doubt, ask your colleagues.

When should I report the severity of user feedback?

Severity is most useful when communicating customer feedback or reported challenges encountered in products. This can be feedback from:

Customer check-in calls
Customer emails
Automated survey responses
User interviews
Formal research projects (e.g., usability studies)

Here are a few ways you can implement severity protocols at your organization:

Severity rating required:

User feedback submissions
Usability test analyses/artifacts

Severity rating encouraged:

Interview snapshots from discovery calls or research sessions that include usability feedback
Research artifacts/summaries where challenges or usability feedback were uncovered

Severity rating not necessary:

Concept testing that involves very early stage prototypes that are going through frequent iteration
Interview snapshots or research artifacts/summaries that involve very broad, generative research not focused on specific issues in the product

📚 Related reading: Evaluative Research Methods - A Field Guide

How do I rate user feedback severity in practice?

When a customer presents feedback, it’s important to follow up so you can better understand how severe the issue really is. Sometimes, the severity will be obvious based on how the customer approaches you or if they provide a ton of detail without being prompted. But, in most scenarios, it’s wise to air on the side of caution and ask some follow-ups.

Ask the user

Here are some follow-up questions that could prove useful, whether they come from user interview, emails, survey, or other forms of customer feedback:

Tell me more about how this issue impacts you/your team’s work?
How often is this issue impacting you/your team?
What is happening as a result of this issue?
On a scale from 1 to 5, with 1 being a very minor issue causing mild annoyance and 5 being a severe issue causing extreme frustration, how would you rate this? Tell me more about why you landed on that rating.
Can you tell me a little bit about how this should work?

Analyze usability test results, interview snapshots, or other research artifacts

Here are some steps ways to put severity ratings into practice based on usability tests and other user research artifacts:

Reflect on and analyze your research insights as you normally would
For any usability issues, challenges, or product feedback insights, reflect on the severity scale and determine where you think items should fall for each customer
Ask the following questions:some text
1. Was the customer blocked or was there a workaround?
2. How much time is the customer losing?
3. How critical is this task to their workflow?
4. Was it a suggestion, minor friction, annoyance, or palpable frustration?
5. How did the customer quantify or describe the impact this issue had on them?
Look collectively across all of your participants to determine what you think the overarching severity score should be for that issue at large.
In your interview snapshot or artifact/summary, add your severity rating when reporting each usability issue uncovered