Defining severity in Usability Expert Reviews

decision-tree

It’s easy to see why there is often push-back on the results of usability tests (nobody likes to be told why their solution doesn’t work), but it’s important to remember that the goal is the same for everyone on the development team: to constantly improve the products we design. Effectively communicating usability results is one step towards designing better products. Via.

Having a standard process for defining severity means that you can be consistent in the way you assign severity and means that you provide the transparency needed for people to check your work. With an expert review,
deciding on the severity is more a matter of judgement, but there are three questions you can ask to improve your objectivity.

From David Travis a concise set of questions to help assign a rating:

Does the problem occur on a red route?
Red routes — frequent or critical tasks — are the most important tasks that the system needs to support, by definition.

Is the problem difficult for users to overcome?
Some usability problems are show-stoppers: users just can’t proceed.

Is the problem persistent?
Persistent problems — problems that keep cropping up — are more severe because they have a bigger impact on time on task and on customer satisfaction.

The complete article: How to prioritise usability problems

More views:
Heuristic Evaluation Quality Score (HEQS): Defining Heuristic Expertise
Rating The Severity Of Usability Problems

20 years ago Jacob Neilson in his article Severity Ratings for Usability Problems outlined the following 5 point scale:

0 = I don't agree that this is a usability problem at all
1 = Cosmetic problem only: need not be fixed unless extra time is available on project
2 = Minor usability problem: fixing this should be given low priority
3 = Major usability problem: important to fix, so should be given high priority
4 = Usability catastrophe: imperative to fix this before product can be released

Problem Prioritization in Usability Evaluation: From Severity Assessments toward Impact on Design

Severity Scales.xls

Effectively Communicating Usability Problems

UXValue as a tool for prioritize bugs

Making sense of usability findings with criteria based severity ratings

Usability Severity Rating – Improved