The PURE method (Pragmatic Usability Rating by Experts) was developed by Nielsen Norman. It consists of a method to evaluate the UX and ease of use of a Software in a fast, cheap and effective way.
The PURE method quantifies the difficulty of using a product and helps us obtain qualitative data on how to solve the problems detected without having a high economic cost.
How to measure UX and Usability
Companies that have a certain need the ability to obtain metrics to make decisions. They usually use a/b tests, surveys, heat maps, etc.
As for the user experience, it can be measured in various ways and various techniques. From to using techniques that evaluate people’s behavior or what they do (analytical, , , etc) and those of attitude or what people say (, , , etc).
All of these techniques are useful for extracting data and drawing conclusions, but there are important limitations:
- Numerical data is not helpful in understanding and explaining why a problem was observed and how to fix it.
- The a/b tests and their results only provide values on the performance of the Software.
- Techniques for obtaining user feedback, such as surveys, tests with users, etc. they are time-consuming, some are expensive, and are not done regularly enough to have regular evaluations.
What is the PURE method
Reviews by UX and Usability experts are not new. The PURE method can remember those that are already done for a long time.
The PURE method involves multiple experts assigning one or more quantitative ratings to a design based on a set of criteria. Those individual ratings are combined into a final value and is represented visually so that it is easy to understand.
This method can resemble or be assimilated to a heuristic analysis. In fact, the experts who do it are familiar with UX principles and heuristics.
PURE scoring system
This type of analysis to measure the user experience focuses on a single aspect, that is, the ease of use of the product.
Each task has a series of colored bars, which represent the steps to execute a specific and particular task. Each step is evaluated, rated and colored, depending on how easy or difficult the expert evaluator considers that step for the target user.
The rating for each step is a simple scale from 1 to 3:
- It is scored with a 1if the user can easily perform the step, due to the low cognitive load.
- It is scored with a 2, if the step requires a significant degree of cognitive load or physical effort on the part of the user. In the end, he succeeds but it requires some effort.
- It is scored with a 3, if the step is really difficult for the target user. Due to a high cognitive load the user would probably fail or abandon the step at this point in the task.
The score for a given task is the sum of the points awarded for each of the steps. The color of the task’s score is determined by the worst score that one of the steps received when evaluating the particular task.
The numbers and colors show the level of user friction running the task. The higher the number and the hotter the color of each step and the color of the task, the more friction there will be for the user.
Because the PURE method measures the friction or ease of use in a set of steps when executing a task, it is important to define both the tasks and the steps that comprise them.
Realistically, not all tasks can be measured, so PURE should only rate core tasks that are critical to users and the business.
How to measure UX with the PURE method
As Norman Nielsen tells us, the PURE method consists of 10 steps. 8 mandatory steps and 2 optional.
Step 1: Identify the types of target users
In order for the analysis and assessment to be optimal, the type of user that will execute the specific task and therefore each of the steps that comprise it must be correctly identified.
The type of user can be defined by the Product Manager of that part of the product, the main designer or it can be decided by the team of UX experts.
Either way, defining the types of target users using the technique will go a long way in this process.
It is possible to identify and score taking into account various types of users. It is important to limit to no more than 2 or 3 types of users. If there are more than 3, the PURE method loses its main objective and purpose.
Step 2: Core Tasks
The tasks to be analyzed by the experts must be important at a strategic and business level. Also, it is important to define and select tasks that allow the target users to satisfy their main needs.
For example, in an Ecommerce or Marketplace, a fundamental task would be the purchase or Checkout process, the login or registration, adding a product or item to the basket from the favorites list, adding an item or product to favorites, etc.
Step 3: Happy paths or steps of each task
A particular task can be accomplished in many different ways. Using this method, to assess ease of use, requires the team of experts to identify the happy path.
The happy path is nothing more than the desired way in which the target user will perform the task.
Another option is to use analytical tools to check, through clicks or other metrics, which is the path or steps that users of the product take the most and review only this path, which is the one most used by users.
Also, both can be analyzed, although it takes more time and effort.
Step 4: Define the limits of each step
Once the happy paths are identified, it is necessary to define and identify where each step begins and ends.
- A step begins when the system presents users with a set of options and they have to decide between one of them.
- A step ends when the user performs an action and waits for a response from the system.
A step can have microinteractions. Fill in form fields: If we’re in a signup flow, a step might start when the user hits the create account button and ends when they fill out the form and hit the submit button.
Step 5: Review by 3 expert evaluators
The evaluation will be carried out by 3 experts in usability and user experience. At this time, each of them performs the evaluation individually, assigning a value from 1 to 3 to each step of each of the identified and defined tasks.
It is very important that none of the experts have participated in the design of the experience to be evaluated. Your opinion or judgment could be biased.
In this step, the team leader will execute the task and the rest will score each step while visualizing how the leader performs it. In this way, all experts see the same thing and can assess individually at the same time.
Step 6: Score Review
To check the individual scores of each expert and the agreement between them, a measure called IRR (Internal Rate Of Return) is used.
It usually takes 2-3 rounds of testing for researchers to fully understand the PURE method. Testing using this method is necessary for the sole purpose of learning how to apply it correctly.
Step 7: Final Consensus Score
The raters meet and agree on the scores for each task together. It consists of going through each step of each task and debating so that each of the experts learns from her professional colleagues.
In this way, the final PURE score encompasses the wisdom of all the participating user experience and usability experts.
Step 8: Create the colored sheet music
In this step, the graphical representation of the scores obtained is made. Both for each step of the task and for the task as a whole.
Using height and colors to represent friction is very effective in conveying trouble spots.
If many steps are yellow (2), it means that the user had to put some effort into it. Even if many steps or even the entire task is green (1), it is important to review the experts’ notes since there are always points for improvement.
Step 9: It’s Not All Numbers (Optional)
While the experts evaluate each step that makes up the task, they must make notes with proposals for improvements and include the reason for that score.
These annotations must be accompanied or documented with screen recordings, screenshots, etc. that help the design and development teams to understand the bugs and improvement proposals.
Step 10: Comparison of PURE over time (Optional)
One of the advantages of PURE is that it allows comparison over time. In this way the evolution of the product can be demonstrated.
Facebook Comments
Disqus