5 November 2015
Usability testing is a great way to identify potential problems with an interface. Quantifying the results can help illustrate what’s not working and to what extent. One common usability metric is the number of users who successfully (or unsuccessfully) complete a task.
For example, let’s say we’ve redesigned our app’s navigation scheme. We want to know whether or not users can successfully navigate the redesign. We can break down the interface elements into features or tasks to test, e.g. swipe left / right to view previous / next pages, find settings by opening the navigation drawer, and so on. To keep our example generic, let’s say we have 5 tasks… cleverly named Task A, B, C, D, and E. We run a small usability test with 8 people. We record the data in a spreadsheet (or CSV file) and use 1s and 0s to denote whether a user successfully or unsuccessfully completes a given task. Each row represents a user.
The recorded data looks like this:
When analyzing and reporting the results, a table similar to this one is often used:
This is a great start. But, visualizing the information can provide clearer insights. A heat map is a great way to enhance the table by adding color to the table of values:
The color scale used in this example goes from dark orange to light gray. If none of the users successfully complete a task, the color is dark orange; if all of the users successfully complete a task, the color is light gray. By color coding the data in this manner, we can immediately see potential problems… the darker the orange, the more we ought to pay attention to it. So, Tasks A and E are on our radar.
Why did I choose dark orange to light gray? There are many color schemes to choose from, but these are my reasons:
There are other valid color schemes to choose. It also depends on the type of data you want to represent. In this case, this particular scheme has worked for me when I want to show teams potential problems in the interface. It gets the discussion started on where to focus the next design iteration.
The first thing we need to do is load the libraries we need: ggplot2 and reshape. reshape is used to transform data. If you haven’t installed these packages before, you can do so by typing
Next, set the working directory where you have the data saved and read it in. Let’s also store basic stats, such as the number of users and the number of tasks:
Here comes the data preparation part. We want the total number of successes for each task in a data frame and in a format that ggplot2 can use. The following code accomplishes this:
The resulting data looks like this:
> total.m task variable value 1 Task A value 2 2 Task B value 4 3 Task C value 6 4 Task D value 7 5 Task E value 1
ggplot2 is a great graphics tool, but it can be a bit overwhelming to use when first starting out. Here’s the code to plot the heat map broken down into several steps.
The following pictures show what’s happening at each step:
This creates color cells using the scale specified. The limits of the scale should go from 0 to the total number of users.
The values themselves are still meaningful, so let's include them.
The size of the plot can use an update. ggplot2 defaults to creating plots where one unit is the same length on each axis. However, it's possible to change this. In our case, we want the y-axis to be just tall enough to fit our data.
By default, ggplot2 creates axis titles, tick marks, and uses a gray background. Let's remove all that and add a title to create the final plot.
There’s our final plot. We can then save it!
The next time you conduct a usability test, consider using a heat map to visualize table summaries of your data. Choose color scales that are meaningful to you, then plot away!