Research: Quantitative usability testing

The Automation team at Ginkgo Bioworks had redesigned the interface for setting up large-scale experiments on lab machines. They wanted to compare success of the new design to the previous one...

This was one research sprint of the rapid usability testing program I built at Ginkgo Bioworks.
See more about the program here.

The Research Need

Compare a new interface to the previous/existing one, to measure success.

The Approach

  1. Worked with the team to define "success" in concrete, measurable ways.
  2. Because of the success metrics we landed on & the available resources, used A/B-style quantitative usability testing, with a within-subjects design to compare the efficiency, number of mistakes, and perceived usability of the two designs.

The Outcome

We found that the new design was easier to use, reduced certain mistakes, and reduced in-lab time. This was a positive outcome because the lab machines were shared by multiple teams, so reducing in-lab time meant more teams could move through using the machines. We also discovered a major issue that impacted the learnability & efficiency of the new design.

More details

Defining "success" & research approaches

Partnered closely with designer, product manager, and software engineers...

  • Writing tasks for testing
  • Defining participant characteristics
  • Setting up testing environment
  • Attended user sessions as observers & note-takers
  • Group synthesis session of their observations, facilitated by me

Mural from the group synthesis session.
This is the result of everyone working together to synthesis the notes they all took during the user sessions.

Findings: Highlights

Here are some of the highlights from the research report. (This is not the entire research report.)

Impacts of this research

  • Because of the improvements in time-on-instrument and the positive usability scores, the team moved forward to roll out the new design.
  • The designer also worked on a design change to prevent the “wrong turn” some users were taking, to be included in a future update.