Diagnosing & solving product adoption issues

The web analytics product I worked on at Acoustic
 had been an industry leader when it first came out,
but had fallen behind the market in recent years...

 

The Research Need

Diagnose & solve issues that were having a large, negative impact on adoption and/or retention.

The Approach

  1. Formed a hypothesis based on existing data (Onboarding data from Customer Success team, multichannel customer feedback & heuristic evaluations from UX team)
  2. Investigated underlying issues via UX research: User interviews, usability testing, audit
  3. Evaluated & iterated on a redesign

The Outcome

Before redesign: New users used to need months of training, and many still struggled to complete key setup tasks, particularly users without a coding background. Existing & technical users also struggled with usability issues & bugs in the setup UI.

After redesign: New & existing users, technical & non-technical users were able to successfully complete key setup tasks without training or with minimal help.

More details

Research phase I:
Understand current pain points

Existing data exposed a months-long learning curve, user-reported complaints and struggles with the setup process, and a list of UI issues from a previous heuristic evaluation.

 

We needed to learn...

  • Where were users' specific struggles when learning and performing key setup tasks? Did this vary based on users' technical knowledge, product-specific knowledge, or both?
  • What were users' expectations of an analytics setup process?

 

Research approaches

Icons by Graphica, Freepik, & Kliwir art

 

 

Key insights from Phase I:

  • The tool's product-specific terms and concepts were a barrier to learning the product, for both technical and non-technical users.
  • The mental model of the setup process didn't align with users' expectations. It was based on the system model, not users'.

  • The current process was intimidating to non-technical users, requiring them to dig through code-based interfaces.

  • The product didn't provide a clear starting point for new users, and multiple usability issues made it difficult for both new and experienced users to easily complete key setup tasks. 

Research phase II:
Testing a redesign

I worked closely with the product designer on a new design. She ran rounds of lo-fi prototype testing & design iteration, which resulted in a hi-fi interactive coded prototype.

We needed to learn...

  • Using the newly-designed interface, could technical and non-technical users successfully perform key setup tasks? What about existing users and new users?
  • What were their attitudes about the new design?

 

Research approach and outcome

Icons by Freepik & Iconjam

Excerpts from the report

This was a huge win, since the previous version required months of learning & training.

 

 

Research lessons learned

  • Stakeholder wins! On past projects, our internal Services Engineers (who also used the product) had not understood product changes that were made based on user research.

    During this research, Services Engineers were included in the research as participants (to have their needs included), and also heard other users' perspectives during the results read-out, so they better understood the design decisions.

“This is some of the best research I've seen.
Excellent job!”
~Services Engineer

 

  • Cross-functional collaboration. The Product manager, Product Designer, and I were all present during user sessions, and debriefed about user needs, struggles, & design iterations. This, along with the skills of the designer (see her LinkedIn here), contributed to a successful redesign, and team alignment!