How to measure UX research impact: A multi-level framework

You might be very busy conducting research, but what if most of your work collects dust in someone’s drawer? Whether you are the only UX researcher in your organisation or part of a larger team: it makes sense to reflect on the impact of user research regularly.

But how to define and measure your UX research impact? What are the frameworks that help us think about and track it? These are the questions I asked myself and the UX research community.

When discussing this topic and exploring available resources, I noticed we discuss many different things—some talk about UX metrics, others about the ROI or mapping impact on strategic decisions. There are a lot of angles to consider here – UX research impact is not one-dimensional.

I propose a framework for defining and measuring UX research impact across different levels in this article. This is an ongoing effort, so please connect to share your thoughts on the topic and this framework.

The framework explained – How to measure UX research impact on 3 Levels

Level 1. Impact on the customer and business outcome

Generally, our ultimate goal is to create the best possible experience with products or services for the end-users. So, we need to consider the impact of our research on the outcomes:

• User Experience Outcomes – our work is about learning ways to improve the end-user experience. How can we make it easier to perform tasks or improve our customers’ lives with new functionalities? We can track UX metrics to see if the changes we propose or implement based on research positively influence the user experience.
• Business Outcomes – but what if our organisation is not yet fully convinced of the usefulness of UX research? It might help to provide insight into our work’s impact on the business KPIs or metrics (for example, through a/b testing). Did a design change a product team made based on your research result in fewer shopping cart dropouts? Or reduced customer service contacts? If not an exact measurement, keeping track of such case studies might help get more buy-in.
  

Level 2. Impact on the organisation

While it is essential to look at the end outcomes of our work, it can often be challenging to establish a direct link with our research efforts. We can, however, consider the organisational impact we are making in terms of the following:

• Organisational learning – in an organisation, all kinds of forces play a role in product development. Research insights do not always lead to product improvements. Solutions might be technically challenging to implement or substantially affect the time to market. As a researcher, it is good to track how research insights affect the organisation’s strategic choices and product development.
• Engagement – if the ‘organisational learning’ level is low, look at stakeholder engagement with research. If product owners, designers, or project leads are not participating, chances are that not much will be done with research results either. On the other hand, if stakeholders experience the value of research, they can be more inclined to consider the learnings.
  

Level 3. Impact on the user research practice

At the most fundamental level, we have an impact on the UX research discipline and its maturity. How much of our time do we spend on research? What research activities are we doing? How much research headcount do we get?

• Structure – When a UX researcher’s work is ad hoc and unstructured, it can be harder to impact the UX outcomes. The researcher spends a relatively large amount of time on project planning and operations. For example: how is research initiated? Is it standard procedure to include a UX researcher at different steps in the design process, or does it depend on the whims of the product owner? Track at what point in the design process the UX researcher is included and how.
• Reach – With ten researchers in the organisation it’s usually easier to make an impact than with one researcher. If we mostly do evaluative research at the end of the development cycle, our impact on the outcomes and the organisation might be limited. So, consider your ratio of discovery and evaluative research. 

How to measure UX research impact? Start where you expect the most significant impact 😉

I hope you don’t feel overwhelmed at this point with all the different levels of impact you can start tracking or considering. In an ideal world, you might want to set up multiple measurements for each.

Think about the status of your organisation and the challenges at hand. If you are building a research culture, you might want to focus on UX and stakeholder engagement metrics. If you are building the case for extra headcount, it could make sense to track business outcomes. What do you want to achieve with the UX research team, and what are the success signals?

Please let me know if you think this framework is of help and/or how we can improve it.

Resources

• Tracking impact of day-to-day UX Research activities – by Kanishk Shukla
• Tracking the impact of UX Research: a framework – by Caitlin Sullivan
• How to Track the Impact of Your UX Research – by Lizzy Burnam
• A Framework for Demonstrating Research Impact – an interview with Brandie Smith
• Impact & UX Research: What Is It and How Do We Know We’ve Achieved It? (video) – by Victoria Sosik
• Impact mapping
• Why UX Outcomes Make Better Goals Than Business Outcomes – by Jared Spool
• The Need for Quantifying and Qualifying Strategy – by Itamar Medeiros
• Track UX Research, Prove its Value (video) – by Ana Filipa Couvinhas
• Measuring the internal success of user research – by Nikki Anderson
• Signs UX Research Is Making an Impact – by Tomer Sharon
• 10 Metrics to Track the ROI of UX Efforts – by Jeff Sauro

🙏 Thanks for the resources and for sharing your ideas on impact measurement: Katie Kelly, Tudor Cristian Bogdan, Bas Bakker, Maurice Beerthuyzen, Kathleen Asjes, Itamar Medeiros, Joana Cerejo, Chris Jager, Julie Krohner, Laura Wells & Ana Filipa Couvinhas!

Image credits to Clint Adair on Unsplash