Validating the effectiveness of Design
Why do we need success metrics?
As designers, we want the products we build to be satisfying and easy to use for a long period of time after we build them. But how can we know that's the case? We need to start tracking UX metrics.
UX metrics are a set of key quantitative data points used to measure, compare, and track the user experience of a website or an app over time. They need to be aligned with business goals. The use of success metrics reduces opinion-based decisions, helps prioritize problems and improves our products.
Key metrics are usually different for each product and should always be determined individually. The metrics that would fit to measure the success of one project may not work for another project. I have five-step guide on how to start with UX success metrics.
Step 1: Core Goal and Success Events
When we are creating success metrics, we need to start with the goal. When thinking about the goals, ask yourself, what will happen if our design is successful? The answer could be, we will increase orders or maybe our call centers will get fewer calls about the login issues.
It’s good to fire such discussion questions to stakeholders as soon as you complete the initial research and discovery. These discussions do not need to begin with data, but with goal oriented questions first. The most frequent goal types are revenue, acquisition, and engagement. Thinking about your goal is very important before you decide what specific metric to track it.
Once your team and stakeholders have a clear goal, prioritize your success metrics and success events. Success events are actions that you want your users to take toward your goal. It is better to have these micro-conversions as your goal versus full-scale conversions.
For example, if you are an online course company and your conversion is to have users registering for a course, your micro-conversions might be, signing up for a newsletter and downloading a brochure. Knowing your success events will help you prioritize them and understand what your objectives are.
Step 2: Actionable Metrics
Make sure to choose actionable metrics. What would I do if the metric is out of line? Do I have the levers that can impact it and measure the final outcomes like revenue or total customers? Do they give you much time to react or guidance about what to do next?
Step 3: Limit your Metrics
Also, focus on three to five key metrics rather than trying to capture and analyze everything. It used to be that having power meant having access to data. Today, having power means knowing what to ignore. Most companies are collecting large amounts of data every day. But having a lot of data or tracking everything we can think of will not equate to successful UX metrics. We need to make sure that we can always read between the lines of what our data is showing us because even simple metrics can be easily misinterpreted.
Step 4: Tell a Story
It’s very important to share insights, not just data. You need to tell a story about the numbers to make it easier for the reviewers to remember your point. Start with setting the scene by explaining the origin of the data. Then introduce the characters and the plot. Make the viewers care about those numbers. And don't forget to add a conflict to your story and a happy ending or a resolution.
Triangulation can be very helpful here to get context to add to your story. We can leverage different methods to collect data about the same phenomenon rather than relying on single signals. Having multiple points around the behavior we want to study enables us to make better decisions. So when you are measuring your UX success metrics with the help of analytics, sprinkle in some moderated usability tests or user interviews. It will enrich your understanding of how users use the product and will help you minimize the limitations and blind spots of single methods.
Step 5: Be aware of the phases of the data
Engagement, or time on site, could be positive but it could be also negative. For example, the time that the user spends feeling confused, distracted, or even frustrated. Even if you track engagement by looking at time spent on your site and pages per visit together, it's not clear how that equates to engagement all the time.
The skill is to read between the lines and I'm sure that will come with practice or with the use of more and more sophisticated tools.
Summary
UX metrics help us to track the success of our products in a way that aligns with business goals. To get the best outcome from your UX metric activities you can follow the five-step process:
- Start with a goal.
- Choose actionable metrics
- Limit your metrics to a manageable number
- Tell a story
- Be aware of the phases of data