Fighting for simplicity. With data.

by Salvador Salazar

2 years ago ・ 8 mins

My name is Salvador and I lead the Data Engineering and Analytics team at Userlane. My team and I have been working on the HEART development for over a year. It’s been a challenging but quite exciting journey, and now that HEART has been released, I want to share some insights on what we did to make measuring digital adoption simple.

HEART is a user experience framework originally developed at Google that we at Userlane have adapted to measure digital adoption. By measuring user experience of software tools, HEART can identify where they struggle or where functionality is not used to its full potential. A holistic view of user experience ultimately drives effective decision making and maximizes the ROI of software investments. If you want to know more about how HEART can help your organization, we published a Q&A article with our CPO here.

In the first part of this article, I will explain how we took the HEART framework and adapted it to work for digital adoption from a data analytics perspective. In the second, I will explain, from a data engineering perspective, the systems we developed to support it.

Why Frameworks are great

Firstly, is it worth understanding why we chose an analytics framework? 

Having key metrics organized within a framework groups them into related domains, giving them context and creating a hierarchy of metrics. This has the following benefits:

• transforming them into helpful concepts and ideas, aiding the diagnosis of potential issues and avoiding “analysis paralysis”.
• domains make metrics easier to communicate, making them more understandable to others less familiar with the metrics themselves.
• a common language for different stakeholders who might need to engage with these metrics at different levels. For example, a CIO might want to stay at the top level of these metrics, with a wide overview of multiple applications. Application owners may need to dig deeper into their specific application.

The HEART framework shines in all these areas by offering a set of easy-to-understand high-level metrics, with a simple common language for different actors to communicate. In parallel, it gives a comprehensive view of the user experience of the software, by combining business-relevant and behavioural data.

The next question to answer is, how do we develop a good set of low-level metrics to enable the benefits of this framework?

Designing effective metrics

Despite HEART already providing a great framework for measuring digital adoption, the metrics we decide to put in it are a “make it or break it” decision. These indicators need to not only fit the framework but they need to be relevant, as well as reinforce each other to be useful. We usually refer to this as “actionable insights” in analytics.

Actionable was one of the most important principles when planning what should feature in our implementation of HEART. The challenge is not adding metrics, but rather leaving them out. In this sense, one of our best practices is to ask ourselves:

If this metric goes up (or down), what does it mean? What do I need to do to change it?

If the answer of either question isn’t clear, as in “I’m not sure” or “it depends”, then most likely we need to remove it. There are some exceptions, but very few. For example, if a given page gets many visits, is it because it is popular? Or perhaps because it is the help documentation of a complex process?  

This poses a very significant challenge because relevance is highly dependent on who is using this information and the system being monitored. You might expect the majority of your users to use your HR management system once a week, but your CRM system should probably be used daily. We always endeavor to keep things simple and relevant for our customers and their individual use cases, to the point of having rejected many more metrics and indicators compared to those that made it in. 

We worked closely with early access customers spanning a wide range of needs and use cases, and we’re confident this helped immensely in having a set of metrics useful for all of them.

Some noteworthy examples are:

• In the case of Adoption, the benefit of knowing how many users you have and what features they use is clear if you are using Userlane on top of an application which users might or might not use at will. For example an internal knowledge base or your own SaaS application. On the other hand, the benefit might not be evident at first glance for those applications employees must use, but also in this case the metrics you find under the Adoption section can help you optimize costs for example, by identifying unused licenses, or unnecessary features or add-ons. 
• Engagement lends itself to a similar situation. What if your users have no choice but to engage with your application? In this case, you still can use those metrics under the E section for understanding where users spend their time. This, combined with Task Success (the T section) can be very powerful. If your engagement is high but your T score is low, your users are probably struggling to find their way to what is important. Once you identify this, you can use Userlane to guide them in the right direction through our other features, with tools like guides and tooltips.

Similar cases can be made with other metrics, and HEART users will find that adapting these KPIs to their use case, and combining them with filters and segments can be a powerful tool to simplify their efforts towards digital adoption. That being said, this is just the beginning! As ever, we expect to keep collaborating with our customers and iterating on HEART.

Final thoughts

We believe that with HEART we can help drive decisions and boost digitization efforts at scale, without overwhelming our customers or their employees. HEART delivers the information you need to know, not more, not less.

As for the future, we will keep refining and enhancing this first version of HEART with more advanced capabilities. We are already exploring some ideas, but we would love to hear feedback from our customers! Are you one of them already? Reach out to us and tell us how you plan to use HEART!