As we didn't have any personas, and there was a hypothesis that light users were largely responsible for low NPS, we divided users up mostly by their frequency of use, creating temporary personas of Heavy, Medium, Light and New (using the product for 1 month or less regardless of frequency of use).
The light user research fulfilled two goals. Validating the hypothesis that light users were responsible for our low NPS was a major factor, but the company felt that one of the reasons for customer churn was that it was primarily project based (for example, a merger or acquisition) and when that project was over, the need for the product was gone. It was believed that if we created functionality for users such as HRBP's we could introduce a certain "stickiness" to the product and it would mean the customer's would renew more often. This research would feed into both the NPS and Customer Churn initiatives.
The discovery took the form of:
Discovery interviews
A script was created and 15 users consisting of 4 heavy, 4 medium, 3 light and 4 new users were interviewed.
All user groups were interviewed because heavy users would support light users in their activities and therefore we could obtain further perspective on pain points.
Mapping workshop
The interviews generated ~250 data points which we conducted an affinity mapping exercise in the office to group the data points and generate themes.
Report on findings
Once the themes had been created, this information was then synthesised into a report. A summary of these findings and how they relate to NPS are shown in the next section.
Research quotes
Analyst of Design (Heavy)
We created this Orgvue starter pack to engage HRBP's to start using it. But even I find it challenging to proceed.
HR Lead (Medium)
Not everyone is technical, that's why you need different user levels so people don't feel intimidated when they log in.
Business Intelligence Intern (Light)
When I'm thinking of Orgvue, it's kind of terrifying because there are too many thing.
User satisfaction scores
At the end of the interview, a user satisfaction score was asked relating to brand, features, product and functionality. These numbers were surprisingly high given the results of the interviews, however, further questioning determined that there was loyalty to the product as it served a critical purpose and saved considerable time from conducting those tasks manually, however, there was deep dissatisfaction with the experience of the product.
Medium users gave the highest score because their usage of the product generally involved one or two tasks that they would learn and then repeat. There was no pattern based around frequency however. Different medium users would use the product at different frequencies. Whilst difficult to learn to start with, once learnt, they were generally satisfied.
As expected, Heavy users gave a high score. Most heavy users utilised the product every day, however their biggest pain point was in the support of light and new users. Because the product was complex to learn and understand, they felt they spent too much of their day supporting others.
Light users gave the second to lowest score with new users giving the lowest which was as expected. The burden on learning the complexities of the product made some users give up and go back to manual methods.
High level recommendations
The full report detailed over 20 recommendations, grouping these by departments within Orgvue. However, as this is a summary of this research I will just detail a few that we planned and actioned that year and were related to NPS.
Admin
Improve role based access, sharing and permissions capabilities. Give light users the ability to see summaries and accomplish simple tasks without the noise or complexities of the whole product.
This insight led to two projects, one that was completed before I left Orgvue and one that was still in discovery.
-
New home page. Whilst working on this project we also improved the site navigation, improving the consistency (most areas of the product had a different formatted header for example).
-
Simplification of permissions. (This project was in discovery when I left therefore can not go into any more detail).
Tracking and monitoring
This did not relate just to light users. In brief, whilst Orgvue was excellent at enabling a business to model themselves and save hundreds of hours compared to using Excel and Powerpoint, there was no ability to track progress to understand why an organisation project had been successful.
Tracking and monitoring was already on the roadmap, however, the data obtained from this research helped to increase the priority. At the time I left this project was still in discovery, hence I cannot divulge any further information.
Language
Again, this did not just relate to light users. In general the users felt that the language of the product was foreign to them and had to be learnt.
From the compelling data we gained from this insight, a UX writer was hired, reporting in to me, who's first major goal was to create a Language and Tone of Voice strategy.
The research for this initiative was extensive and resulted in far more detailed insights that enabled us to form a strategy to change the way we converse with our users in the product.
Unfortunately this document is confidential and cannot be shared. However, the strategy included a simplification and change of tone to various parts of the system by employing a tone mapping framework (for example, the tone of voice for an informational popup would be different to that of an error message). This enabled us to provide far more consistency in the manner in which we wrote copy.
The goal at the time of me leaving Orgvue was also to create a roadmap to update the entire product.
A further project my team were working on was a UX writing helper bot using AI. By plugging in the copywriting guidelines and choosing which function we were writing copy for, we could obtain a well formatted piece of copy which during initial tests, exceeded the writing capabilities of almost everyone on the team, with the exception of the UX writer. This, if completed, would have created huge efficiencies and saved a considerable amount of time.
Onboarding
Reduce the cost of onboarding by removing complexities including data onboarding and user onboarding. Bringing new users onboard takes more advanced users far too long and requires a lot of their time to train.
Currently no projects specific to onboarding are scheduled although a contextual help system is now on the roadmap.
The simplification of permissions will also speed up user onboarding.
Exports
One of the biggest pain points for light users occurs when exporting data and charts to Powerpoint. Due to slides having inconsistent amounts of data on them, the size of the visuals on each slide would be different - for example, a organisation chart with 5 levels on it would have a considerably smaller font than that of an organisation chart with 2 levels.
This resulted in users having to reformat the slides before being able to present them to their senior management teams.
This was a known issue and extra resource was allocated to the project team that owned this issue. In brief, greater configuration was added to Orgvue to allow customisation of the visuals before they were exported to Powerpoint.
Feedback
Whilst there was an existing mechanism for gathering user feedback via Pendo, generally users felt they weren't listened to.
Even when a piece of feedback made its way into the roadmap, there was no return process to inform the user that their desired change or functionality was now in the product. Whilst the release notes would detail such functionality, these weren't always read by the users or sometimes they didn't spot that it was related to their feedback.
This was easily solved. I created the concept of a Research Community and invited all the users to join. The community then received a monthly newsletter informing them of upcoming research opportunities and highlighted features that had been developed based on the feedback and research insights they had given us.
In addition to this, we improved the exchange of information with our CX team, detailing the feedback from users that was addressed with each new or updated piece of functionality. This allowed the CSM's to address this in their discussions with their clients.