NPS
(reading time ~ 6 minutes excluding sub case studies)
When I first started at Orgvue I was given 3 goals for the first year:
-
Improve the product's NPS to zero
-
Mature the UX function at Orgvue
-
Simplify the product
This case study will focus on the first one which is to improve the NPS. I've done my best to summarise this initiative, however, the strategy and results for this case study is quite extensive, therefore, if you require more detail, please feel free to contact me.
Upon joining Orgvue in May 2022, the NPS stood at -21 (negative 21). It was widely acknowledged within the organisation that there was a lot of dissatisfaction amongst users due to the user interface being extremely complex. The belief was that this was preventing wider adoption within our clients' businesses and this was further validated through subsequent research (along with no support for accessibility).
I was provided with two hypotheses for the low NPS and my task was to investigate, devise a strategy to improve NPS and implement it.
-
Customer churn correlates to NPS
-
Light users are largely responsible for the detractor scores
By the time I left Orgvue, just over a year later, NPS stood at 17 (positive 17) a difference of 38 points and I believe there is still more than can be achieved.
Initial research
After some initial research, I determined we needed to perform the following research tasks to help understand why the NPS was low. Note, this is in addition to introducing general UX best practices, improved collaboration with internal stakeholders, greater emphasis on research and raising the bar in the UI being created.
You can click the links below for a deeper dive into each area, however, below I will summarise how each area of research contributed to such a huge jump in NPS.
(reading time ~ 7 minutes)
Research initiative primarily aimed at the light users of the product to identify pain points via interviews.
(reading time ~ 3.5 minutes)
Determine who Orgvue are building their product for. Define and validate personas.
(reading time ~ 4 minutes)
Deep dive into the NPS trends via Pendo and consolidate feedback from disparate sources.
Summary of findings
The following section details at a very high level the findings and recommendations that came from each area of research. If you wish to see more detail, please click the links above for a deep dive into each area.
Light user research
-
Heavy users were surprisingly dissatisfied with the product due to the level of support they must give light users.
-
The act of setting up new users and creating permissions for them for certain projects was extremely complex and time consuming and a huge factor in the dissatisfaction of heavy users.
-
Most users didn't understand the language the product used. It was too technical and they felt they had to learn it to use the product.
-
Navigation around the product was difficult especially for light users. The heavy users wanted to be able to give them a simplified and customised navigation structure for the areas they would need.
-
Exporting data to Powerpoint was particularly painful.
Personas
The initial hypothesis that personas were based on frequency of usage was incorrect. Research within the light user initiative and subsequent specific persona research identified the following personas:
-
Analyst (Orgvue collaborator and influencer)
-
HR lead
-
Workforce planning lead
-
Business leader
-
HR business partner
-
Senior HR business partner
More information on how we arrived at these personas can be found on the Personas page.
NPS analysis & feedback
A deep dive into the NPS results was extremely enlightening:
-
Over 20% of the user base (~2000 total users) was with one customer and were predominantly HRBP's.
-
This one customer had a solution set up by a partner consultancy firm who created custom scripts that as the numbers of employees went up, didn't perform and frequently crashed.
-
If this customer was excluded from the NPS results, it created an 8 point positive jump.
-
A trend was identified that promoters generally used the readily available packs in Orgvue, where as detractors wanted greater flexibility and customisation.
-
24% of respondents gave passive scores. Converting these passives to promoters would increase the NPS by 16 points. Converting users with a score of 6 (highest detractor score) to a 7 (passive) would jump NPS by 5 points.
-
15 accounts were identified as constant detractors over 2+ quarters. Interestingly they had 95% more detractor responses than promoters.
-
46% of detractors cited the user experience as the reason for the poor score, 10% performance and 8% visuals and exports.
Results
The research detailed above was conducted over the majority of a fiscal year and therefore certain actions were more progressed in that time before I left Orgvue.
At the time of leaving, NPS had exceeded the goal I was given (which was zero) to positive 17, a jump of 38 points.
Whilst the NPS jump is impressive, it is caveated by the fact the response rate was considerably lower. We know from our research that detractors are far more likely to answer the survey than promoters (across a sample of 15 clients who were consistent detractors we saw 555 responses in the previous fiscal leading up to April 2023 where as there were only 30 promoters across those 15 clients).
As can be seen in the deep dive around NPS analysis, as certain events happened, the numbers for detractors dropped off the following quarter, therefore the analysis concludes these events to have had success. This was further validated by qualitative research via our CX teams with the users where we were informed that whilst far from perfect, they felt we were moving in the right direction, especially in the areas that mattered most to them. We also conducted surveys and Pendo validation popups (thumbs up/down) to assess the level of satisfaction on certain features.
The biggest shifts in the NPS could be measured on the following events (more detail can be found in the NPS deep dive):
-
Improvements to the export functionality to give the user greater control on the visual appearance of the slides
-
Release of the new home page which greatly improved navigation and simplified the experience in "getting to what mattered", especially for the light users.
-
Rewriting the data cleaning script for a particular client who was identified as the largest detractor (they had over 20% of the user base). More focus was applied from the CX team on this client as they had a considerable amount of HRBP's (previously identified as light users).