top of page

NPS: Deep dive

(reading time ~ 4 minutes )

Whilst Orgvue had just started using Pendo for analytics and gathering NPS data, it had not been set up to capture any information that would help us identify areas of the product to target, or clients that we could lend extra help to in the form of research and customer support. This was corrected shortly before I joined and gave us a bench mark for the areas of the product that respondents cared about the most.

The analysis and research performed during this initiative was a combination of quantitative, mostly using Pendo data and also qualitative interviews and data captured from our CX team who spoke to the users on a more regular basis.

We began the analysis with the following areas before rolling this data into a report.

nps trends.png

Historical trends

What did the NPS look like over the course of the year?

account trends.png

Account trends

Have certain accounts shifted from positive to negative or vice versa and are there any accounts particular heavy with detractors.

interview.png

Qualitative feedback

Are there any particular areas of the product that are brought up the most in qualitative feedback?

Historical trends

The data for this analysis was obtained by utilising Pendo. Data was exported to excel and grouped.

The trend saw a general positive increase over the course of the year with a large leap in Q1 of FY24, however, for this quarter, I only had half a quarter's data by the time I left Orgvue.

Screenshot 2023-06-02 at 19.09.33.png

It's also important to understand why there was an upward trend. Were there any particular events that occurred? Does the quantity of the data that made up these trends match. 

As we can see from the table below, whilst  Q1 '24 is only half a month's data, the other quarters' response rates remains relatively stable throughout

Screenshot 2023-06-02 at 18.46.45.png

The following table shows how the NPS was measured by customer type. We can see a far higher score from consulting partners (companies that use the Orgvue software for projects with their own clients). This was to be expected as the product saves them considerable time and almost all users are expert.

However, as we can see in Q4 '23, we're starting to see spikes now and can investigate what these are attributed to. In the case of the consulting partner spike in this quarter, it was investigated and found it was mostly because we had increased their enablement support.

Screenshot 2023-06-02 at 19.16.25.png

Lastly we can plot events to points in the historical chart. In August 2022, we had identified through the early analysis of this NPS initiative, that one customer who had more than 20% of the user base, was hugely dissatisfied with the product. They had received a poor implementation from a consulting partner, their custom functionality was not performing and the system kept crashing. 

Through this analysis we were able to identify that client, rewrite their scripts and provide extra support for them, allocating resource to help them understand how to perform their work in a more efficient way. Whilst NPS is anonymous to the user, we know from further analysis that the positive spike in NPS for direct customers in Q3 '23 was partially due to less detractors from this client (we received just over 30% less detractor votes from this client).

We also know that the new Home Page went live in January 2023, which we believed, through our research and discovery, would be of great benefit to more novice users. Again, being that NPS is anonymous to the user, we cannot categorically confirm that the spike in direct customer NPS is due to this, but there is a strong likelihood and qualitative (CX and UX follow up interviews) and quantitative (Pendo thumbs up/down votes) feedback around this feature has shown very positive results that users are finding the ease of navigation a real benefit.

A further contribution to the positive NPS spikes in Q3 and Q4 was the updates to the export functionality. Export improvements have occurred every release over the previous fiscal however the biggest user requested improvements around font control and colour palette control occurred at the end of quarter two. A survey that was sent out at the end of Q4 indicated 38% of respondents felt improvements to exports was the area that mattered most to them. 

NPS account trends

The purpose of this research and analysis was to see whether there were any patterns in accounts that were trending downwards and conversely any that were trending upwards.

Unfortunately the information contained within this area of the portfolio is very brief as I cannot divulge information about Orgvue clients and is therefore just summarised in this section.

The data was again obtained via Pendo and was categorised into the following four areas:

Negative

Accounts that have ranked lower in the most recent quarter than in previous quarters.

Positive

Accounts that have ranked higher in the most recent quarter than in previous quarters.

Mixed

Accounts that have fluctuated between their statuses over the last fiscal year.

Stable

Accounts that have maintained the same status over the course of the last fiscal year.

The following graph shows the trends that were identified from this analysis. This helped us to pin point particular clients that required extra focus from our CX teams and enabled us to dig further into issues that might be occurring for certain clients.

account trends graph.png

Qualitative feedback

Through the previous two quarters we had asked CSM's to map feedback to a number of different categories in their discussions with users. We would then use this data along side the themed responses from Pendo to be able to map areas of concern that would drive our roadmap. It should be noted that whilst valuable to validate through already occurring conversations, this feedback provided very little surprises.

The following data was obtained from 130 users, of whom 15% were admins.

NPS Qualitative feedback.png
bottom of page