Has anyone seen any documentation on Text Analytics NPS Impact Score | XM Community
Skip to main content
Michael_Cooksey
Administrator
22 days ago
Question

Has anyone seen any documentation on Text Analytics NPS Impact Score

S
Level 6 ●●●●●●
Forum|alt.badge.img+32
  • ScottG
  • Level 6 ●●●●●●
  • 241 replies

Hi community,

 

Been looking at NPS Impact Score within our Text Analytics model which an XM scientist helped us set up a while back when we got started with text analytics.  This provides us the overall impact particular topics are having on our overall NPS.  I have a colleague of mine who is seeking to better understand what it’s trying to show/calculated (they are more of a statistician than I am).

 

The formula for it is… NPS Impact Score = (Topic Count*(Topic NPS-Overall NPS))/(Overall Count-Topic Count). This is set up as a custom metric, but was a metric encouraged by Qualtrics to monitor text analytics.

 

The question my colleague has is why we include the topic count as both numerator and denominator; and to that I can’t answer (it seems ok to me, but this isn’t my speciality).  Surprisingly I recall a bit more documentation on the Qualtrics side when we introduced it, but now nothing seems to come up.  I’d like to have more confidence around this if we were to keep using it, but can no longer see any live Qualtrics documentation, if anyone has seen anything on this?  I have an offline document which confirms the formula, but doesn’t explain the metric.  Cautious maybe the approach is no longer recommended either.

Anyone also use this or see anything specific to ‘NPS Impact Score’.

6 replies

T
Forum|alt.badge.img+1
  • Tino_p
  • 6 replies
  • January 16, 2024

Hey @ScottG,

Qualtrics has some info on calculating NPS here, but i’m not sure its exactly what you’re looking for. Your formula seems correct if your goal is to calculate the impact of a topic on your overall NPS score. In the numerator, you are looking at the impact (+/- difference to overall NPS) and frequency (number of times the topic occurred). In the denominator, you are weighing the impact against the other topics. Thus, you exclude the topic you're looking at. 

Hope this helps! I recently built a free GPT on the OpenAI store that analyses NPS scores. Feel free to try it out and compare if you get to the same result. We’re always looking for feedback. 

M
Forum|alt.badge.img
  • monica.c
  • 2 replies
  • August 2, 2024

Hi @ScottG, I haven’t come across this metric before - would this calculation still work if one comment can be tagged with multiple topics?

By ‘Overall Count’ in the denominator, is this total number of comments or total number of tags?

S
Level 6 ●●●●●●
Forum|alt.badge.img+32
  • ScottG
  • Author
  • Level 6 ●●●●●●
  • 241 replies
  • August 5, 2024

@monica.c  this definitely works for comments with multiple tags. 

As for the count it should be the count of comments that mention that topic, not the number of total tags.  Noting the tag and number of comments containing that particular tag by Qualtrics should be the same count in theory, as each tag is only associated once - at least in our set up. 

M
Forum|alt.badge.img
  • monica.c
  • 2 replies
  • August 6, 2024

Hi @ScottG, thanks for your response.

 

I’m not sure I fully understand though. Would “the count of comments that mention that topic” not be the Topic Count rather than the Overall Count?

If we take a dummy example:

Out of 10 comments, 8 comments are tagged with Topic X and 4 comments are tagged with Topic Y.

In the denominator for calculating the Impact Score of Topic X, would it be 10-8=2 or (8+4)-8=4? Or neither, if I’ve misunderstood?

P
QPN Level 2 ●●
Forum|alt.badge.img+2
  • PAllen5
  • QPN Level 2 ●●
  • 4 replies
  • March 20, 2025

Resurrecting this thread to ask if anyone has run into the Topic Count and Record Count matching, resulting in a denominator of 0.

If you have, how did you rectify it?

HarryO87
Level 2 ●●
Forum|alt.badge.img+6
  • HarryO87
  • Level 2 ●●
  • 10 replies
  • March 28, 2025

I think I had something similar (but maybe misinterpreting). 

I created a custom field in my dashboard settings called ‘Topic Representation’ as follows: 

count of total responses / count of response * 1

In the above, the count of total response is all responses on my dashboard the count of response is filtered with my topic field and selected to ‘all’. 

I then created a widget on my dashboard (a number chart) and included my Total Representation custom metric. 

Finally I added a filter to my dashboard. 

Therefore, if I was to select ‘Topic X’ on my dashboard filter, my number chart widget tells me the percentage contribution to all NPS responses. 

Hope this helps/answers the question.

Thanks

Harry

Leave a Reply


Terms & Conditions