Has anyone seen any documentation on Text Analytics NPS Impact Score | XM Community
Skip to main content

Hi community,

 

Been looking at NPS Impact Score within our Text Analytics model which an XM scientist helped us set up a while back when we got started with text analytics.  This provides us the overall impact particular topics are having on our overall NPS.  I have a colleague of mine who is seeking to better understand what it’s trying to show/calculated (they are more of a statistician than I am).

 

The formula for it is… NPS Impact Score = (Topic Count*(Topic NPS-Overall NPS))/(Overall Count-Topic Count). This is set up as a custom metric, but was a metric encouraged by Qualtrics to monitor text analytics.

 

The question my colleague has is why we include the topic count as both numerator and denominator; and to that I can’t answer (it seems ok to me, but this isn’t my speciality).  Surprisingly I recall a bit more documentation on the Qualtrics side when we introduced it, but now nothing seems to come up.  I’d like to have more confidence around this if we were to keep using it, but can no longer see any live Qualtrics documentation, if anyone has seen anything on this?  I have an offline document which confirms the formula, but doesn’t explain the metric.  Cautious maybe the approach is no longer recommended either.

Anyone also use this or see anything specific to ‘NPS Impact Score’.

Hey @ScottG,

Qualtrics has some info on calculating NPS here, but i’m not sure its exactly what you’re looking for. Your formula seems correct if your goal is to calculate the impact of a topic on your overall NPS score. In the numerator, you are looking at the impact (+/- difference to overall NPS) and frequency (number of times the topic occurred). In the denominator, you are weighing the impact against the other topics. Thus, you exclude the topic you're looking at. 

Hope this helps! I recently built a free GPT on the OpenAI store that analyses NPS scores. Feel free to try it out and compare if you get to the same result. We’re always looking for feedback. 

Hi @ScottG, I haven’t come across this metric before - would this calculation still work if one comment can be tagged with multiple topics?

By ‘Overall Count’ in the denominator, is this total number of comments or total number of tags?

@monica.c  this definitely works for comments with multiple tags. 

As for the count it should be the count of comments that mention that topic, not the number of total tags.  Noting the tag and number of comments containing that particular tag by Qualtrics should be the same count in theory, as each tag is only associated once - at least in our set up. 

Hi @ScottG, thanks for your response.

 

I’m not sure I fully understand though. Would “the count of comments that mention that topic” not be the Topic Count rather than the Overall Count?

If we take a dummy example:

Out of 10 comments, 8 comments are tagged with Topic X and 4 comments are tagged with Topic Y.

In the denominator for calculating the Impact Score of Topic X, would it be 10-8=2 or (8+4)-8=4? Or neither, if I’ve misunderstood?

Leave a Reply


Terms & Conditions