How is Preference Share calculated for MaxDiff report? A Softmax function returns different probabilities than Qualtrics' default "Preference Share". | XM Community
Skip to main content

Qualtrics’ MaxDiff projects produce an output called “Preference Share”. It is defined as:

The Preference Share is a measurement of the distribution of preference that respondents have for the tested attributes of the maxdiff. The preference shares sum to 100 and can be understood as the likelihood that the attribute truly is the most preferred item.

 

My assumption is that Qualtrics is probably taking the “Average Feature Utility” it produces and passing it through a softmax function to converts the utilities into probabilities that sum to 1.

 

However, when I applied a softmax function (i.e., exp(Value)/sum(exp(Value)) ) to the Average Feature Utility values, they do not produce identical values to what Qualtrics produces. See below:
 

Average Feature Utility 1.87602 0.891439 1.004267 0.197074 -0.65923 -0.65549 0.068604 -1.63446 0.47346 -0.01903 -0.08351 -1.80165 -2.01015 0.082374 2.715848
Softmax of Average Feature Utility 0.185307 0.06923 0.077499 0.034573 0.014684 0.014739 0.030405 0.005537 0.045579 0.027854 0.026114 0.004685 0.003803 0.030826 0.429165
Qualtrics Preference Share 0.113599 0.086067 0.055882 0.041274 0.025717 0.028783 0.050811 0.032024 0.038879 0.039782 0.028486 0.030358 0.026629 0.05372 0.34799

 

Does anyone have an explanation of how Qualtrics is estimating / deriving it’s Preference Shares and why they may deviate from what you see when passing Feature Utilities through a Softmax?

see, if you find this helpful


@jbk Interestingly, that link indicates that the way Qualtrics estimates the probabilities is the exact same way I describe above, using a Softmax function, i.e.,:
 

and is derived by exponentiating the item utility and dividing that by the sum of all of the exponentiated items’ utilities.

 

However, it generates different probability which is quite puzzling. I don’t really know how to reconcile that.


Leave a Reply