Chat GPT Integration Use Cases

I have been exploring that task a bit and I think it has a lot of potential when it comes to building a database of categorized comments. I have been using it in conjunction with the Microsoft Excel task to save the ResponseID, the Open Ended response, and the ChatGPT categorization output to an Excel document.

I think that the categorization is probably most effective for analysis when specific topics are included in the query. Separately, I have given ChatGPT a couple hundred previously collected comments all at once and asked it to identify all the topics within. I then used that response to develop a topic library and asked the ChatGPT task in Qualtrics to look for those specific topics when it runs for newly created responses, separating them with a comma in the output.

TextIQ uses keywords and logical strings to categorize the comments. "X" near "Y" but not near "Z". This will categorize the comments that meet the definition of the topic, but the definition can be time consuming to build and might not always get it right because of the different contexts in which a keyword might be used, even if the topic definition is robust. For example, it can be tricky for a telecommunication company to use keywords to distinguish between a device charger, a phone not holding a charge, and being charged for a phone bill.

ChatGPT works differently and would probably be better at distinguishing between the different contexts in which "charge" is used and without as much set-up. However, you get more with TextIQ than just categorization as it is fully integrated in Qualtrics versus having to build ChatGPT output in an Excel document. Having the TextIQ assigned topics available within the Qualtrics dataset and reporting tools is really helpful, TextIQ in the Survey Flow is great, its sentiment model is really good, and it supports topic hierarchies. 

I think ChatGPT is helpful to develop Topic libraries, so you could use it to get started and then build definitions for the identified topics within TextIQ. I created an account at https://chat.openai.com/auth/login and then asked ChatGPT the following, though it could use some refining:

"Summarize the sentiment and experiences expressed in the following comments into Topics. In your reply, include a count of how often each Topic was touched on and sort the Topics with most mentions at the top and fewest mentions at the bottom. Also include a brief description of each Topic.

[Comment1]
[Comment2]
[Comment3]
etc."

ChatGPT returned a summary within a couple seconds. The “counts” weren’t totally right but the topics it identified were solid.

It looks like ChatGPT does support Vietnamese so I think it should do okay in distinguishing between different contexts. I have found the "Actionability" field produced by TextIQ to be pretty useful for identifying comments that could use follow-up, but I fortunately don't see sarcasm too often in the survey responses I deal with.

You can include multiple ChatGPT tasks in a Workflow before the Excel task. Then, when configuring the Excel task, output from the ChatGPT tasks can be configured as separate columns in the Excel document. So you could have the following columns in the Excel document if you had 3 ChatGPT tasks as part of the Workflow: ResponseId, Open Ended Comment, ChatGPT: Topic Categorization, ChatGPT: Comment Sentiment, ChatGPT: Sarcasm Indicator.