Qualtrics for annotation of text data | XM Community
Skip to main content

Hi everyone,
Qualtrics noob here, so I may be asking an obvious question. I appreciate your help.
I'm trying to use a Qualtrics survey for manual annotation of text data. That is, rather than trying to get the participants' opinions or demographic attributes, the survey should show them a passage of text and prompt them to answer some questions about it. I've written a toy survey with the appropriate questions and answer choices, and realized I can put the questions in a block and "loop and merge" to get one survey session to handle multiple pieces of text.
What I can't figure out is how to pipe the texts into the survey. In fact, I can't figure out where in the Qualtrics data model to store them. The texts aren't an attribute of the user (so an "embedded data" field doesn't seem to fit) and they need to exist before the responses do (so a "custom field" doesn't look right either). There are thousands of pieces of text with multiple human annotators, so I can't easily embed one block per passage in the survey.
Is this possible with Qualtrics? Any tips about how to make it work would be helpful. If doing it involves writing and integrating some external web service, I have plenty of programming experience and can do that (though I 'd rather keep it simple).

Your texts should probably be stored in a loop & merge field. You can set your loop & merge to display X loops (texts) to each respondent. You can copy and paste the texts from a spreadsheet into the loop & merge setup.

thanks! I'll check this out.

For anyone else who runs into this problem, I ended up using loop and merge fields together with a web service.

  1. In the first, introductory block of the survey, run a web service (under the "Survey Flow" tab), which fetches 30 texts to code. The web service returns a json document, formatted as a list of dictionaries, and I have the Qualtrics web service block set to parse this into 90 "Embedded Data" fields. There are 3 per example, containing the text and some metadata about what kind of text it is.

  2. The question has loop and merge turned on with 30 loops; each loop has three fields that refer to the corresponding embedded data fields.

As for the thing on the other end of the web service, I wrote a little flask app with a database. I tried pasting the texts directly into the loop and merge fields, but it got unwieldy very quickly with how many of them there are.

Leave a Reply


Terms & Conditions