I need to annotate a large dataset in batches, so that each question is seen by 5 people | XM Community
Skip to main content

Basically say for e.g we have around 10,000 samples, and we need each instance/row to be annotated by 5 people. Problem is , obviously we can't upload the entire dataset to the qualtrics portal, we need to find a reasonable chunk size, say 50 or 100 and then annotate in batches for 5 people.

Is there any provision in Qualtrics to make it in multi-batch format, bc the only other alternative I'm seeing is copy pasting 50 sizes of 200 or sth like that and keeping track of which ones done manually which is very tedious. If you get my query, can you help with that, it will mean a lot!

The simplest solution would be to use a backend server where your stimuli are hosted and randomly pull 5 using Web Service in the survey flow.

@ahmedA   where would I do that? Like hosting the data.

 

So heres the scenario:

 

So I have the loop and merge configured such that each person sees each of the data and I can make a quota when publishing the survey so that its only filled by 5 respondents.

 

The problem is dividing the 10,000 samples into X amount of surveys so that each person when scoring only gets 50-100 samples to annotate per batch to reduce cognitive load, etc . Is that feasible?

 

Yes it's possible. But the exact answer would depend upon what your stimuli are, what is your question format, how do you plan to collect data, what is the amount overlap acceptable. 

In general terms, if you can convert your dataset to  JSON format, then upload to the Qualtrics library. Alternatively, you could look at loop and merge. While you can't have 10k rows, you can have 100 rows and 100 columns. 

Leave a Reply


Terms & Conditions