Roast my workflow, open for suggestions! [Loop & Merge, Multiple Surveys in Parallel, MTurk] | XM Community

Roast my workflow, open for suggestions! [Loop & Merge, Multiple Surveys in Parallel, MTurk]

  • 11 September 2022
  • 3 replies
  • 91 views
C
Badge +1

So I'm using Qualtrics with MTurk (Amazon Mechanical Turk, the human annotation job board).
My task is to get 1000 items of data annotated by 20 people. I've split the 1000 items into smaller bins to make survey creation easy. Please roast my workflow and suggest points where I could automate.

  1. I have created a reference Qualtrics Survey (RS1), (a) this has a loop and merge. I usually copy the data from a bin to this loop and merge. (b) I am using a quota system based on the screening questions.

  2. I make a copy of RS1, S1 copy and paste the data for a bin to the loop and merge. Make sure quota is reset, then publish.

  3. I copy the S1 distribution link, edit the project on MTurk end (replacing the S1 link) and publish.

I have to repeat Step 2 and 3 for each bin. How can I improve this to automate things so I can run multiple batches easily?

3 replies

bgooldfed
Rank
Userlevel 5
Badge +25

Hi Cyril,
I think you've got this locked up pretty tight - assuming you're using an anonymous link to share the project in MTurk, you'll be stuck having to edit the contents of the Loop & Merge manually unless you want to get into some API scripting which may or may not be successful.
My reasoning being that the only way I know of creating dynamic question content is via embedded fields and personalised links (not anonymous ones). The API, meanwhile, only allows you to update a survey's name/activity, not its content.
The only way "around" this would be to export your template, attempt to automate the editing and then import it back in.
This API call will accept QSF, TXT or DOC files as a template. I am not familiar with templating from scratch, but the one time I did export a QSF and take a look, it was a very verbose JSON file so I'd maybe suggest investigating TXT or DOC. You can find everything about preparing templates here. I do not know if Loop & Merge is supported via TXT/DOC format.
Assuming it is, a successful workflow might look like this:

  1. Run script to create project template, containing the desired data bin as a Loop & Merge

  2. Script makes the API call to import the template

  3. Get the ID from the successful API response and create your anonymous link (all anon links use the same format, so you can make this yourself: https://DATA_CENTER_ID.qualtrics.com/jfe/form/SURVEY_ID). Thanks to this comment for the tip.


I imagine there will be a lot of troubleshooting to get something like this working. By the time you solve it, it may have been quicker to just stick to your current workflow if its a one-off thing.
Good luck!

C
Badge +1

Hi bgooldfed thanks for the reply!
So I like your idea of personalized links and then routing people. Does qualtrics allow us to create personalized links or do we need to do it through contacts?
Also, I like your steps and the QSF file. I will look into this to see if it's possible.

bgooldfed
Rank
Userlevel 5
Badge +25

Hi Cyril,
You can generate personalised links within Qualtrics but it does require you to upload the contacts you want to send it to. Head to your Survey -> Distributions and there will be an option for Personalized Links. There's also an API for it.

Leave a Reply


Terms & Conditions