Can Qualtrics import massive stimulus data from an online csv file and randomly loop to people?

Upload the csv to a database table on an online server. Write a web service to return 100 of 10000 ids and their associated fields (700 embedded data fields). Pipe the 600 fields into your loop & merge setup (100 rows, 6 fields). Pipe the loop and merge fields into your questions as needed.

Hi Tom, thanks for your prompt reply. Could you please let me know how can I randomly return 100 of 10000 ids and their associated fields? And should I create 700 embedded data fields in the survey flow manually at first?

https://community.qualtrics.com/XMcommunity/discussion/comment/51245#Comment_51245Hi Tom, thanks for your prompt reply. Could you please let me know how can I write a web service to randomly return 100 of 10000 ids and their associated fields? I only know how to return all 70,000 embedded data fields now. Should I use the 'parameter' function?

As I mentioned earlier, you should import the csv into a database, then your web service script would execute an SQL query. In that you query, you would randomize the order and limit the returned rows to 100.

https://community.qualtrics.com/XMcommunity/discussion/comment/51250#Comment_51250Hi Tom, thanks for your explanations. I am sorry that I am new to SQL and JavaScript, so it is a little bit hard for me to understand how it works. Could you please explain a bit more to me?
When you say "write a web service script " and "SQL query", are you referring to the JavaScript codes I should write on the question (PICTURE 3) in Qualtrics or somewhere else (server?) ? I am curious about how can I connect the database (I guess it should be a QSL database on the server) with my Qualtrics question. Should I use JavaScript or use the Web Service function in the Qualtrics survey flow?

The script runs on the server. It should execute the sql and output the data as json.
Use a web service call from the survey flow to execute the script.

https://community.qualtrics.com/XMcommunity/discussion/comment/51264#Comment_51264Thank you Tom! It is clear to me now and I will work on the codes later. Now I have two follow-up questions to understand the logic.
I am wondering that when the 100 ids are called by the web service, will the 700 embedded data fields be defined automatically? According to my previous experience, I feel that I need to manually select 'All ' fields and then these fields would be added as embeded data fields (please see attachment, I did this according to the instruction here under the  'Set Up the Web Request Data' section, but I am not sure if this "SELECT" step is compulsory or not). Because I want to randomize the embedded field data from the online database for each participant, so I cannot pre-select these values. Or do you think I should define these 700 fields in advance (manually) in order to pipe them into the Loop and merge function?
In addition, as you mentioned that I should "Pipe the 600 fields into your loop & merge setup (100 rows, 6 fields)", does it means that I should copy & paste the combination of the names of the field data into the Loop and merge function set up (I found the method here under 'Create a Loop and Merge Data Field Sheet' section)? Or is there a better way to 'pipe' these 600 fields into Loop and merge setup that you are referring to?

The field names will be defined based on the json the web service script outputs. There is a dot for each object level in the json. Since you'll be writing the script you have control over what the fields will be named and their structure.
In general, you'll want to select All - it is a lot easier that way.
In your example above you'll have seven fields for 100 ids numbered 0-99. When I set up a loop and merge, I create an Excel spreadsheet with the fields. Your spreadsheet would have rows (loops) with a label 0-99 and columns with names (6 or 7 depending if you want to include id as a column). Then create a formula to populate the cells (the first cell would contain ${e://Field/0.id} and the last ${e://Field/99.Road}. Then just copy and paste all the cells into the first loop & merge cell.

https://community.qualtrics.com/XMcommunity/discussion/comment/51281#Comment_51281Hi Tom. Many thanks for your clear reply and examples. I see. Can I confirm with you about my understanding?
So basically, the URL I put in the Web Service of the survey flow is not a link to the original database, instead, it is should be a js. file or a .php file (which one do you think is correct or better?). In the .js file or .php file, I should write codes to achieve functions including (1) randomly choosing 100 ids and associated fields values from a CSV file that contains all 10,000 ids, (2) transferring them to a .json file with the structure and names shown in the ATTACHMENT 1.
Then I put the URL to the Web Service of the survey flow, click 'test ' and choose 'All' fields to include as embedded data. After all these, I copy and paste the Excel spreadsheet created in terms of your suggestions (see the top 16 rows in ATTACHMENT 2).
After setting this up, when each participant clicks on the survey link, the randomization function will run, and it will generate new 100 ids and store them in the 700 embeded data fields. In the end, I am able to export (1) all responses and (2) the associated values (and the order) of the loops presented to each participant (based on embedded data).
Please correct me if I misunderstand anything or if I miss anything that I need in order to achieve my goal.

https://community.qualtrics.com/XMcommunity/discussion/comment/51283#Comment_51283Yes, you’ve got it.
The script language depends on what your server supports. I usually use php/mySQL, but you could use nodejs if that’s what your server supports.

https://community.qualtrics.com/XMcommunity/discussion/comment/51286#Comment_51286Thank you so much, Tom. It is very helpful! I will start to work on PHP/MySQL to initialize the database. : )