Restriction on embedded variable length as of March 11? Output getting truncated... | XM Community
Solved

Restriction on embedded variable length as of March 11? Output getting truncated...

N
Rank
Userlevel 3
Badge +10
  • nroot
  • Level 3 ●●●
  • 35 replies

I have a survey that has been running for months, and for part of it, it is constantly appending data to an embedded variable, which gets quite long sometimes (~100k characters). This has not been a problem until midday on March 11, when suddenly Qualtrics started truncating the embedded variable on subsequent responses (at least a dozen now). To reproduce this, create a survey with two embedded variables (say, test1 and test2), and set their values to long strings programmatically:
Qualtrics.SurveyEngine.setEmbeddedVar("test1","t".repeat(50000));
Qualtrics.SurveyEngine.setEmbeddedVar("test2","t".repeat(100000));

For me, the var "test2" is recorded as 70,000 "t" plus an ellipsis "...".

Is this an intended change? Or is this something to do with my account hitting a hidden data storage size limit or something?

icon

Best answer by AmaraW 20 April 2021, 19:44

View original

16 replies

A
Rank
Userlevel 7
Badge +21

Thank you for raising this issue. I also have a few surveys running where the emebedded data is more than 70K chars. Just checked them and also performed your tests.
It appears that ED variables are being truncated at 70K chars.
LaurenK can you please let us know whom we should get in touch with regarding this. Also, was there any announcement for this update.

LaurenK
Rank
Userlevel 7
Badge +13

Hi all - check out this section on our support page about Embedded Data. It states that the Embedded Data Name and Field each should not exceed 1024 characters. In the case that this information does not help answer your question, you'll want to reach out to our Support team, as they are best equipped to help! 😊

M

Hi LaurenK,
thanks for the mesage. We also experienced this issue of truncated embedded data since March 11th.
We're using an institutional account from our university to host an experiment and this change was not been notified to us in due time.
We will explore with our institution whether this constitutes a violation of contract by Qualtrics with our university.

Best
M

N
Rank
Userlevel 3
Badge +10

Yes, it's a shame that this change was not publicized, I lost several dozen subjects' worth of data as well. I suspect quite a few people use embedded variables for things like mousetracking, which can eat up a lot of characters quite quickly.

N
Rank
Userlevel 3
Badge +10

Just an update for those who search for this question. Embedded variables are now truncated at 50k characters, down from 70k; again without any update. I understand that this is potentially a corner case use of embedded variables, but I don't understand what's going on that this is changing multiple times with no documentation or updates.
I designed around the March 11 change by creating more embedded variables and doing nested if statements so that it would use a new variable if it was going to exceed 70k characters. But this code broke when the 70k->50k happened.
Does anyone know any more information about whether this number is going to change again?
In the meantime, does anyone have any workaround for this? Could you somehow call a survey from a survey flow that fills an embedded variable and returns the number of characters? Then you could dynamically change how you allocate your data to the different embedded variables depending on how long they are getting.

A
Rank
Userlevel 7
Badge +21

I suggest keeping you embedded variables to a max of 20k chars. This seems to be the limit on questions as well as on input text areas.
One solution you can look at is to call a webservice to update your data on a backend server. I've done it for files as big as 300KB (> 1 million chars). Send me a private message if you'd like to know the details.

AmaraW
Rank
Badge +48

Hi there! We have updated this support page to reflect the new size limitation. In response to issues surrounding exports and data processing, we released this update limiting embedded data to 15 KB when the data is recorded. Each kilobyte equates to approximately 1,000 standard (not special) characters. Any values exceeding this limit will be truncated. Our Engineering team is looking into the possibility of allowing these larger values to be exported as files. In the meantime, on a case-by-case basis, some brands may increase this limit by contacting Qualtrics Support.

A
Rank
Userlevel 7
Badge +21

AmaraW Thank you for sharing the update. However, isn't it weird that we can store upto 20K characters in text input question, but only 15K here?
Futhermore, questions are allowed a limit of 16MB (as per the response I received from Qualtrics' support), which would amount to 16 million characters, can't that be done for embedded variables also?

AmaraW
Rank
Badge +48

Hi @ahmedA! We conveyed your point to our team internally and they adjusted the Embedded Data character limit to 20KB. We've also gone ahead and updated this support page with the new limit.

A
Rank
Userlevel 7
Badge +21

https://www.qualtrics.com/community/discussion/comment/36792#Comment_36792Thank you AmaraW. This consistency will help a lot with managing the data.

N
Rank
Userlevel 3
Badge +10

Hi @ahmedA! We conveyed your point to our team internally and they adjusted the Embedded Data character limit to 20KB. We've also gone ahead and updated this support page with the new limit.

 

Hi @AmaraW ,

Sometime in the past few days, the byte limit seems to have been drastically reduced: I am now unable to store more than 5k characters in an embedded variable. Can you advise whether this is an intended change or a bug?

JoseS
Rank
Userlevel 7
Badge +20

Hello @nroot, this sounds like it may not be intended functionality. If you haven't already, I'd recommend reaching out to our Support team, as they are best equipped to handle an issue like this. 😁

N
Rank
Userlevel 3
Badge +10

Huh, seems to have been fixed in the past 24 hours, before I had a chance to open a ticket.

JoseS
Rank
Userlevel 7
Badge +20

Huh, seems to have been fixed in the past 24 hours, before I had a chance to open a ticket.

Glad to hear it was resolved! 🙂

D
Badge +1

Hi - I too am having this issue. When I attempt to upload a file to embed data into a survey, I am shown the following error message:

Edit fields in your file to be under these character limits:

Language: 20
First name: 100
Last name: 100
Email address: 254
Phone number: 20
External reference: 100
Embedded key: 200
Embedded field: 1,024

Upon reaching out to support (as suggested on this page), I was told they might not be able to resolve this issue after all (meaning that no one is able to upload data files that would include embedded fields over 1024 characters.) They encouraged me to share this post here as well.

Can a Qualtrics Admin Help? 

JoseS
Rank
Userlevel 7
Badge +20

@dtharp2 Thank you for reaching out! I recommend creating your own new post regarding this issue to expand your reach and increase the probability that someone with a potential answer will respond. 🙂 

Leave a Reply


Terms & Conditions