Basic Transform Task: Regular Expression to Remove Emoji from Open Text Entry Verbatim | XM Community
Skip to main content

I am working with the basic transform task in a workflow to export survey responses daily. We are using the basic transform tasks Replace function to strip out any emoji from the response and replace with an asterisk *. The replace functionality has a Regular Expression option to find text via regex. I currently have the following expression in my Replace Value transformation.

 

((\x{2700}-\x{27BF}]||\x{E000}-\x{F8FF}]|\x{D83C}}\x{DC00}-\x{DFFF}]|\x{D83D}}\x{DC00}-\x{DFFF}]||\x{2011}-\x{2017}]||\x{201A}-\x{201B}]||\x{201E}-\x{26FF}]|\x{D83E}}\x{DD10}-\x{DDFF}])

 

 

Most emoji are getting successfully masked, but there are some that are not getting masked. See the emoji below.

 

💥,👍,🤬,👍,🥰,🥳,🤕

 

I used Regex101.com to confirm, and it looks like these emoji should be matching the regex we have right now. Anyone dealt with a similar issue or know of a better regex to use for identifying all emoji?

 

 

Try using below regex: 

/\p{Emoji}/gu

 

Try using below regex: 

/\p{Emoji}/gu

 

Hi Shashi, thank you for the suggestion! Unfortunately, this regex is not valid for Qualtrics’ basic transform task. See the failure in the workflow results below.

 

Replace value FAILED OPR_9 Error encountered performing transformation: unable to compile regex /\p{Emoji}/gu: error parsing regexp: invalid character class range: `\p{Emoji}` Value to find /\p{Emoji}/gu , Use regular expressions (regex) true , Value to replace **

In that case, you can try following approach: On new response event trigger, inside the code task, run the below script to get text with emoji replaced with *:

const result = text.replace(/\p{Emoji}/gu, '*');

Now, using the update response API, update an embedded data with value in result variable. Use the embedded data in the extract responses task.

In that case, you can try following approach: On new response event trigger, inside the code task, run the below script to get text with emoji replaced with *:

const result = text.replace(/\p{Emoji}/gu, '*');

Now, using the update response API, update an embedded data with value in result variable. Use the embedded data in the extract responses task.

Hi Shashi. I was trying to avoid JavaScript at all costs but I think you are right that this will be the only way to accomplish this until Qualtrics updates their acceptable regex formats.

 

I first attempted to do the code task as you described, but the code task seems to use ES6 syntax and does not accept ES9 syntax that allows for the /\p{Emoji}/gu regex. While it works in the code task test environment, actually running the workflow causes a never ending re-try.

 

 

We have instead had to add it to the survey question JavaScript which is the last place we wanted to implement this, but here we are. See the code we are using below. It sets a new ED field with the cleansed text. It also handles errors with back ticks/double quotes entered by the participant that may break the JavaScript. We will still be doing more testing but think this is the only path for now until Qualtrics updates their engines. We may still run into issues for some users on older devices/browsers that do not support ES9.

Qualtrics.SurveyEngine.addOnPageSubmit(function() {

    let EDFieldName = "Example_Field_Name"

    let input = this.getChoiceValue(0)
    let backticksRegexPattern = /`/gmu
    let emojiRegexPattern = /\p{Emoji}/gmu

    input = input.replace(backticksRegexPattern, "\"")


    try {
        input = input.replace(emojiRegexPattern, "**")
    } catch (e) {
        console.error('Regex replacement failed due to out of date browser not supporting unicode character class escape')
    }

    Qualtrics.SurveyEngine.setJSEmbeddedData(EDFieldName, input)

    console.debug('Set ' + EDFieldName + ' to ' + input)
})

 

Leave a Reply


Terms & Conditions