Archiving and purging XM directory

No response on this one - can someone advise if possible?

Hi Ingrid_M,
I do this exact process for a recurring survey we run which also follows users over their journey. I solved it by writing a script which uses the Qualtrics API to do the following:

1. Get all users from a specified mailing list


2. Iterate through each user in the list and update their details, assigning the fields to be purged with a blank value

I can't share the code I am using, but hopefully the linked docs will guide you. I am using Python, but you could probably create a weekly Workflow task on Qualtrics and use the built-in WebService function and some Javascript to make the automation happen.
Good lucK!

Hey @Ingrid_M (and anyone who finds this thread in the future),

This has effectively been solved recently with a new feature called “private list data”:

https://www.qualtrics.com/support/iq-directory/lists-tab/creating-mailing-lists/#PrivateListData

 

Basically if you set your mailing list to use private data, any embedded data used is exclusive to that mailing list and does not update the contact’s embedded data at the directory level. There is no need to purge existing data anymore 🎉 I am still testing this out myself though, so please do your own due diligence for your surveys.

There is an edge case to be considered - if your survey references embedded data that is not in your private mailing list but does exist for that contact at the directory level, it will pipe in the value from the directory (rather than default to a blank value). If your survey uses embedded data to create lists with dynamic length, you will want to double check that you don’t accidentally trigger this behaviour if you plan to rely on private lists. I will put an example at the bottom of this post if it is relevant to you and you’re unsure what I mean.

 

A HUGE thank you and congratulations to whoever developed this at Qualtrics, this is going to save me and my team lots of time and reduce the likelihood we make an upload error when running our repeating surveys. Here king/queen, you dropped this 👑

 

=== Edge case example, skip if irrelevant to you ===

You have a survey that asks users to leave feedback on the movies they watched each month. You have a multiple choice question which lists Movie1, Movie2, Movie3 etc, which is populated from the embedded data in the user’s contact.

A user watches 3 movies in January, and you do not use private lists. Movie1, Movie2 and Movie3 are now all embedded fields for this user at the directory level. The MCQ displays Movie1, Movie2 and Movie3. These values can now be accessed across the entire directory, meaning if you create a new mailing list with that contact on it, it will already have Movie1, Movie2 and Movie3 populated.

The same user then watches 1 movie in February, and this time you do use private lists. The MCQ displays Movie1 (with the relevant value for February), but also displays Movie2 and Movie3 with the values from January. Just to clarify, this should be the intended behaviour and is not a bug or oversight from the developers.

The simplest solution is to just purge all of the relevant embedded data fields across the entire directory, and make sure you exclusively use private lists for your survey. This should prevent the creation of any leftover values that you don’t want to accidentally pipe in when they are no longer reelvant.

Hey @Ingrid_M (and anyone who finds this thread in the future),

This has effectively been solved recently with a new feature called “private list data”:

https://www.qualtrics.com/support/iq-directory/lists-tab/creating-mailing-lists/#PrivateListData

 

Basically if you set your mailing list to use private data, any embedded data used is exclusive to that mailing list and does not update the contact’s embedded data at the directory level. There is no need to purge existing data anymore 🎉 I am still testing this out myself though, so please do your own due diligence for your surveys.

There is an edge case to be considered - if your survey references embedded data that is not in your private mailing list but does exist for that contact at the directory level, it will pipe in the value from the directory (rather than default to a blank value). If your survey uses embedded data to create lists with dynamic length, you will want to double check that you don’t accidentally trigger this behaviour if you plan to rely on private lists. I will put an example at the bottom of this post if it is relevant to you and you’re unsure what I mean.

 

A HUGE thank you and congratulations to whoever developed this at Qualtrics, this is going to save me and my team lots of time and reduce the likelihood we make an upload error when running our repeating surveys. Here king/queen, you dropped this 👑

 

=== Edge case example, skip if irrelevant to you ===

You have a survey that asks users to leave feedback on the movies they watched each month. You have a multiple choice question which lists Movie1, Movie2, Movie3 etc, which is populated from the embedded data in the user’s contact.

A user watches 3 movies in January, and you do not use private lists. Movie1, Movie2 and Movie3 are now all embedded fields for this user at the directory level. The MCQ displays Movie1, Movie2 and Movie3. These values can now be accessed across the entire directory, meaning if you create a new mailing list with that contact on it, it will already have Movie1, Movie2 and Movie3 populated.

The same user then watches 1 movie in February, and this time you do use private lists. The MCQ displays Movie1 (with the relevant value for February), but also displays Movie2 and Movie3 with the values from January. Just to clarify, this should be the intended behaviour and is not a bug or oversight from the developers.

The simplest solution is to just purge all of the relevant embedded data fields across the entire directory, and make sure you exclusively use private lists for your survey. This should prevent the creation of any leftover values that you don’t want to accidentally pipe in when they are no longer reelvant.

I love seeing people test new functionality and hearing how it works, before I have chance to test it.  So thanks from someone who did find this updated thread!