How can I mark survey responses with skipped questions?

BrianVisconti
BrianVisconti Florida, USACommunity Member Qubie ✭
edited April 5 in Custom Code

I have a large survey with many branches and logics, which will collect responses from about 2,000 panelists. I cannot force responses to all items, but I do "Request response" on each item.

What I need to know for each survey response is, did the respondent skip any prompted items without answering? One way I thought to do this was to score the prompt question, when Qualtrics asks "There are 2 unanswered questions on this page. Would you like to continue?" and they can choose "Answer the Questions," or "Continue without answering," I just want a single piece of embedded data, or a scoring method or Javascript, anything that will tell me whether or not a participant chooses "Continue without answering" at any point in their survey, so that it can be marked Incomplete.

Reason being, I need Qualtrics to ONLY increment the completed surveys with no skipped items from the quota. I will not be able to just export the data and look for blanks because we need to know in real time whether a survey is complete or not.

Answers

  • ahmedA
    ahmedA IndiaCommunity Member, XMPN Member Wizard ✭✭✭✭✭
    edited March 29

    I'm not sure if this exactly matches your requirements, but while downloading the data, there is an option to mark "seen but unanswered" questions with a specific value (defautls to - 99). You could look at that.

  • BrianVisconti
    BrianVisconti Florida, USACommunity Member Qubie ✭

    Hi Ahmed,

    I did see something in the forum about that, but again, it can't be done in real time. The survey site sending out the incentives for completing the survey need to know whether a participant did or did not complete the survey when they hit "Submit". It can't be done on an export-by-export basis.

    I'll honestly very surprised if the functionality does not exist for Qualtrics to say "This user completed 98% of the items in their survey" if there were 100 questions and they skipped 2. Or really, if it could just say "Yes" or "No" as to whether 100% of questions were answered. Even that would be enough, and it is so very little to ask.

  • ahmedA
    ahmedA IndiaCommunity Member, XMPN Member Wizard ✭✭✭✭✭

    Yes. this seems like something that should be available by default. Something that gives us a ratio of answered/seen. You could look at adding this JS to your header and see if it does the job for you. It creates two embedded variables answered and unanswered. The unanswered one will only be populated if they have not answered the question even after the response request. So, at the end of your survey, you could check to see if its empty or not:

    Qualtrics.SurveyEngine.addOnReady(function () {
        document.querySelector("#NextButton").onclick = function () {
            let page_ans = [],
                page_uans = [];
            document.querySelectorAll("div[id^=QID][class*=QID]").forEach((question) => {
                let chosen = question.querySelectorAll(".q-checked");
                if (chosen.length == 0) {
                    page_uans.push(question.id);
                } else if (chosen.length > 0) {
                    page_ans.push(question.id);
                }
            });
            let old_ans = String(Qualtrics.SurveyEngine.getEmbeddedData("answered"))
                .split(",")
                .filter((el, ind, ar) => {
                    if (ar.indexOf(el) != ind || el == "null") return false;
                    else return el;
                });
            let old_uans = String(Qualtrics.SurveyEngine.getEmbeddedData("unanswered"))
                .split(",")
                .filter((el, ind, ar) => {
                    if (ar.indexOf(el) != ind || el == "null") return false;
                    else return el;
                });
    
    
            page_ans.forEach((item) => old_ans.push(item));
            page_uans.forEach((item) => old_uans.push(item));
    
    
            old_uans = old_uans
                .filter((a) => {
                    if (old_ans.indexOf(a) > -1) return false;
                    else return a;
                })
                .filter((el, ind, ar) => {
                    if (ar.indexOf(el) != ind || el == "null") return false;
                    else return el;
                });
            old_ans = old_ans.filter((el, ind, ar) => {
                if (ar.indexOf(el) != ind || el == "null") return false;
                else return el;
            });
    
    
            Qualtrics.SurveyEngine.setEmbeddedData("unanswered", old_uans.join());
            Qualtrics.SurveyEngine.setEmbeddedData("answered", old_ans.join());
        };
    });
    


    Right now it only works with multiple choice and matrix questions, but it can be easily extended to others also.

  • BrianVisconti
    BrianVisconti Florida, USACommunity Member Qubie ✭

    Hi Ahmad,

    That sounds like it could work! I do also have some questions that use sliders and a couple that use a pair of dropdown menus (like "Select the Month and Year" items). I'll try it as it is, but can I edit the code to include those?


    Brian

  • ahmedA
    ahmedA IndiaCommunity Member, XMPN Member Wizard ✭✭✭✭✭

    This line:

    let chosen = question.querySelectorAll(".q-checked");
    

    Basically checks if a MC option has been selected and then pushes the question id to the appropriate array. You could add more checks for other question types.

  • BrianVisconti
    BrianVisconti Florida, USACommunity Member Qubie ✭

    OK. Is there also a reason this text would be coming through dead? I tried pasting from your original JS post from last night (I'm a javascript nubie, btw) and the terms aren't lighting up the way they usually do in the coding window. I am remembering to change Source to "code" before pasting, but I still just end up with an actual header full of javascript, which is making it impossible to test it.

  • ahmedA
    ahmedA IndiaCommunity Member, XMPN Member Wizard ✭✭✭✭✭

    Put it between these two tags in th header:

    <script>

    // Code Here

    </script>

  • BrianVisconti
    BrianVisconti Florida, USACommunity Member Qubie ✭
    edited March 30

    Ahhh OK so I've got the script in and I see the Answered and Unanswered variables in my data output. In each one, it's giving me a list of the question labels that were or weren't answered. Is there a way I can just turn that output value into a count for each?

  • BrianVisconti
    BrianVisconti Florida, USACommunity Member Qubie ✭
    edited March 31

    @ahmedA

    So it looks like this is in the ballpark, but the script seems to only count the Answered and Unanswered questions out of the entire survey, not just the items each person has seen. This makes trouble because depending on which of 3 groups they belong to, each person only fills out one section of the survey. Currently, it's listing items form the sections they don't see in the Unanswered variable.

    If there's a way to limit it to only the items each person has viewed, or better, if there's a way to count how many times people choose the option "Yes, continue without answering," then, problem solved.

  • ahmedA
    ahmedA IndiaCommunity Member, XMPN Member Wizard ✭✭✭✭✭

    @BrianVisconti

    1. Could you please recheck, because the script is not interacting with the server at all, but just with the questions that are on the page. So, if a user takes a branch, where let's say Q2 is not displayed, then the script will never come across Q2. Unless you are using JS to hide the questions, if that's the case, then things get significantly more complicated.
    2. I have a non-free function which highlights the unanswered questions based on the different validations. It can be modified to take also record questions that were initially unanswered, but then answered after a reminder. Please send me a message if you'd like to purchase it.
    3. I thought you were only interested in the ratio of answered/seen?


  • BrianVisconti
    BrianVisconti Florida, USACommunity Member Qubie ✭

    @ahmedA


    1. There is a JS "Withdraw from Survey" button on each page, done using a dummy question and a confirmation question, which are hidden by javascript.
    2. Yes, if it works on Qualtrics's platform messages, like the confirmation to continue without answering then I'd be interested. My first thought with this was, if I could just score those conditionally-appearing confirmations to skip items, then bam, problem solved; anyone with a score over X has skipped items. But, I don't think I can do anything of the sort with one of Q's internal questions.
    3. I'm interested in anything that provides me a pre-export datapoint that specifically indicates whether or not a user has skipped items without answering.


  • ahmedA
    ahmedA IndiaCommunity Member, XMPN Member Wizard ✭✭✭✭✭

    If you've got that hidden question, then that should appear, but not any other question. Could you please check and confirm if it also recording questions that skipped via branches and display logic?

  • BrianVisconti
    BrianVisconti Florida, USACommunity Member Qubie ✭
    edited March 31

    @ahmedA

    It's hard to tell because the labels it's using in here are not the labels I gave my items, they are named using block ID. I attached a screenshot of the Answered and Unanswered data output cells for one respondent.

    The respondent (me) skips zero questions, yet there are far more items in Unanswered than in Answered. For this survey section, there are 17 pages, each with a Withdraw item hidden by JS and a second "Confirmation to Withdraw" question hidden by a display logic.



  • ahmedA
    ahmedA IndiaCommunity Member, XMPN Member Wizard ✭✭✭✭✭

    Those are mostly sliders. Like I said earlier, at present its only capturing MC questions.

    Here's the one I was talking about which caters to different question types and also records the number of request response pop-ups.

  • BrianVisconti
    BrianVisconti Florida, USACommunity Member Qubie ✭

    So I would have to use this JS in each page of the survey for it to work?

  • ahmedA
    ahmedA IndiaCommunity Member, XMPN Member Wizard ✭✭✭✭✭