@lillianc You're welcome! I wish I had the opportunity to run more tests, however I've taken a new job elsewhere. In my new role I do plan to run some subject line tests, but it may be a while until I have the opportunity. I'll keep this forum posted.
@VirginiaM each of the questions on the Course Eval have a 1-5 scale, so the points on this widget are the average of the ratings. 1=Strongly Disagree...5=Strongly Agree Please note that on that widget, the scale range does not begin at 0! The lowest Fall 2016 figure is above 4points...
@sbrons1 I'll have to try that out. It seems like a perfect way to do some testing. Thanks!
@JasonHill I have forms set up for: * Course Evaluations * Tax Intercept (online form that allows students who owe us money to appeal us digging into their state tax return) * Employee evaluation (somewhat like a 360 but without the added 360 feature) * Student Developed App contest form * Strategic Planning brainstorming activities * Registration forms for various events * Academic Program Review * Online data collection for intervention where faculty/staff made phone calls to 'at-risk' students to provide them with support to stay on track (we call it Project Early Success) * Data request forms for our office (IR) and the registrar * Clinical evaluation form that a proctor completes based on observations of students - used for accreditation of a healthcare program * many more...
@Dax If you go to your Dashboard Settings page and add the metadata field 'Start Date', and then add this as the denominator of your Custom Metric, that might do the trick of dynamically calculating the response percentage based on the count of survey starters.
@MichelleC I think I spoke with our rep about setting it. It's been a couple years ago now...
I appreciate this @JenF ! I'm going to set this up tomorrow.
@AlexB if you want to collect data from the same survey over time, but are less concerned with sending to same group of people, or your group is slightly different each interval, the setup is really easy. I have such a project set up for Course Evaluations. I'm not concerned with linking a individual person's responses over time, instead I'm concerned if the overall 'satisfaction' of various factors is going up or down over time, or is drastically different for certain groups, course subjects, instructors, etc. So, I have a single project that I use every semester, and this has a corresponding Vocalize dashboard. Each iteration of the course evaluation period, I load a contact list with the students, and the key here is to have a variable in the contact list that includes the semester (and all other factors we want to monitor). Once you have several iteration of responses with several different 'semesters' of data, you can create longitudinal widgets that show your 'satisfaction'
The jacket from the Summit earlier this year came in handy when I went for a jog on a brisk Salt Lake February morning... I was lightheaded after my job from the altitude, but I was at least somewhat warm...
@Poppy go into your dashboard, click settings to get to the page where your survey is mapped. To add the Text Topics, scroll to the bottom of the list and to the left you'll find the 'add field' button. Add a field, find one of your Text Topic items in the list where it currently says 'Not Mapped', and then change the field type to Text Set. Then give this a label, click save and go see if you can create a widget on your dashboard. If it looks the way you like, go back to settings and add the rest of your Text Topic items in a similar manner.
@Goldie I think you should be able to do this by creating a Pivot Table widget. You can add the people in the rows so you have a list for your target group, then add the Test Name field in the column and the Passed Y/N embedded data field as your Metric, filtered to only display the Y result.
@Poppy, you'll want to make sure you load your contact list with columns that identify each user by Location and Department. On your Dashboard, click the 'Share' icon, then set up a role based on their Location and Department. The role needs to point to the embedded data field in your contact list. In my attached example, I'm allowing faculty access to their course evaluation responses by creating an 'Instructor' role that points to the 'Instructor Username' field in my embedded data. You'll notice in the screenshot that I'm only allowing them to 'Export' the data (essentially allowing them View-only access), and they can see 4 of the pages of my dashboard. !
@Dax, you need to go into the 'Settings' and find that variable and change the Field Type from a Number Set to a Numeric Value. I'm assuming that your CSAT score does in fact have a limited range, but here you want it to act as if it's a raw number instead of a number in a preset list.
@Dax The Vocalize dashboard only displays completed surveys whereas the Data & Analysis tab in the Research Suite show pending surveys not yet submitted. Also, @AlexB is correct that there is a little bit of lag on Vocalize, so a simple page refresh should help bring Vocalize counts up to date if responses are rolling in.
You want to create a Top / Bottom box widget... In Vocalize, add a new chart widget, add two metrics that begin as a 'count'. Click on the first and you'll get a menu that allows you to select a question. Select a question/Field with some sort of scale that makes sense to combine the top 2 and bottom 2 (or whatever number of responses that makes sense). In the same menu, change the Metric type to Top Box/Bottom Box and then adjust the slider for 'Box Range' to '1 to 2'. Then on your second metric, select the same question/Field, and again change the metric type to Top Box/Bottom Box, but adjust the slider to '4 to 5' (in my example with a 5pt likert scale). You'll also want to make sure your Labels are descriptive, so change the text here as well. See attached.!
@MichelleC For all of our users, the default from is a generic departmental email address instead of the noreply@qemailserver... I've asked for it to be whitelisted about a half dozen times now, and I'm just not sure why the original whitelist doesn't 'stick'. I curious if I just need to put it on my calendar to request it to be whitelisted at some regular interval... Any ideas?
You know @AnthonyR , I've looked at response time of day for students and there are definitely some interesting things. I send out about 150K course eval emails each fall so I get a lot of data back each term... Peak response times are early morning, and mid afternoon. I typically send the Invite email at 6am on a Tuesday to avoid weekend/Monday email overload, but still get it in their hands early in the week. Then reminders every 3 to 4 days alternating between just before the two peak response times. @KatC I've got another 150K+ distribution set for next Tuesday... Maybe I'll do a couple different tests on Subject Line again this semester. Would anyone on this thread want to get together offline later this week and hash out some ideas together?
Any specific examples that you found especially impactful?
@Dax - I'm not sure if the Denominator for this custom metric can be dynamic... You can certainly set the denominator to match the # of invitations sent. However, I'm not sure you'd want to have your denominator match the # of started surveys. You'd have to constantly change that number, and I'm not 100% sure that Vocalize will notice the difference between Survey Starts and Completed Surveys.
If you are sending a survey using a Contact List, add a column for 'Sample Size' where each row's value is equal to the number of people in your contact list. Then, add a 'Number Chart' widget that points to a Custom Measure that you create in the 'Custom Metrics' tab. Your custom metric should be the 'count' (of responses) divided by the 'average' of your 'Sample Size' variable. Since all respondents will have the same value here, the Average of all of these will match the total potential respondents in your Contact List. Custom Metric = Count / average('Sample Size') If not using a Contact List, or you already have the survey in the field, you can simply create this Custom Metric, but instead of taking Count / 'Sample Size', you can simply pop in the total number of people on your Contact list. Custom Metric = Count / ####
Already have an account? Login
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.
Sorry, our virus scanner detected that this file isn't safe to download.
