Using Qualtrics survey logic to eliminate redundancy in simple experiments

I just reviewed a Qualtrics survey for a 3×2 between subjects experiment which repeated 3 times within subjects. The survey designer was fairly inexperienced so it’s not surprising that they used 36 blocks with 5-8 questions each and had incredibly complex logic to accomplish this. Unfortunately, there was so much redundancy and complexity that I wasn’t able to error-check the survey instrument. I gave the student suggestions for how to go from 36 blocks with complex survey logic to 1 simple block with straightforward logic. My recommendations follow below in case others find it useful.

The stimuli were text-based (no images), so my suggestions here focus on that. 

Between subjects differences: Use Embedded Data. 

Embedded data allows you to assign conditions and specific text to individual participants. The conditions can be randomly assigned by simply using a randomizer in the survey flow. 

Here are the steps:

1. Edit survey flow
2. Add randomizer
3. Within the randomizer, add embedded data. In a 2×2 experiment, you should create 4 blocks of embedded data, one for each condition. I’ve only included the first two below but you get the idea. Alternatively, you can have two randomizers, one for each factor, and you can set the embedded data separately for each factor. 

That’s it for the survey flow. Now when you create the question block, you can insert the variable text you’ve assigned to each participant using “Piped Text,” “Embedded Data Field.” 

Repeated measures: Use Loop and Merge.

If you ask the same set of questions three times but only vary a few words in each set of questions, Loop & Merge can help you do that without having to copy all of the questions. The reason you want to avoid duplication where possible is because you may go back later and tweak the question. Then you’ll have to tweak it in multiple places and error can enter in. 

Within Loop & Merge, each row is a repetition and each column is a set of text that varies across the repeated measures. So if I wanted to ask the same set of questions about 3 different celebrities, I would delete Field 2, add the 3 names in Field 1 and delete rows 4 and 5. 

Bringing in the dynamic text for Loop & Merge is very similar to how we did it for Embedded Data. 

Again, you would use the piped text tool but this time you would select “Survey Question” to pipe in text from a previous question. 

That’s how you can go from 36 blocks with an incredible amount of complexity and redundancy down to one very simple block. I hope it helps. 

Aaron Charlton

Aaron Charlton, PhD, MBA is a marketing professional who currently works in industry for Away Clinic and Metascience & Marketing Lab and lives in Mesa, Arizona, USA. He is formerly an academic and still takes interest in improving the quality of research in the field of marketing.