Picture is the author attempting to replicate the red chip protocol with his children acting as lab assistants.
Prior to Brian Wansink being relieved of his duties due to scientific misconduct and his subsequent resignation, he ran the “Red Potato Chip Study” (Geier, Wansink & Rozin, 2012). The study claimed that people would eat fewer potato chips, and therefore be more healthy, if there were red chips mixed in with the pale yellow chips. The idea is that the red chips would act as a stop sign to signal to the chip eater’s unconscious mind that they had had enough. The study has long been discredited but Health Psychology for whatever reason (it may be a very good reason by the way) has not taken any action on it. Because it has not been retracted, the study recently gained new life by substantially propping up the effect sizes of a meta-analysis of nudge interventions published in PNAS (Mertens et al. 2022). One of the interesting aspects of the study is the huge effect sizes. The four effects from the paper that Mertens et al. (2022) chose to include in their meta-analysis were: d=3.0, d=2.8, d=2.1, d=1.9. These effects would normally be considered to be way too large for any kind of nudge intervention so it is indeed a suspicious situation.
The criticism
The Red Potato Chip Study was heavily criticized for numerical inconsistencies by James Heathers, one of the data sleuths who outed Wansink originally. I can’t attribute it but somehow people started talking about the possibility that the study never happened and that it might not even be possible to create red, stackable potato chips following the protocol in the study:
Richard Morey subsequently made three attempts to replicate the red potato chips: attempt #1 | attempt #2 | attempt #3. Despite his best efforts, including several deviations from the original protocol to give it more of a chance of success, he was unsuccessful in recreating a red, stackable potato chip that looked and tasted like the original chips except for the color difference which was intentionally manipulated.
Our follow-up to the criticism
Building on Richard Morey’s initial experiments, I assigned my class of 36 marketing research students to attempt to replicate the red potato chip stimuli. They were provided with stackable potato chips, red food coloring, and six weeks to complete the project working in teams of 5-6 students. They also had the advantage of learning from Morey’s approach. They were instructed to exactly follow the described experimental protocal (see screenshot above) but also to run trials with subtle deviations that Wansink may have done but forgot to mention when writing up his protocol. Some of their deviations included using a mold so the chips wouldn’t flatten out when drying, sprinkling salt rather than dipping them in saltwater, high temperature drying (rather than low), and using an air fryer. I was surprised to learn that air fryers are standard cooking appliances in midwestern college student apartments.
The results were not incredibly promising. Here is a preliminary overview:
- It is very easy to get the red color
- It is not possible to get the shape right without a mold. Remember that the chips need to be stackable. The original protocol doesn’t mention a mold.
- No team was able to get the flavor right by dipping the chips in the saltwater. The chips simply lose more salt than they gain. The only team to successfully match the flavor actually sprinkled salt on the chip. I spoke with one team that used water that was saturated with salt to the extent that the salt was settling out to the bottom. So it can’t be done by following the protocol but sprinkling seemed to work well.
- Size was an issue no matter what. It seems that when you dip the chips in water they shrink 1/8″ (3.175 mm) all the way around the chip no matter what you do. It’s very noticeable.
Overall assessment
- Data fabrication
- Confounds
If the red chips were different enough, suspicious in any way, or disgusting to the taste, that could explain why people ate far fewer in the red chip condition.
 |
| These are the data from Mertens et al. 2022 meta-nudge study |
At the end of the day, it may not matter whether the results are caused by an improper study design resulting in a confound, or due to data fabrication. It’s clear that the results shouldn’t be trusted.
Next steps
Why should anyone care?
Aaron Charlton, PhD, MBA is a marketing professional who currently works in industry for Away Clinic and Away PR and lives in Mesa, Arizona, USA. He is formerly an academic and still takes interest in improving the quality of research in the field of marketing.
Leave a Reply