Another Jayati Sinha paper with impossible means

My opinion: The effects don’t pass the smell test

Sinha and Wang (2013) claim there is a type of loneliness x time horizon crossover interaction effect on impulsive behavior. Any time you see a bunch of massive crossover effects it’s interesting. This is really the holy grail of marketing. The best way to get this kind of crossover is by studying an extremely obvious difference between conditions. In this case the difference between emotional (i.e. do you feel lonely with regard to friends?) and social loneliness (i.e. do you feel lonely w/regard to family, romance?) is fairly subtle with no obvious difference in terms of impulsivity on its face. Moreover, being primed with a simple statement: “Life is long. Enjoy it forever, over a long period of time!” versus “Life is short. Enjoy the moment, right now!” seems unlikely to yield huge downstream effects on an unrelated task involving consumption. This intuition is confirmed by Maier et al. (2022) which shows an utter lack of evidence of time horizon manipulation effects in the entire body of construal level literature, making it very surprising to see such massive crossover effects here.

All the crossover figures from Sinha and Wang (2013)

And there are data anomalies (impossible means)

I also simulated the data to learn more about the properties of these numbers. From this I learned that many of the means and standard deviations also fall outside of the expected ranges (see below for Study 1a). For the purpose of simulations, note that there are no differences in loneliness measures across conditions according to the authors.

Simulation of means and standard deviations for Study 1a

Code for simulations above.

Ruling out obvious errors as an explanation

1. Typos. There are too many problems scattered throughout the table to consider typos a plausible explanation.
2. Centering but not standardizing. Would not explain why the means do not average to 0.
3. Standardizing but not centering. All the means would be positive.
4. Copy/paste errors. Since half the numbers fit the expected pattern and half don’t and the numbers that fit and don’t fit are mixed together, this seems unlikely

A bit more context

The author who created the means table in question, Jayati Sinha, has a bit of a history: Sinha retraction | Sinha EoC 1 | Sinha EoC 2. EoC 2 also related to impossible means in a priming paper with massive effect sizes. Impossible means can sometimes be explained by a complete lack of participant data (i.e. the statistics are completely fabricated) but in this case there does appear to have been data at one time (though not now).  I say this because their study 2 data was later reanalyzed in a paper about mediation.

Author response

I reached out to the authors when I discovered the issue. Alice Wang responded very professionally and kept it short, mainly just saying that Jayati Sinha had created the tables and Sinha would reach out to me shortly. Here is Jayati Sinha’s response in full:

I don’t want to add too much of my own interpretation to this email but I did find the RA thing a bit odd and I was confused by how she would “address this error” without any data.

My efforts to report this issue

I reported these issues to both Journal of Marketing Research and Florida International University, her employer. I have no idea what the status is of those investigations (if there are investigations). This was in August of 2021.

Update

This paper was retracted in May of 2022.

Notes

Photo credit: Jeff Nissen

Aaron Charlton

Aaron Charlton, PhD, MBA is a marketing professional who currently works in industry for Away Clinic and Metascience & Marketing Lab and lives in Mesa, Arizona, USA. He is formerly an academic and still takes interest in improving the quality of research in the field of marketing.