On The Nib: Psychology Has a Reproducibility Problem

Psychology has a replication problem. Since 2010, scientists conducting replications of hundreds of studies have discovered that a dismal amount of published results can be reproduced. This realization by psychologists has come to be known as “replication crisis”.

For me, this story all started with ego-depletion, and the comics I had drawn about it in 2014. The idea, briefly explained in the excerpt below, is that your self-control is a resource that can be diminished with use. When you think about all the times you’ve been slowly worn down by temptation, it seems pretty intuitive. When I drew the comics, there had been new research pointing to blood sugar levels as the font of self-control from which we all draw. It also made sense—people get cranky when they’re hungry. We even made up a word for it. We call it being “hangry”.

Fast forward to July of this year when Martin Hagger and Nikos Chatzisarantis published the results of their multi-lab effort to replicate the results of a seminal ego-depletion study. The verdict? Over two thousand participants across twenty-three labs were not able to produce a statistically significant result. In addition, the studies pointing to blood sugar levels as a “fuel” of sorts, were also discredited by further studies. Alarmed by the implications, and looking to set the record straight on the comics I had drawn about ego-depletion, I reached out to Martin. Originally, I had planned to draw an erratum of sorts in comic form, but as I spoke with Martin about the issue, it became clear that there was something bigger that I had been missing.

Reproducibility problems were arising wherever they were studied, in several different fields, and they all pointed to various systematic problems and creeping biases within the publishing process itself.

Repeat after me

Read the full comic on The Nib »

But it hardly stops there. I had more material than I knew what to do with, and several interviews happened too late in the process to make it into the comic. I want to start publishing the unused material, as well as further interviews with the scientists trying to make science more accountable. Hopefully, together with the Nib piece, it will give people a greater understanding of this current problem in science, as well as how scientists are working to fix it.

Like this post? Consider supporting me on Patreon!