RS123 - Daniel Lakens on P-Hacking and Other Problems in Psychology Research

Release date: December 14, 2014

Daniel LakensWhat's wrong with the social sciences? In this episode, Massimo and Julia are joined by Professor Daniel Lakens from the Eindhoven University of Technology, who studies psychology and blogs about research methods and open science. The three discuss why so many psychology papers can't be trusted, and what solutions might exist for the problem (including how to fix the skewed incentives in the field).

Daniel's pick:   The Open Science Framework

Comments

[Jay Tanzman]

Readers might be interested in our new paper in PLOS One, Excess Success for Psychology Articles in the Journal Science, in which we found evidence of p-hacking or other questionable research practices in 83% of analyzed articles.

[Max]

How much of the problem stems from Psych majors hating math and majoring in Psych because they think it won't involve math?

[Mark Shulgasser]

This was a fascinating chat – thanks so much.

It seems clear, particularly in view of the PLOS article cited in the comment above: What with p-hacking, data-torturing, weaknesses of peer-review, lack of replication, publication biases, and the farce of meta-analysis, that the field of social psychology, which has always been haunted by the specter of triviality, has been and is increasingly permeated with ethical violation and amazing disingenuousness, clearing huge space for mediocrity. This is not to say that the field is worthless, only that the reforms described here are an essential start if it is to be salvaged.

This moment where the interaction between the socio-economic structure of academic science intersect with the analysis of a methodology should be closely observed. Psychologists have skated along too long on the assumption that the same principles of explanation obtain in the human sciences as in the physical sciences.

Given the ease with which spurious demonstrations are churned out by academic psychologists, it’s always struck me as a mark in astrology’s favor that it makes such a poor showing.

[Ernesto]

Another fascinating show. Greatly appreciated my peeps. ;)

[Jameson]

If over 50% of these experiments fail to replicate, we have wasted a lot of money, since we have spent a lot on this research.

I love the quote "With sufficient torture, you can get the data to tell you anything".

Perhaps science journals should publish results with p>.05, negative results, and replication studies, as well as requiring disclosure of data at the time of publication. Also, we should have open access journals without a subscription fee so anyone can inspect the work, especially since much of this research receives public funding.