Love Silhouette/Shutterstock

Science Is Facing a Serious Replication Crisis, But There Are Solutions

What can the scientific community do?

SCIENCE AF STAFF
17 JAN 2018
 

The scientific method relies heavily on systematic replication of research. Nevertheless, many existing results in medicine, life science and psychology cannot be reproduced.

For instance, one study, published in 2015, found $28 billion is spent in the US each year on preclinical research that cannot be reproduced.

 

This poses a serious problem. The lack of replication not only impedes scientific progress, it wastes research funding and often results in ineffective treatments for patients.

So what can the scientific community do?

A new report from the Royal Netherlands Academy of Arts and Sciences (KNAW) proposes several solutions for this "replication crisis."

"It's a nicely balanced report that highlights the challenges for science in general," said Daniël Lakens, an experimental psychologist at Eindhoven University of Technology who was consulted by the KNAW panel.

"It's good that it acknowledges that this is an issue that we should think about."

To improve the quality of research, the report suggests institutions should place greater emphasis on training in research design and statistical analysis. Plus, these institutions should actively teach scientists how to conduct replication studies, an area of research that is often overlooked.

As far as upholding the scientific rigor of papers, the report suggests journals should issue guidelines for reports as well as require scientists to preregister their study method for evaluation before data collection even starts.

 

But perhaps the most important suggestion is this: The scientific community needs to change the social and monetary incentives for replication studies.

Compared to original research, there is very little glory (or tenure) for scientists conducting replication studies - no matter how fundamental replications are for empirical research.

The report offers three major solutions to this problem:

  • Funding agencies should put aside more money for replication studies.
  • Journals should provide more space to publish both replication studies and null results.
  • Institutions should give more credit in career evaluations to researchers who conduct replication studies.

In a newspaper interview, co-chair of the panel Johan Mackenbach said eventually he would like to see between 5 and 10 percent of research funding spent on replication studies.

Right now, the US federal government spends more than $30 billion on basic scientific research, yet virtually none of that is put aside for research replication.

Even still, some scientists, like Harvard psychologist Daniel Gilbert, have criticized the Dutch solution, which provides 3 million euros from the Organization for Scientific Research to replicate foundational research.

"If the Dutch government wants to spend its money on research whose sole qualification is its unoriginality, then that's their prerogative," he said.

 

"Will we learn something valuable from such research? Probably. Will it be more valuable than what we would have learned if the same amount of money had been spent exploring important new ideas? That's a difficult question to answer. But it is the critical question, and it is the question no one asks."

Luckily, the conversation is beginning to pick up steam, even in the US, where the National Academy of Sciences is now studying reproducibility and replication.

"This doesn't stop at the border," said Lakens.

"Other countries can easily take over these conclusions because they apply just as much. And I hope they will."