top of page
  • Writer's pictureEva

Research, Reproducibility and Retractions

Nobel prize winner, Dr Frances Arnold, retracted a paper she published in the prestigious journal Science on Thursday.


'Retracting' a paper means that the paper should not have been published in the first place because it is flawed in some way. Although not as scandalous as implied by some news headlines - the contents of the paper were not related to the work for which she was awarded the Nobel prize - this story has caused surprise and, as with almost everything these days, a twitter reaction.



As of today, Dr Arnold's apology tweet has got over 4000 likes and hundreds of replies on twitter commending her bravery and honesty to retract the paper.

The journal's statement concerning the retraction was a little less emotive:


'Reproducing' work literally means repeating the experiments and getting the same results. Scientific progress relies on the fact that all published articles are reproducible. Being reproducible implies it is 'true' (as in, not made up or just a fluke), but it is also important for scientists in a practical way:


~~~~

Imagine you're a scientist who wants to make a drug to kill pancreatic cancer cells. You read a scientific article which shows that Chemical A, at a certain concentration, kills a breast cancer cell line. You order Chemical A to test it on your pancreatic cancer cells, and find it doesn't kill them. You are undeterred (scientists are used to things not working), and want to rule out the possibility that you're using Chemical A incorrectly - maybe the way you measured it out wasn't quite the same as was done previously. The only way to check this is by reproducing the results of the paper. You follow the method they give to dilute Chemical A to the right concentration, and test it on the same breast cancer cell line. Once you manage to get the same results that they did (i.e. the cells die), you can change one thing: using pancreatic cancer cells instead of breast cancer cells. By changing just the cell type and keeping everything else the same, you can make sure that the reason Chemical A isn't working is because it doesn't work in pancreatic cancer cells, not because something else has gone wrong.



~~~



Unfortunately, according to a survey of 1500 scientists done by the journal Nature, over 70% of biologists say they've not been able to reproduce an experiment - this makes it very difficult to continue the research as its impossible to figure out whats going wrong.


Why are papers not reproducible?

Outright faking results is quite rare (thankfully), so most of the time its usually because experiments are technically difficult and require every little detail to be correct (maybe Chemical A has to be prepared at exactly the right temperature). According to the Nature survey, the main factors that scientists think contribute to irreproducible research are 1) pressure to publish and 2) pressure to publish something new.

1) In the science world, publishing articles, especially in high-end journals, is essential to further your career. As a professor, your lab's funding is often based on how many papers you publish. The pressure to get your results in an article can mean that you need to do things quickly, maybe cutting corners or not keeping note of extra experiments or details which might be important to reproduce or build on your data.


2) To be published in a top journal you need to have exciting, new data - nobody wants to know about all the other experiments you did that failed or that attempt you did at replicating another person's data that didn't work. This means that what ends up being published is often not the whole story. Negative results (when nothing happened) or optimising steps (when things didn't work) stay in the back of a drawer somewhere, and they might be essential for the next scientist to better repeat the experiment or set up a similar experiment.



Some of the reasons described here may explain why even a Nobel prize winning scientist can author a paper which needed to be removed due to it not being reproducible. Whilst her honesty is commendable, this retraction serves as an important reminder of the importance of being thorough when publishing, and perhaps could serve as a warning to funding bodies and journals that the pressures placed on scientists may be driving quantity of results over quality.

74 views0 comments

Recent Posts

See All
bottom of page