Can’t see the method for the madness? Reach for the STAR!

Empowering reproducible methodologies through structured, transparent and accessible reporting

You know my methods. Apply them.

^{― Arthur Conan Doyle, The Sign of Four}

Open science describes a more inclusive, collaborative, transparent world of research. At Elsevier, we’re enabling open science through our approach to open access, open data, research integrity, knowledge exchange, metrics and more, to benefit research and society and drive research performance. One of the areas we are focusing on is that of research methodology.

Background

For many years the “methods” sections of articles were treated as an afterthought. Readers tended to skip straight to the results much of the time and methodologies were sometimes actively stigmatized by journals – being relegated to “supplementary” sections or typeset in tiny font sizes. Authors for their part, conscious of often strictly-policed word limits, tended to cut from the least ‘’sexy’’ part of their article – leaving methods severely truncated.

With the reproducibility crisis opens in new tab/window, it has recently become clear that this culture of methods being treated an afterthought was a mistake because researchers can rarely replicate a paper from the published methods on their own without having to locate the original authors and ask them a number of questions. When it transpires that the original authors are no longer available or simply don't themselves remember anymore, replication becomes a nigh-on impossibility!

Reaching for the STAR

Why does this matter, you might ask? Well, reproducibility promotes trust in individual research findings, in researchers, and in science more broadly. Elsevier is committed to making reproducibility a reality and actively supports the proposals in ‘’A manifesto for reproducible science opens in new tab/window’’ by Munafò et al., 2017 and the consensus reached in the 2014 NIH workshop on the issue of reproducibility and rigor of research findings “Principles and Guidelines for Reporting Preclinical Research opens in new tab/window”.

Scientific rigour forms the foundation of reproducibility and in 2016, Elsevier’s Cell Press introduced STAR methods opens in new tab/window, which outline the features of a robust, reproducible methodology: one that is structured, transparent, accessible and reported. Components of STAR methods include:

• A key resources table organizing resources and reagents, including catalogue and identifier numbers (RRIDS)

• Unrestricted word limits allowing full reporting of methods and details

• An update to the authorship policy to establish the lead contact as the contact responsible for fulfilling information requests for reagents and resource sharing (this is also clearly displayed on the PDF and HTML of the article)

• A standardized structure for presenting methods

• Promotion of data and software sharing

Inspired by the success of this Cell Press innovation, we are looking at translating aspects of STAR methods to other Elsevier journals.

Introducing improvements

In November 2017 we introduced updates to our guides for authors’ texts with the aim of improving the reporting of methods in manuscripts. Specifically, we started to require that authors: “…provide sufficient details to allow the work to be reproduced by an independent researcher. Methods that are already published should be summarized, and indicated by a reference … Any modifications to existing methods should also be described.” (Incidentally, we hope that you think this new text is clear and useful but if you have any suggestions for how it could be improved; please comment below!)

Inspired by the success of STAR Methods with Cell Press journals, Valentina Sasselli opens in new tab/window, the Publisher for Elsevier’s other cell & developmental biology journals, was keen to share the idea with her editors. Valentina, prior to becoming a Publisher had been active as a cell biologist so she was ideally placed to work with her editors to introduce the key resources table opens in new tab/window - one of the most important components of STAR methods. The key resources table highlights the genetically modified organisms and strains, cell lines, reagents, software and source data essential to reproduce the results presented in a paper. We are currently capturing feedback from the editors, reviewers and authors of the journals involved and welcome other journals to join this pilot (please contact your Publisher if you would like to explore this initiative for your journal).

How machine learning can lend authors a helping hand

STAR methods ask for some more work from authors right now. Ultimately the beneficiary may be the reader, more specifically the future researcher who tries to replicate the work or perhaps even the future author themselves given that even replicating one’s own work years later can be challenging! Of course, it’s also good for the author in terms of ‘’virtuous signaling” because STAR methods enable them to demonstrate how open they are – a quality that is increasingly demanded by institutions and funding bodies. Mindful of the additional load that this represents, we are also working on semi-automation of the key resources table. By extracting methodological meta-data from the manuscript (e.g. antibodies, cell lines etc. used in the research), we aim to start off the creation of the key resources table for the author and save them valuable time and effort.

Moving beyond biology

Clearly this work is very laudable but how can we translate Cell Press’ STAR methods to non-biology journals and thereby further increase the reproducibility of research? The methodology for doing so will differ from field to field but the goal will always be the same: to make methods clearer, more detailed and therefore easier to replicate. What we would really like is for editors in diverse fields to get involved and help us reinvent this concept in a way that’s appropriate for their field. If you have thoughts on how this could be accomplished for your journal/field; please get in touch with your Publisher or leave a comment below.