Getting to grips with peer review

istockphoto.com/VictoriaBar

Introduction to Sense about Science

More than ten years of running our well-established programme of Voice of Young Science: Standing up for Scienceopens in new tab/window workshops has shown us that early career researchers (ECRs) want to find out more about peer review. They want to know how to get involved and what to make of public discussions about fraud and misleading research claims in science and medicine. To address this need, our peer review programme equips early career researchers with an understanding of the process and how to participate, and it encourages them to play an active role in discussions about scientific reasoning, peer review and research quality. We hold two peer review workshops a year for ECRs in London and Glasgow, supported by our Peer Review: the nuts and bolts opens in new tab/windowguide. These workshops involve two group activities and a panel discussion with experts in the field. Our format, involving open and honest discussion with a diverse panel including an academic editor, a peer review or publishing specialist and a peer reviewer, is effective at illuminating the “black box” of peer review from different angles. The workshops have proven to be very successful in equipping researchers with the tools to confidently involve themselves in peer review, whether as a reviewer or author. Attendees report that they leave the workshops feeling empowered with the knowledge and confidence to actively engage in discussions about research quality and how it is served by the peer review process.

We are running our second Peer review: the nuts & bolts workshop of 2019 at Glasgow Caledonian University on Friday, 18th October. Apply for your free place via this linkopens in new tab/window. Don't miss out on the opportunity to find out about research quality, the peer review process and how you can get involved.

Elsevier has been a firm supporter of Sense about Science and often participates in its workshops. Bahar Mehmani, Elsevier’s Reviewer Experience Lead was the peer review expert at the latest workshop, held at the Institute for Materials, Minerals and Mining in London. Below is Bahar’s take from the event.

Report from Sense about Science’s latest workshop

The panel

• ●

  Pete Chapman - Managing Editor, Women’s Health

• ●

  Elisa de Ranieri - Editor-in-Chief, Nature Communications

• ●

  Eamonn Carrabine - Editor, British Journal of Criminology

• ●

  Bahar Mehmani - Reviewer Experience Lead, Elsevier

Overview

Pete opened the panel by explaining what peer review is, illustrating the pros and cons of different models and showing how the peer review process works. Elisa then discussed how editors evaluate manuscripts before sending them out for review and how they choose reviewers. She also described how editors choose to proceed after receiving different recommendations from reviewers. Finally Eammon told us about how he got involved in peer review process as a reviewer and how reviewing manuscripts and publishing papers eventually led him to his editorial role. I noted a few themes emerging when the participants reported back on their group brainstorming sessions (which took place before the panel discussion). I therefore based my presentation around these…

Recognition

The majority of attendees noted that the lack of proper reviewer recognition is a weakness of the system. Although many journals have either their own reviewer recognition program or use services like the Elsevier Reviewer Recognition Platform or Publonsopens in new tab/window or acknowledge their reviewers by publicly thanking them, the community apparently still feels that more could be done. There is also a lack of recognition at the institutional level and beyond. It is important for academics to see funding agencies and evaluation committees taking note of peer review activities as well as the more traditional metrics. In my view, the future will involve a system in which not only journals but also universities, institutes and funders take researchers’ peer review activities into account in their incentive and evaluation systems.

Co-reviewing

Some of researchers in the audience had experiences of reviewing a manuscript at their supervisor’s request. They found the experience enlightening, but at the same time, they wanted to see the editors’ feedback on their review reports. Working as “ghost reviewers”, this isn’t normally the case. I urged everyone to help make this process transparent, for the benefit of all. Editors can facilitate this by making it clear that they are happy to receive a co-review. Doing so means that (co-)reviewers can be registered and will receive updates and information about the paper (as well as due credit for their efforts, of course!).

Speed

Although everyone in the room agreed peer review should not be rushed, most attendees agreed that the process is very slow and affects authors negatively. I mentioned the process can be accelerated if using technology: some checks can be done at the initial triage phase so that reviewers can spend their time efficiently and focus on the research. The editors on the panel encouraged any author affected by a slow-down to contact the editorial office. Some researchers in the audience feared that a manuscript decision might be delayed by a reviewer they suspect of having a conflict of interest (CoI).

Conflict of interest

All those on the panel emphasized the importance of the editor’s role in avoiding reviewers with conflicts of interest. Eamonn reminded us that it is sometimes impossible to know if a CoI exists. He went on to urge any reviewer with a potential conflict to be open and honest with the editorial office. Elisa reminded authors to keep CoI in mind when suggesting reviewers.

Bias 

There was an active discussion about the potential for bias in the system. I mentioned we are all human beings with both conscious and unconscious biases. What is more, peer review takes place within a hierarchical system where publications are prized most highly. In such an environment the best publishers and editors can do is to raise awareness and be conscious of their choices of editorial board members and reviewers. Elisa pointed out that unless we capture ethnicity and gender in a systematic way, it is hard to set a base line. In the meantime, we should all try to increase awareness of these issues in the broader community.

Summing up

At the end we all agreed peer review is not the ideal system but one which does generally provide credibility and trust. It can and should be improved, however. My concluding statement was that the future of peer review is a more transparent, collaborative, efficient and acknowledged one, enhanced by technology and better integrated systems.