Elsevier’s Reviewer Recognition Platform prepares for next phase

Earlier this year, we reported on the new Elsevier Reviewer Recognition Platform in the Reviewers' Update article How we can better support and recognize reviewers.

Since the platform's launch, we have been monitoring the feedback and advice you have shared with us and we are now ready to begin work on phase 2. Recently, Professor Dr. Lutz Prechelt, Professor of Informatics and Head of the Software Engineering Research Group (AG SE)  opens in new tab/windowat the Freie Universität opens in new tab/window in Berlin, joined the program as an advisor. 

In this article, Elsevier's Dr. Joris van Rossum and Dr. Bahar Mehmani take a closer look at reactions to the platform's launch and Professor Prechelt talks about our ambitious plans for the initiative's future.

What is the Reviewer Recognition Platform?

We know from your feedback that while you find reviewing important, you feel your work is near-invisible to the outside world and is hardly rewarded. The Reviewer Recognition Platform aims to change that. The platform provides participating reviewers with a personalized profile page where their reviewing history is documented. Moreover, reviewer statuses are awarded based on the number of reviews they have completed for a specific journal; a reviewer who completes at least one review within a two-year time period becomes a 'Recognized Reviewer', while those in the top 10th percentile become 'Outstanding Reviewers'. The platform also offers various discounts for Elsevier services, such as the Elsevier WebShop opens in new tab/window, and reviewers can download a variety of certificates. Future extensions will also describe the quality of reviewing, by means of percentiles among the reviewers of the particular journal, measured with journal-specific criteria.

Initial reactions to the platform

When the Reviewer Recognition Platform went live in March this year, there were 40 participating journals. It now features more than 200 Elsevier journals from a variety of disciplines and that number is growing - each month, we add the five-year review history of reviewers for 50 new journals. When a review has been completed, reviewers receive an email providing them with a direct link to their Elsevier review profile on the platform. So far we have sent emails to 7,700 reviewers.

Our survey results show that reviewers find the Reviewer Recognition Platform a valuable initiative. They have given the platform a score of 8.20 out of 10 (n=488). More than 68 percent of respondents found the information contained in their profile useful and more than 41 percent mentioned they plan to share their profile and status with others (their department head, colleagues, followers and friends on their social media channels, etc.). We have also received a number of novel ideas and suggestions from reviewers about how we can target rewards and further recognize their invaluable efforts.

Professor Dr. Lutz Prechelt's story

I joined the Reviewer Recognition Platform project as an advisor, because I have been thinking about initiating a similar program - the Review Quality Collector opens in new tab/window - since 2012. I would like to ensure scientists are publicly recognized for good reviewing performance, journals receive reviews of the best possible quality, and research institutions include reviewing quality information in their evaluation of scientists' overall performance.  Elsevier's Reviewer Recognition Platform had already started and could be extended to include the Review Quality Collector's goals, so we decided to join forces.

During the next stages of the project, we want to onboard more journals to the platform. Also, we want reviewers to be able to download their review history so that the reviewing record can easily be shared.

The crucial next step, however, will be the introduction of quality elements in the reviewer status. So far, the status is based on the number of completed reviews only. We want editors and authors to provide feedback about the helpfulness of the review and also record timeliness so that we can take these important aspects into account when recognizing reviewers.

We have already started the process by asking editors and authors of selected journals a few simple questions right after the reviewers have delivered their reports. This includes asking them to rate the submitted manuscript reviews in a generic fashion. Over the next couple of months, we expect to tweak the questions, analyze the results, and hold discussions with editors. Our aim is to arrive at a criteria framework that can be customized for each journal in such a way that it will provide a sound method to gauge the qualitative contribution of a reviewer.

In the long term, we hope the statuses and procedures we develop will evolve into industry standards that will be adopted by other publishers. It is time reviewers are appropriately recognized for their important contribution to the progress of research.