Ten tips for a truly terrible peer review

Editor Bert Blocken highlights some of the major mistakes early-career researchers can make when acting as a reviewer for a journal article

Peer Review Week 2017 新しいタブ／ウィンドウで開く (#PeerRevWk17) is taking place between 11^th-17^th September. The theme of this year is transparency (watch out for content #TransparencyInReview). Here at Elsevier we are pleased to announce a free webinar on Sept 20^th’ on the subject of “how to write a terrible review” (sign up here 新しいタブ／ウィンドウで開く). As an accompaniment, we offer below Dr Blocken’s article on the same subject.

There is probably no rule as applicable to the activity of peer review as the so-called “golden rule” or law of reciprocity: do not treat others in ways that you do not want to be treated yourself. This holds for the relation of the peer reviewer to the authors, the editor, and even the wider scientific community. As publication pressure and the quantity of submissions continue to increase, so does the need for high-quality peer reviews. Several best practice guidelines and rules have been published before. This document takes a different approach, by highlighting 10 things you should certainly NOT do. I’ve based them on my experiences in the exact sciences/engineering fields, but some of them may well hold true, whatever your discipline.

You are not alone in experiencing pressure to publish. Many authors find themselves in this situation and, when submitting their papers, do so in the hope that their articles will receive fast, fair, complete and constructive peer reviews. Without such peer reviews, the publication system cannot exist. Some systematically decline review invitations with the argument of being too busy. Yes, certainly, we all are. But if you want to see your own papers published, you should realize that, for every submission (not publication!) of your own, you should be willing to review at least two or three papers yourself. But even then there are plenty of ways to thoroughly disappoint the editor, the authors and even to negatively affect the wider scientific community: just follow one or more of the tips below.

1. Accept an invitation to review when you should not If you respect reciprocity, you will be willing to accept review invitations, certainly from journals in which you have published yourself or to which you want to submit your own work in the future. However, do not overstretch yourself. Do not accept a review invitation when you have never experienced the peer review process as an author. Do not accept to review papers on a topic that does not match your area of expertise, or when you have a conflict of interest. If you agreed too fast and realise these issues only after accepting to write the review, don’t worry. Notify the editor instead of delivering the least poor peer review possible. Certainly do not take no action at all. The author(s) and editor(s) are waiting for you. Don’t leave them in suspense.

2. Submit your review very late or not at all Reciprocity is often but not always advised. If you have experienced waiting for a (very) delayed review as an author yourself, do not echo the treatment if acting as a reviewer. Unfortunately, this does happen too often and a vicious circle can emerge that is very damaging to the journal and to the research community in that field as a whole. Evidently, this will not benefit you in the long run either. Accepting a review also means committing to a deadline. Sure, unexpected circumstances can occur. In that case, notify the editor and respectfully ask for an extension. The worst option is to accept the invitation but to never send your review. It delays the whole process because it causes the editor to invite new reviewers. This also increases the burden on your colleagues who now have to do the  review you should have done. If your dream is to impersonate the editor’s nightmare: then tip no. 2 is for you.

3. Ignore messages from the editor If you deliver a good review in time, you will probably only receive automated messages. In other cases, the editor might contact you personally. Either to ask for the status (in case you are running late) or to ask for additional information (in case your review is unclear or incomplete). Not answering a question is considered rather impolite in daily life. Ignoring an editor’s question communicated by email is no different. If you want to give the editor a major headache and want to delay the whole process as much as you can, you might find this tip suits your needs.

4. Do not check for adherence to publication ethics Publication ethics contains many elements. Plagiarism, for one, is a serious breach of publishing ethics and in some cases may also be copyright infringement. As publication pressure continues to increase, so do attempts at plagiarism. Although most academic publishers have installed very elaborate procedures to detect plagiarism, such as Crossref Similarity Check, probably the best detection instrument for plagiarism in a given subfield of science is you: the peer reviewer who knows the literature and the state of the art in the field intimately. Sometimes issues can be solved by reminding authors that they should add the proper references in the proper places, to indicate that some statements and findings are not theirs but should be attributed to others. Reviewers should clearly indicate to the editor when they find plagiarized content, which evidently includes claiming achievements or ideas from other authors. But this also holds for publications by the same authors, which add little to the field. This is probably the only case in which a review with a single paragraph will suffice and in which following tip no. 6 is allowed – to help save you time – as true plagiarism cancels the need for any further comments.

5. Provide a useless review So, you have accepted the review invitation, but the deadline is approaching and you find yourself having better things to do than reviewing others’ work. It happens to all of us. When this happens, it can be tempting to just browse the article and write a few general and vague lines that are absolutely of no use to the editor and the authors. This is not only utterly disrespectful to the author’s work, but also requires the editor to invite new reviewers, which will increase the burden on others and delay the whole process. You will also directly damage your own reputation: your advice (accept/revise/reject) might be in direct contrast to the advice by the other reviewers. But even without that, the editor will not consider your reviews seriously anymore and might stop inviting you in the future. There is no taking without giving. The law of reciprocity implies that you cannot expect that others will keep providing useful reviews for your papers, if you do not return the favour. Maybe you think you will only give a very poor impression to this single editor, but the issue might very well be shared with the editorial board. If you do not care that much about your academic reputation and you do not mind causing frustration to editors and authors, this tip can save you a lot of time.

6. Forget to address the key components of the article These are questions about the key components of an article that editors need your review to answer: - Does the article fit the scope of the journal? - Is the research novel? - Is the title representative of the article contents? - Does the abstract summarise the contents clearly? - Is the state of the art well described and the knowledge gap clearly defined? - Are the objectives well articulated? - Is the applied research methodology solid? - Are the results reliable and have the objectives been reached? - Are the limitations correctly mentioned? - Are the conclusions justified? If necessary, provide recommendations to the authors on how to improve these components. In addition, advise authors to remove subjective statements expressing personal bias, emotional involvement etc. While the authors may find that the 20% deviation between the results from their new experiment and those of a previous experiment is a really good agreement, your solemn duty is to suppress their enthusiasm by asking them to replace subjective statements such as “an excellent agreement was obtained” by an objective statement like “the deviation was 20%”. The authors might not like it, but it is in their best interest. Also recommend improvement of the title, abstract, highlights, tables, and – certainly – figures if necessary. Author guidelines often call figures “artwork” but often they are work rather than art (at least where the reader is concerned!).

7. Provide unclear, unfair and biased statements Good ingredients for a truly terrible peer review are unclear, unfair and biased statements. Your comments and recommendations should be clear and unambiguous. Statements like “the methodology is not solid” or “the conclusions are unjustified” have no value per se apart from being in line with tip no. 5. Clarification is important as editors and authors cannot read your mind; they can only read your review. Do not only state what should be improved but also suggest how it can be improved. Avoid dismissive, unfair and biased statements. You would have done a much better and much more thorough job than the authors, right? Think again. Generally, it is much easier to identify deficiencies in others’ work than in your own. This does not mean that your work is better; it just means that you, too, will need peer reviewers that provide clear, fair and unbiased comments pointing you to the deficiencies in your own work. You should be grateful for that, and you should make sure the authors will be grateful to you. No research is perfect, and every publication, no matter the research area or topic, is at best a very small link in the long chain of incremental knowledge advancement in a tiny subfield of science. Modesty suits authors well, and the same goes for peer reviewers. It will happen occasionally that you have to review a very good paper. Compared to many other papers, this will be a relief. In such a case, don’t squeeze yourself to provide critical comments, but spend a few lines on expressing your appreciation for the authors’ work.

8. Unjustly obstruct publication or breach confidentiality If you want to be sure never to receive a review invitation again from a given journal, you can follow this tip. It is highly unlikely that you or your co-workers are the only ones on planet Earth working on a specific topic in a specific field. If others are working on the same topic, it is rather likely that you will be invited to review their paper. So what if you have just submitted a paper on this topic yourself to another journal, or are currently preparing such a paper? The most severe breach of reviewer ethics would be to unjustly obstruct the publication of the author’s work and/or to use your knowledge of their work to gain a competitive edge. Less severe but still very unethical is to breach confidentiality by sharing the contents of the submission with others prior to publication. If you cannot bring yourself to provide a fair, fast and constructive review – which is ok, we are all human – then you should immediately declare a conflict of interest and notify the editor. Honesty and virtue always outrank personal advancement and temporary fame.

9. Suggest that the authors violate publication ethics As a reviewer, you have some power. Use it fairly, wisely and in moderation, always in view of supporting the journal and your research field. Do not ask authors to provide unfair or biased statements about their work or the work of others. Do not ask the authors to add citations that are not relevant – certainly do not ask them to add irrelevant citations to your own papers to inflate your citation record and h index. If you do not like reviewing, if you want to thoroughly disappoint the editor or if you want to make sure he or she will never invite you again, this tip is for you.

10. Forget to review your review with the “golden rule” Finally, with or without the above-mentioned tips, you have reached the stage where your review is ready. Right? No, not really. It is not unlikely that you have written something that you will regret afterwards, when reading your review again. So make sure to reread your review before submitting it to the journal. At this stage, in particular, apply the golden rule: do not do that to authors which you would not care to receive from reviewers yourself. Examine the review and remove potentially harsh or offensive statements. Rephrase potentially unclear, unfair or biased comments. Above all: be polite, fair and kind. If you recommend revision of the paper, do so kindly. If possible, wrap up your criticism by adding a compliment or two. Even if you have to recommend rejection of the paper, do so kindly and politely. Authors will feel bad in this case, there is no need to make things worse.

Oscar Wilde* stated: “Experience is simply the name we give our mistakes.” As a reviewer, you have the opportunity to help building the experience of the authors in publishing their work. Conversely, your review activities will also help you to become a better author yourself. Reviewing is an essential task of a scientist to advance the research field and the journals in this field. Do not take it lightly.

_{*Oscar Fingal O'Flahertie Wills Wilde, Irish writer and poet, 1854-1900.}