The 5 most common mistakes to avoid when you are publishing a systematic review

Key takeaways from Researcher Academy’s latest webinar

While adding transparency to a study and saving resources by avoiding repetition, systematic reviews also offer unparalleled credibility to your research. Yet, more often than not, researchers make avoidable mistakes which keeps their hard work from getting published. To quote Paul Whaley, Associate Editor for systematic reviews for Environment International opens in new tab/window, 43 out of 46 systematic reviews fall at the first hurdle, and “it only takes two minutes to reject two years of researchers’ hard work.”

A systematic review answers a pre-defined research question by collecting and summarizing all the evidence that fits into a prespecified eligibility criteria. It can be both qualitative or quantitative.

- Naomi Lee, Executive Editor for The Lancet

For more information on systematic reviews see: https://researcheracademy.elsevier.com/writing-research/technical-writing-skills/systematic-reviews-101 opens in new tab/window

1. The topic has already been covered Systematic reviews are attempting to answer a very specific research question and will therefore be redundant if the topic has been covered already. Even if your review checks all other boxes, if the question you are attempting to answer has already been answered by other systematic reviews, and brings no new insights, it will not be publishable. Tip: Do your homework – are there other recent reviews covering this topic that have already answered your question? Do you have a sufficiently novel angle?

2. Your literature search strategies have been poorly defined Systematic reviews are meticulous, well-planned, and exhaustive. If your literature search strategy has not been properly crafted and documented, transparently capturing all the literature which may be of relevance to answering your question, you are at high risk of producing biased results. Tip: Document all the search terms, study types, databases, and all the choices you make in your research to keep the search strategy valid.

3. You failed to adequately assess the risk of bias

   Bias in research can skew the results of a study, and including biased studies in a systematic review can therefore undermine its validity. While, it can be very difficult to evaluate the extent to which a study is biased, authors of systematic reviews can assess the potential for the risk of bias. Tip: By invoking domain-specific methodologies, especially those which avoid the use of scores/scales, can help assess the risk of bias.

4. Your interpretation of the strength of the evidence is unstructured or unsystematic Systematic reviews require a systematic assessment of the strength of scientific evidence which you synthesize in answering your research question. Your approach should be to identify issues which are important in determining confidence in your results (such as overall risk of bias in the evidence, heterogeneity of results, publication bias, etc.). The higher the quality of the material, the more reliable the results of your review will be. Similarly, the better your appreciation of the limits of the evidence base, the more valuable your review will be to other researchers. Tip: There are existing protocols and tools for assessing the level and strength of evidence of studies. Be sure to find the right one for your research and field.

5. You didn’t plan enough in advance

   Systematic reviews are a complex, multi-step research tool which require you to bring together multiple skill-sets, many of which may not be familiar to you. There is a lot of help, with many textbooks, method papers and training courses which you will be able to draw upon. Take advantage of this when planning your protocol, and get expert advice from strategic review specialists on how to conduct your systematic review. Tip: If you intend to publish the results, you should seriously consider publishing your protocol – an option afforded by an increasing number of systematic review journals.

Learn more

The tips presented here are just a short preview of a truly enlightening 60 minutes with two experts on systematic reviews. You can watch the full webinar recording, entirely free, at the Elsevier Researcher Academy opens in new tab/window.

Final tip of the day: You can add questions to the Researcher Academy Mendeley group opens in new tab/window that you feel were not answered in the webinar and we will endeavour to find expert answers for you.