A community-driven boost for research integrity

The last few years have been challenging for research integrity. The proliferation of new behaviors — including systematic manipulation of the scholarly publishing process and the emergence of generative AI — have made it easier than ever to create papers with fake data, images and text. Last year, a record number of papers were retracted across the publishing industry, many produced by paper mills.

So it would be fair to expect that as Program Director of STM Solutionsopens in new tab/window — part of the global STM association with a mission to help STM publishers protect research integrity — Dr Joris van Rossumopens in new tab/window is feeling the strain of attack on all fronts.

But that is far from the truth. While Dr Van Rossum is all too aware of the rising tide of submissions from paper mills and other grave ethics-related breaches, his organization’s mission has received a welcome boost by the recent addition of a powerful new tool — a technology that can check for duplicate submissions based on the full text of the manuscripts.

Submitting the same manuscript to more than one journal simultaneously goes against most publishers’ policies. When duplicate manuscripts are published in multiple journals, it creates redundancy and affects the provenance of the research, thereby interfering with the accuracy of the historical record. In some cases, duplicate submissions may also indicate a deeper systematic manipulation.

This new tool enables users to detect these simultaneous submissions across journals and publishing houses.

Game-changing technology

STM and its members had been working on their Integrity Hub to develop a duplicate manuscript submission checking tool. Until now, they were using technology that was limited to scouring a paper’s metadata, such as the abstract and title. The new technology, developed by Elsevier, goes further by scouring the full text and comparing it to previous submissions.

“We expect it to, first of all, significantly improve the efficacy of the tool, but also it will make it much more difficult for bad actors to manipulate the process,” Dr Van Rossum said.

Elsevier is sharing the technology for free with the Hub and all its members. Dr Van Rossum sees it as a “game-changer.”

A strategy based on cooperation

This kind of collaboration is the cornerstone of the Integrity Hub’s mission to protect research integrity. The cooperation works on several levels:

• For knowledge exchange, publishing managers can share their experiences on topics such as how paper mills work, best practices for screening manuscripts and strategies for recognizing manipulation.

• To establish policies and legal frameworks, members can work together to build strong and agreed policy frameworks aimed at devising the most effective ways to combat bad actors.

• For technology, it provides a trusted infrastructure that integrates data flows from different journals and connects those to various screening tools; a trusted environment to collaboratively develop tools based on shared signals; and an environment where Elsevier and other publishers and organizations can share the technology they have developed with the wider academic community.

“What we’ve seen in the last years is that we have a lot of journal article retractions, but that’s basically too late, right?” Dr Van Rossum said. “You want to make sure that the bad papers don’t enter the peer review process let alone the published corpus. So, what we try to do with the Hub is to ensure that we catch those bad manuscripts at submission. … And for that, technology plays a very important role.”

Poised for success

The duplicate submission detection service was pilotedopens in new tab/window in October 2023 following a working group assessment. This technology represents the first time duplicate submissions can be detected across different journals, publishers and submission systems while upholding robust confidentiality and privacy standards. Dr Van Rossum said the technology has been warmly welcomed by publishers, and its detection rate is on target and positioned for further growth. As of May 2024, 12 publishers were using the tool, and 35,000 manuscripts per month were being screened.

So far, the tool has revealed a 1% duplication rate, meaning that 1% of all submissions from those journals were simultaneously submitted to other participating journals, he explained. And that number will grow as more journals are included in the pilot.

In a survey his organization conducted recently, they found that about half of duplicate submissions were thought to come from paper mills.

Now, with the new capability, Dr Van Rossum expects detection numbers to rise, resulting in a higher number of detections and rejections of duplicate submissions and a higher number of paper mills caught at the submission stage.

The Elsevier service is currently being tested with a select number of publishers and is expected to be rolled out to all participating publishers later this year.

The power of community

The sheer scale of the problem has brought the scientific community together in a new way, following the recognition that only by working together and sharing resources can they turn the tide. Already, more than 100 volunteers are working on detection among STM members, which Dr Van Rossum sees as an important sign of the community’s commitment and growth potential.

The strength of the community is a critical resource and a significant achievement for publishers, who have typically worked in silos with little sharing of information. It is exactly this way of working that has enabled paper mills to target multiple publishing houses in one go, knowing they don’t share information on submissions with each other.

Building trust is difficult, Dr Van Rossum acknowledged. It comes more easily in smaller communities, and it’s harder to build trust across the new larger global community, where submissions pile in from new and often largely unknown sources.

For Dr Van Rossum, the solution lies in continuing to build cooperation and strengthening the community so publishers large and small can share and benefit from the latest technology.

“The most important success is the engagement,” he said. “It’s the more than 40 organizations and high number of people participating in the Integrity Hub. It’s the willingness to share information. It’s the willingness to share technology in a truly collaborative fashion.”

STM Solutions is well positioned to act as the natural forum for the community, Dr Van Rossum said:

For publishers to share content — confidential content — you need a trusted environment, and STM Solutions can cover that environment. Because we are the operational arm of a global association with the aim to advance trusted research, there is a trust in our systems and in our teams and our technology to do that.

He is keen for more publishers to join the Hub, especially smaller organizations that may not have the capacity to establish their own integrity checking systems but that can benefit from the knowledge sharing of larger players.