When two solutions are better than one – meeting the challenge of tracking longitudinal studies

Tasked with creating a reporting framework for the ALSPAC study – recognized as the most detailed study of its kind in the world – most research information analysts would feel more than a little daunted.

But it didn’t take long for University of Bristol’s Zosia Beckles* to come up with a time-saving solution. ALSPAC, or the Avon Longitudinal Study of Parents and Children, is based at the University of Bristol, and has been following families in the region since the early 1990s.

Zosia’s next step was to ask the steering committee how researchers typically cite the data, which led to the discovery that the university had published four landmark, cohort profile papers on the key data slices. This solved her first problem – she could use citations to these key cohort papers as a proxy for data use.

Her plan was to use SciVal to prepare and generate the reports; however, as she explains, that led to her next challenge. “Although SciVal gives you a lot of flexibility and tools, the entity of interest in this case wasn't a researcher, a group of researchers or even an institution, it was a set of documents. And that document set wasn’t static – we weren’t only interested in the four cohort profile papers, we also wanted to learn more about the set of documents citing those cohort profiles, and that is an inherently dynamic, constantly updating, moving target.”

Another issue was that the date range filter options in SciVal's overview module weren’t broad enough – Zosia wanted to track uptake and impact from 2012 onwards – and options to drill down into citing documents were too limited for her needs.

The answer – use Scopus and SciVal together

Zosia has developed a simple two-step process, which draws on the key strengths of both solutions. Importantly, it means that each quarterly report now takes her less than 15 minutes to prepare. The process breaks down as follows:

• STEP 1: Use Scopus to find the documents that have cited one of the four landmark papers.

Zosia has saved the four papers as a list in Scopus: “Now it’s just a couple of clicks to go from that list to a set of documents which cite them. I then use Scopus to export that document set to SciVal.”

• STEP 2: Leverage SciVal's benchmarking module to collect metrics for the desired date range and generate the quarterly reports.

This is where the bulk of the reporting work takes place. “The great thing is, those imported document sets automatically appear in the active publication set panel in SciVal, so it’s very easy to incorporate them into new analyses,” says Zosia.

Using the SciVal benchmarking module, which allows her to set the date range she requires, Zosia collects the following metrics:

• Citation count

• Citations per publication

• Field-weighted citation impact (FWCI) 새 탭/창에서 열기

• Outputs in the top 10% citation percentile

• Collaboration-related metrics

• Raw document count

She continues: “That last one helps me see the size of each individual dataset. I don’t currently collect any metrics on societal impact – the steering committee tends to look at publications like case studies to understand how their data is influencing policy, etc.”

Zosia then copies one of her previous ALSPAC reports: “Because I’m just updating the datasets, rather than running new analyses, it’s quicker than creating a new report. All I have to do then is instruct the analyses to look at the new datasets – which involves checking and unchecking a few tick boxes – select export, and I’m done! Once the initial report setup is done, it's a very quick and easy process to update it as required.”

Lessons learned and next steps

For the steering committee, the insights the reports have uncovered have proved fascinating. Zosia explains: “They can now see at a glance how and where their data is being used. For example, they are interested in levels of academic/corporate collaboration, and internationally and nationally single-authored papers versus larger collaborations. Via the reports, they’ve learned that the study data is not only being used internationally, it has also fed into some very large research projects.”

Another interesting discovery is that it’s not only the four key cohort studies that are highly cited. “We’ve found that the papers using the data are also quite highly cited; not to the same extent, but a lot of them have an FWCI well over two and many are in the top 10% of citation percentiles. The committee can now see that the ALSPAC data has a demonstrable downstream impact, even over and above the cohort profiles themselves.”

In the future, Zosia hopes to enhance her analysis. She says: “Citations to key papers can only ever be a rough proxy data use; obviously, things will be cited for other reasons. And I don’t currently capture research that's ongoing or citations to the data record in our university data repository.” One of the first improvements she plans to explore is collecting the references to that data record and deduping them against her core results.

About the ALSPAC study

The Avon Longitudinal Study of Parents and Children (ALSPAC), also known as Children of the 90s, is a long-term study of three generations.

The study is now being used to look at the prevalence of Covid-19, including asymptomatic cases, and its effects on physical and mental health.

*Zosia shared her case study at Elsevier’s virtual Pan-European User Conference for SciVal & Scopus held in May. The 630+ live attendees included university research officers, librarians and strategy specialists from more than 41 countries in the region. It was the first Europe-wide event of its kind, set up in response to interest from Scopus and SciVal users in how they can measure and understand the impact of Covid-19 on their research activities.