Science is the business of uncertainty

In its 30^th year, the Oxford Science and Ideas Festival S’ouvre dans une nouvelle fenêtre was designed to bring people together to explore innovative ideas. As part of our support for the festival, we convened a panel to explore a provocative and timely question:

Why is getting it wrong good for science?

With science in the public spotlight more than ever, we wanted to talk about the implications of the uncomfortable truth that science is the business of uncertainty.

Moderator Tracey Brown OBE introduces the panel (Photo by Domiziana Francescon)

The panel, chaired by Sense about Science S’ouvre dans une nouvelle fenêtre Director Tracey Brown OBE S’ouvre dans une nouvelle fenêtre, featured experts with a wide range of experience and expertise:

• Nick Ishmael-Perkins S’ouvre dans une nouvelle fenêtre, Senior Consultant at the International Science Council

• Dr Marian Knight S’ouvre dans une nouvelle fenêtre, Professor of Maternal and Child Population Health at the National Perinatal Epidemiology Unit, University of Oxford

• Dr Alexander M Korsunsky S’ouvre dans une nouvelle fenêtre, Professor of Engineering Science at the University of Oxford (AM)

• Dr Stuart Ritchie S’ouvre dans une nouvelle fenêtre, writer and Lecturer in the Social, Genetic & Developmental Psychiatry Centre at King’s College London

In her introduction, Tracey alluded to the courage it takes to have this conversation:

1. Science isn't always right

This is something that can be uncomfortable to acknowledge – or admit. During the roundtable, we heard from Prof Marian Knight about the safety implications of excluding pregnant women from the initial vaccine trials, and the importance of ensuring that researchers talk to the people they’re trying to help.

As Marian explained:

There were no trials of vaccines including pregnant women until June of this year. And yet we were recommending vaccination for pregnant women from end of December — on the basis of no evidence as to the effectiveness and, importantly … no evidence about safety. So because of a default exclusion of pregnant women, not even offering them the choice to take part in a trial, we're now in the position of huge vaccine hesitancy. I lead investigations of all maternal deaths in the UK, and we've got more women dying in pregnancy of COVID now than we had in the first wave or the second wave because of the very, very low vaccine uptake rates. Contrast that to the treatment trials. The recovery trial, which many of you will have heard about, discovered the benefits of dexamethasone, which then transformed our treatment of COVID. Those trials included pregnant women from the start. So we've therefore been able to ensure that pregnant women can get evidence-based treatments – but we've not got the evidence we need to guide prevention. I hope that we change what we're doing next time to make sure that we include pregnant women in those trials so that we can treat them as well as everybody else.

Watch Prof Marian Knight on the panel

2. Science is a never-ending process

As Prof Alexander Korsunsky explained, the scientific system can be self-correcting. You can never remove “wrong” in science; the question is how to control it:

I’m a researcher, and members of my research group know that saying that we jokingly sometimes use: ‘Never repeat a successful experiment.’ There is a temptation to believe that this is the truth that you found. But you have to be resilient and persistent – to end up wanting to check again and again, again and again. And the point is that it's a never ending process.

As Prof Alexander Korsunsky explained, the scientific system can be self-correcting. You can never remove “wrong” in science; the question is how to control it:

Watch Prof Alexander M. Korsunsky on the panel

3. Science and its communication are at odds

The way we communicate about science belies the way we practice science, Nick Ishmael-Perkins noted. People often want to hear the facts and what they mean, and this is perhaps reflected in how scientists are represented as ‘heroes.’ He said we need to create a public space in which to talk about the actual practice of science:

The way that we communicate about science is very much in tension with the practice of science. Science is the incremental, systematic reduction of uncertainty. But when you think about what you communicate, it's, ‘Tell me the facts, tell me what they mean.’ And that's what people expect.

Those of us who are involved in the practice of science, those of us who are involved in the communication of science – we indulge that: this narrative of the hero scientist, like Einstein went off in a lab somewhere and came up with a solution then told the world and we were forever changed. This is very enduring, but very problematic.

It's really interesting to me because I do think what we need to be doing is to be communicating at least as much about how science functions, how it works, the practice of science, as we are about the people involved and the findings.

Watch Nick Ishmael-Perkins on the panel

4. We're reconsidering the role of peer review

Dr Stuart Ritchie highlighted the Registered Reports S’ouvre dans une nouvelle fenêtre initiative by the Elsevier journal Cortex, whereby study protocols are reviewed before experiments are conducted. He called for this approach to be more widely adopted as a way of focusing more on scientific methods rather than purely results:

The idea is, you send in your method for peer review, and the peer reviewers say, ‘I think you should change that, that won't quite work, use this statistical analysis, do all this, plan it all out, have it all set.’ And then you go off and collect the data. The journal says – and this is the fundamental change – ‘We'll publish it whether it's an exciting result, whether it's completely disappointing, whether it's ambiguous (and) I don't know what it means; we'll publish it either way. ....’ Then as long as you've done what you said you would do, the journal publishes it. I'm actually working on a register report right now with one of my PhD students. And I don't understand why we don't do this for every scientific paper. There are some areas of science where it might not work, but this should a common approach.

Watch Dr. Stuart Ritchie on the panel

5. We need to start getting comfortable with uncertainty

In summary, moderator Tracey Brown suggested that we all need to be prepared to have difficult conversations about the reliability of science and the practice of uncertainty, however disappointing it may be to face the reality that sometimes there aren’t ready answers. That can include rethinking the language we use when talking about failure, for example, and how we communicate risks involved:

Watch Tracey Brown OBE summarize the panel