How to assess medical breakthroughs in a fake news era
For readers to work out what is accurate requires them to read like a scientist – but without the same training
Fake health news is a growth industry. The coronavirus pandemic has provided a ready platform for all kinds of myth-making and false news reporting. But there is another aspect to the reporting of health news that deserves our attention.
It’s how we report and read about genuine health and medical breakthroughs in mainstream media. Usually prompted by the issuing of a press release by a university or hospital, it is interesting to track how it is reported on.
Recently, researchers from the University of Westminster analysed 520 academic papers and the media articles that reported their findings. They wanted to trace how the presentation of scientific knowledge changes as it makes its way from researchers to the general public via the media.
They found that when scientific knowledge is re-contextualised to be disseminated to different audiences, it is not merely rephrased or simplified to make it more accessible. As part of dissemination it also undergoes transformational processes that involve issues of social power, authority and access.
Two of the researchers, Doug Specht and Julio Gimenez, outline some key things that readers of the news can do to spot when science is being reported in a misleading or inaccurate way, and thereby get to what the evidence really shows.
“The main focus of a study is often changed in a way that makes assumptions about how the results might affect people, even in cases when this was not an aim of the research”, they write – citing one of my pet gripes which is the frequency with which research in rats is casually taken to have implications in humans.
“Highly technical language can be changed not just to more common phrases but also more evocative or sensational descriptions. Charts and graphs are replaced with images that make articles appear more related to human experimentation or applications, even where this isn’t the case.”
The reasons for poor quality journalism are many and include time pressures, space limitations and a reduction in specialist reporting roles. But researchers and their PR advisers are not blameless: a small number of press releases blatantly set out to exaggerate the significance of a piece of research.
For readers to work out what is accurate requires them to read like a scientist – but without the same training.
Here are some tips to help us from Specht and Gimenez:
– Be aware of how important information in the original source may be reinterpreted, modified and even ignored depending on what a journalist chooses to present.
– Look at how precise and unambiguous the details presented in the article about the research are. Saying that an experiment has proven a particular fact is a lot stronger than saying it suggests that something might happen in the future.
– Look for a reference or a link to the original source in the report you’re reading. If there is one it’s more likely that the journalist has read the original research.
Check whether the arguments in the article come from the scientists who carried out the research or the journalist. This could mean looking for quotes or comparing with the original research paper.
– Look to see if other radio stations or newspapers are reporting the same stories. If only one news outlet is covering an “amazing breakthrough”, its probably time for some healthy scepticism.
And here are some questions I find useful when assessing the quality of medical research:
– Was the research carried out on humans?
What was the sample size?
–Are side effects discussed?
–Is there a reference to the cost of the research or new drug?
–Is it published in a reputable journal or website?
–Is there a quote from an expert not involved in the research?
Above all, watch out for the breathless reporting of “cures” based on rodent research. In my opinion it should be mandatory for them to carry either a “mouse-trap” or a “smell-a-rat” label.