LULU GARCIA-NAVARRO, HOST:
Now we're going to look at the staying power of bad science and a group of volunteers trying to fight it. Take the infamous 1998 study by a British scientist that suggested a link between childhood vaccines and autism. It was retracted more than a decade later. And its author Andrew Wakefield had his medical license revoked. But the myth that vaccines cause autism persists today with serious public health consequences. It's this kind of bad science that James Heathers is constantly looking out for. He's a postdoctoral researcher in behavioral science at Northeastern University. And he's been influential in catching mistakes in publications and getting those papers retracted. Heathers says there are two ways he can tell a paper might have a problem.
JAMES HEATHERS: One is your kind of spider sense as a researcher, which is very difficult to explain. It's a series of heuristics, I suppose, kind of mental shortcuts about what might be wrong with something when you read it. And a lot of the time, it tips you off into the second thing that you do. And the second thing that you do is try to mathematically determine if there's anything in the paper that you see that's inconsistent, if there's a test that doesn't work or numbers that can't exist or something similar to that at least. Between the two of those and, of course, the fact that researchers will tell you sometimes if they have papers in their areas that they don't trust - they think - have problems with - sometimes, they will tell someone like me.
GARCIA-NAVARRO: But isn't there already a system in place for this? I mean, if you get published in a peer-reviewed journal, that is literally you putting your paper out into the scientific community, and then people get to comment.
HEATHERS: Well, that's absolutely what it is. And that's also not the end of the story. Peer review, a lot of the time, is very necessary and very good. The problem with it, when it comes to research that's really bad, is that it's not really designed to detect problems. It's done with the assumption that everything that underlies the research is just fine. It's conducted under a kind of an umbrella of trust. There's not really a culture of strong criticism of bad science that happens through peer review.
The other thing that happens, of course, is that if you - if I write a paper and the paper is deeply problematic and it's rejected from a journal and the journal writes back to the author and says, no, look. We don't believe any of this. This is wrong. This was done incorrectly. These have mistakes that we can't explain - all the authors will do is send it somewhere else. This is called journal shopping sometimes. It happens, of course, with papers that are perfectly fine and accurate as well. But eventually, there's a - there's kind of a saying in science is you can get almost anything published somewhere.
GARCIA-NAVARRO: James, the Wakefield study was eventually retracted. But science is a discipline that builds on itself - right? - with new research citing old research. Have you ever found anything like that, something so egregious that could have an effect on public health?
HEATHERS: There are things that are very similar to that.
GARCIA-NAVARRO: That's very ominous.
HEATHERS: Well, if that sounds ominous, it probably should. Bad medical studies can tip larger results one way or another. So imagine we have six studies, and one of them is problematic for some reason. And when we add up the results of all the six studies and we do a meta-analysis, that one, individual, bad study changes the outcome of the meta-analysis. That can influence how governments buy drugs, how frontline hospital care is conducted and what is considered to be standard care as evidence-based or not so evidence-based. So at its absolute worst, there's very definitely problems like that that exist right now. I'm afraid I can't really speculate any further than that.
GARCIA-NAVARRO: Right now.
HEATHERS: Right now - watch this space.
GARCIA-NAVARRO: James Heathers is a fellow at Northeastern University. And he is the host of a podcast "Everything Hertz." Everyone's got a podcast. Thanks so much.
HEATHERS: (Laughter) You're welcome. Thank you. Transcript provided by NPR, Copyright NPR.