ROB SCHMITZ, HOST:
MRI scans that allow researchers to peer inside the human brain are great to reveal things like stroke damage, but brain scans have yet to offer much insight into mental health conditions like depression. NPR's Jon Hamilton reports on a new study that suggests why.
JON HAMILTON, BYLINE: In 2018, Scott Marek was working on a paper on how children develop cognitive abilities - in other words, intelligence. Marek, a researcher at Washington University in Saint Louis, planned to use data from a federal study that was scanning the brains of thousands of adolescents.
SCOTT MAREK: What we wanted to do is just ask the question with this huge sample - how is cognitive ability represented in the brain?
HAMILTON: Previous studies had found that intelligence is strongly linked to the thickness of the brain's outermost layer and to the strength of connections between certain brain regions. So Marek's team analyzed nearly 1,000 brain scans. Then, they checked their work using 1,000 more scans.
MAREK: And what we noticed was that we couldn't replicate everything. It didn't look great.
HAMILTON: An area or connection that seemed important in one set of scans might appear insignificant in the other. It was only when they increased the sample size to thousands of brains that the results became more reliable. The team wondered whether this was also the case with other brain scan studies, especially those involving mental illness. So they got brain scan data from about 50,000 people, then used a computer to conduct lots of simulated studies. Dr. Nico Dosenbach, another team member, says they were looking for brain differences associated with a range of conditions.
NICO DOSENBACH: Mood, anxiety, potential abilities, you name it.
HAMILTON: Once again, the team found that it took thousands of scans to get reliable results. But a typical MRI study includes only a few dozen scans. And Dosenbach says those small studies have produced lots of scientific papers on mental illness.
DOSENBACH: But it hasn't really translated to tangibles for patients. And I think these results give us a clue as to why.
HAMILTON: Dosenbach says one problem with small studies is they can only find brain differences that produce large effects, but the effects associated with mental illness tend to be quite small. Another problem, he says, is something called publication bias.
DOSENBACH: If multiple groups are doing the same research using small samples, just by chance, one of the groups of several will have a significant result, and that's what's going to get reported.
HAMILTON: And become the conventional wisdom. Dosenbach says this doesn't mean all of these small studies are necessarily wrong.
DOSENBACH: Even a tiny study could hold true, right? It's just the chances of that happening are much, much, much, much smaller than for an extremely large study.
HAMILTON: So it's risky to apply the findings from a small study to the general population. The team outlined these concerns in March in the journal Nature. And their paper has been getting a lot of attention in the brain-scanning world. Paul Thompson is a neuroscientist at the University of Southern California who was not involved in the research.
PAUL THOMPSON: It's a little like an earthquake in Los Angeles, so it sent a few aftershocks through the neuroscience community here.
HAMILTON: Thompson says the results weren't a complete surprise, though. The field of genetics went through a similar experience with small studies.
THOMPSON: Twenty years ago, you'd hear someone had discovered a gene for criminality or for psychosis or a gene for autism. And then another group wouldn't find the same thing, or they'd find another gene, and people would be scratching their heads.
HAMILTON: The solution has been genetic studies that include millions of people. Thompson says brain scan studies don't have to be that large, and the problem could be addressed by combining the scans from many small studies into one large database. One group already doing that is the ENIGMA Consortium, which Thompson helped create. It maintains a database with more than 50,000 MRI scans. And Thompson says scientists have already used the database to identify brain features associated with schizophrenia.
THOMPSON: There's huge differences all over the brain in schizophrenia. The auditory centers that are involved in hallucinations are abnormal. There's alterations in memory systems and vision systems.
HAMILTON: The differences are far less obvious in depression and bipolar disorder, which means it may take even larger studies to find them. But Terry Jernigan of the University of California, San Diego, says just making studies bigger isn't enough. They also need to be more diverse.
TERRY JERNIGAN: You want to know to what extent your observations are generalizable to all the groups in our society.
HAMILTON: And so far, she says, we don't.
Jon Hamilton, NPR News.
(SOUNDBITE OF EMPRESARIOS' "SIESTA") Transcript provided by NPR, Copyright NPR.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
