Of Flavonoids, Cocoa, Brains And Bad Science

Hot chocolate improves brain function, a recent study reports. Turns out it actually didn’t. Here is a tale from the world of bad science.

Flavonoids

You probably already heard about flavonoids, that group of approximately 6,000 chemical compounds found in plants and thought to work as antioxidants. From acting antibiotic to protecting blood vessels, a lot of tales have been woven around them. Some more warranted, some less so.

Previous research already hinted that especially the flavonoids in cocoa could have the potential of  protecting you from heart attacks and cancer. As far as getting headlines from research goes, things can’t get much nicer: tell people to eat chocolate to stay healthy.

And now a group of scientists at Harvard’s Medical School tried to get their bit of that limelight by looking at the potential of flavanol cocoa improving brain functioning. How they did their research is nothing but amazing.

Hot Chocolate Fix

So what did they do? They took two groups of volunteers and gave one of them two daily cups of hot chocolate rich in flavanol (the flavonoid in cocoa), while the other got their two daily doses made from powder where the flavanol had been removed. On the first, second and thirtieth day both groups had to take reaction and cognitive function tests. Connected to that was measuring blood flow in an artery important to the brain (what they call “neurovascular coupling” below) when solving difficult mental tasks – the better the blood transport worked there, the reasoning was, the more access the brain had to the resources it needed.

Where the flavonoid goodness was supposed to come from: cocoa beans.

This so far is nice and dandy and a perfectly valid experimental design, especially when we learn that it was “double-blind,” meaning neither the researchers nor the participants knew who got the flavanol motherloads and who didn’t – no unconscious influencing there.

In the now published abstract, that is available to anyone for free, the results were reported like this:

[…] 30 days of cocoa consumption was associated with increased neurovascular coupling (5.6% ± 7.2% vs ?2.4% ± 4.8%; p = 0.001) and improved Trails B times (116 ± 78 seconds vs 167 ± 110 seconds; p = 0.007) in those with impaired neurovascular coupling at baseline.

and

Conclusion: There is a strong correlation between neurovascular coupling and cognitive function, and both can be improved by regular cocoa consumption in individuals with baseline impairments. Better neurovascular coupling is also associated with greater white matter structural integrity.

Looks good, doesn’t it? Accordingly it was reported like that by various news sources. “Hot chocolate could help fight dementia” the New York Daily News reports.  “Cocoa might prevent memory decline” writes the BBC. “Two cups of hot cocoa a day sharpen seniors’ brains” CBS adds. And on and on.

There Actually Wasn’t Any Effect

But do you notice any discrepancy? Yes, the experiment started with measuring flavanol and brain function, but somehow we end up with cocoa and brain function.

Why? Simple: when you read the full paper, you see that there was no difference in brain function between the groups that got the flavanol and those who didn’t. The researchers were sitting there with a study that showed… nothing.  And that certainly doesn’t get you news coverage. It’s like reporting that you found snow is cold and water wet.

Then, in what I imagine was a rather interesting brainstorming session at Harvard, they found a way to solve their dilemma: they simply declared that they were studying cocoa, not flavanol, and that the two groups actually were one group. Because if you took all participants together, the hot chocolate had made a difference: in 18 of the 60 participants brain functioning had improved. Who cares that it was only those who had started out with a badly working neurovascular coupling to begin with?

Harvard’s motto: veritas (“truth”).

This, put simply, means there was no control group and that anything participants were subjected to could be responsible for the positive outcome: the cocoa of course, but also the water it was dissolved in or simply having to do similar brain tests three times. It is as shoddy as science can get.

I could now again go lament how badly science is reported in the media, but this case here is a bit different. This was not some newspaper hack trying to wring a great headline out of a study that in daylight is less than sensational. This is pedigreed scientists working at a reputable institute doing it themselves and a highly respected scientific journal publishing these doctored results.

Under Pressure

It is simple to explain why this happens: scientists are under pressure to publish papers to get funding and to publish they need good-looking results, while scientific journals need good-looking papers to stay relevant.

When you are a layman interested in fitness and health, who thought that science is the only valid and trustable resource in a sea of snake oil selling, that truly leaves you devasted.

Pictures courtesy of Andrea Goh, Indi Samarajiva and Nathan Forget.