When I opened one of my social media pages today, I saw a message from a friend of mine which was urging people to not believe everything they read, particularly when it comes to issues like safety and health. Instead, one should go directly at the original research articles on a particular issue. In case the reader is not familiar with the scientific jargon, the message was accompanied by one of the many very useful links to blogs that teach a non-scientist how to cleverly read a scientific paper without any specific science training.
Needless to say, I had to spread the message, as I believe in it wholeheartedly. All good and well, but what happens when you encounter two research papers with drastically opposite views on the same topic? What do you do then? Who do you believe?
So I thought pertinent to tell you my short experience with one of these issues and see if we can find a way out of this conundrum. A few days ago, the British Chancellor of the Exchequer (the rough equivalent of a Secretary of the Treasury or Minister of Finance in other countries) announced the introduction of a new tax on sugary drinks: the more sugar a company puts in its drinks, the more taxes it would pay. In his speech announcing the law, Mr. George Osborne was saying that the reason for this law is that there is a positive association between sugar consumption and obesity, meaning the more sugar you eat, the fatter you get. Naturally, he did not cite any studies (he would be a really odd politician if he did so).
Therefore, I started looking for these studies. As a scientist, but not a specialist in nutrition, the first thing I did was searching for reviews on the association between sugar consumption and obesity on peer-reviewed databases (like the Nature journals, the US NIH Library of Medicine, and the Stanford Search Engine). My next step would have been skimming a handful of reviews and then look at their references and select some dozens or so of research papers and read those. But I didn’t get that far and here is why.
At first glance (that is, skimming about a hundred abstracts or so), it seems there are overwhelmingly more papers out there that say there is a positive correlation between sugar intake and obesity in both children and adults. But, when looking at reviews, there are plenty of reviews on both sides of the issue! Usually, the reviews tend to reflect the compounded data, that’s what they are for and that’s why is a good idea to start with a review on a subject, if one knows nothing about it. So this dissociation between research data and reviews seemed suspicious. Among the reviews in question, the ones that seemed more systematic than others are this one and this one, with obvious opposite conclusions.
And then, instead of going for the original research and leave the reviews alone, I did something I am trying like hell not to do: I looked the authors and their affiliations up. Those who follow my blog might have noticed that very rarely do I mention where the research has taken place and, except in the Reference section, I almost never mention the name of the journal where the research was published in the main body of the text. And I do this quite intentionally as I am trying – and urge the readers to do the same thing – to not judge the book by the cover. That is, not forming a priori expectations based on the fame/prestige (or lack thereof) of the institution or journal in which the research was conducted and published, respectively. Judge the work by its value, not by its authors; and this paid off many times during my career, as I have seen crappy-crappity-crap papers published in Nature or Science, bloopers of cosmic proportions coming from NASA (see arsenic-DNA incorporation), or really big names screwing up big time. On the other hand, I have seen some quite interesting work, admittedly rare, done in Thailand, Morocco or other countries not known for their expensive research facilities.
But even in research the old dictum “follow the money” is, unfortunately, valid. Because a quick search showed that most of the nay-sayers (i.e. sugar does not cause weight gain) were 1) from USA and 2) had been funded by the food and beverage industry. Luckily for everybody, enter the scene: Canada. Leave it for the Canadians to set things straight. In other words, a true rara avis poked its head amidst this controversy: a meta-review. Lo and behold – a review of reviews! Massougbodji et al. (2014) found all sorts of things, from the lack of consensus on the strength of the evidence on causality to the quality of these reviews. But the one finding that was interesting to me was:
“reviews funded by the industry were less likely to conclude that there was a strong association between sugar-sweetened beverages consumption and obesity/weight gain” (p. 1103).
In conclusion, I would add a morsel of advice to my friend’s message: in addition to looking up the original research on a topic, also look where the money funding that research is coming from. Money with no strings attached usually comes only from governments. Usually is the word, there may be exceptions, I am sure I am not well-versed in the behind-the-scenes money politics. But if you see Marlboro paying for “research” that says smoking is not causing lung cancer or the American Beverage Association funding studies to establish daily intake limits for high-fructose corn syrup, for sure you should cock an eyebrow before reading further.
Reference: Massougbodji J, Le Bodo Y, Fratu R, & De Wals P (2014). Reviews examining sugar-sweetened beverages and body weight: correlates of their quality and conclusions. The American Journal of Clinical Nutrition, 99:1096–1104. doi: 10.3945/ajcn.113.063776. Article | FREE PDF
By Neuronicus, 20 March 2016