Numerous researchers have pointed out nutritional research’s worrying prevalence of poorly conducted studies and irrational conclusions. Without a drastic reform in our research methods, we will continue to stray further from good scientific practices until we lose all respect. John Ioannidis put it best in his “Challenge of Reforming Nutritional Epidemiologic Research,
“Nutritional research may have adversely affected the public perception of science. Resources for some of these studies could have been better spent on unambiguous, directly manageable threats to health, such as smoking, lack of exercise, air pollution, or climate change. Moreover, the perpetuated nutritional epidemiologic model probably also harms public health nutrition. Unfounded beliefs that justify eating more food, provided “quality food” is consumed, confuse the public and detract from the agenda of preventing and treating obesity.”
If researchers themselves do not approach the subject critically, media outlets and social media influencers will continue using studies to generate attention rather than informing the public of the need to interpret this information cautiously. These concerns have a solid basis.
Nutrition online
A systematic review published in Public Health Nutrition aimed to summarize the level of quality and accuracy [1] for nutrition-related information on websites, e.g., healthy eating, nutrients, nutritional supplements, health outcomes associated with foods, and dietary patterns. Additionally, the review sought to determine whether there was variation between these two factors across websites, social networks, or publishers.
The final analysis included 64 studies published between 1996 and 2021.
- There was a growing trend in the number of publications on the subject, jumping from 1 publication in 1996 to 11 in 2020 (the year with the most publications).
- Most studies did not focus on information published in a specific region; those that did generally evaluated information published in high-income countries
- The most studied topics were disease management (26.6%), general nutrition (23.4%), and maternal and child nutrition (15.6%).
- The number of postings on social media and websites ranged from 4 to 2,770, averaging 165.7 each.
- Studies were classified as having a risk of bias. [2]
Most studies were rated as neutral (51.6%) or negative (28.1%), primarily because of the potential risk of bias in sample selection. More critically, some studies did not use or report methods to measure result reliability.
Quality
Forty-one studies assessed study quality using three different methods [3]
- 48.8% of the studies were classified as poor – “the study's final conclusion was cautious or unfavorable.” The absence of references in the source material was the primary factor contributing to this rating.
- Social media postings were more frequently “poor” 62.5% than websites 47.1%
- Topics related to weight loss and supplements had higher proportions rated as "poor" in quality.
- Topics on child and maternal nutrition showed a higher proportion of "good" ratings – “conclusions that were positive or did not raise concerns.”
Accuracy
Accuracy was analyzed in 47 studies, assessing accuracy compared to authoritative guidelines, academic literature, or scoring systems.
- 48.9% of the studies were classified as poor
- While social media and websites were equally poor at about 50% each, YouTube was even worse.
- Topics related to weight loss and supplements had higher proportions rated as "poor" for accuracy.
- Although evaluated by only one study, some topics received poor ratings (e.g., immune function and sports nutrition).
Quality and Accuracy
Finally, 27 articles analyzed both quality and accuracy.
- Government and commercial websites received mixed scores, with some studies rating them as high and others as poor.
- Organizations and academic institutions were the most favorable for both quality and accuracy.
- Interestingly, Wikipedia was analyzed in two studies: one rated it as high in quality and accuracy, while the other rated it as moderate.
While the review had some significant limitations, such as excluding studies that did not provide the specific data sought and omitting readability (a factor that eases reading) in the search terms, the authors concluded that low-quality and inaccurate nutrition information is prevalent on websites and social networks. With social media becoming an increasing source of nutritional information, this becomes of increasing concern. The authors suggest that accredited experts and nutrition professionals should publish and promote high-quality content while social networks and online platforms implement stricter oversight of the information shared on their sites.
I remain skeptical that accredited professionals alone can resolve this problem. We've all encountered profiles of professionals—often with impressive credentials—who use scientific jargon and claim to base their practices on evidence. Yet, these individuals selectively present biased studies. Pointing out errors or writing critical posts about such professionals may not change the views of their followers. Those who support these professionals will likely continue believing their credibility while dismissing dissenting opinions.
Who watches the watchdog?
Imagine two online sources posting divergent information on a particular topic, both run by reputable professionals. Which one is providing high-quality information? You might say it would be the one presenting the best evidence, and I agree with that. However, it’s not enough to simply “read” the best evidence; proper interpretation is crucial. More critically, could one of these perspectives be dismissed merely because it does not align with the prevailing consensus?
I don't have answers to these questions, but I believe there is no “definitive solution” to the problem; there are currently no gold standard guidelines. Fortunately, some authors have explored the issue and suggested measures to address it.
A review in Health Policy aimed to identify interventions to manage the excess of information, both accurate and inaccurate, which complicates the search for reliable sources. [4] The authors offer four interventions.
The interventions were categorized into four levels of prevention, as follows:
- Primordial: Interventions focused on developing resilient communities and health systems resistant to misinformation. These are the cornerstones of transparency, encouraging dialogue, and providing access to high-quality scientific information – all re-building and maintaining trust in government, science, and health professionals.
- Primary: broad strategies that reduce vulnerability to misinformation through education and training programs to improve digital health literacy and identifying common manipulative techniques, e.g., emotionally evocative content
- Secondary: identifying and addressing specific false or misleading claims and taking action to contain them.
- Tertiary: This stage focuses on reducing the adverse effects of infodemics on public health. This includes fact-checking, forming partnerships with the media sector to prevent inaccuracies or exaggerations in reporting, and platform-based interventions that hinder the spread of harmful misinformation through content moderation and platform regulation.
Many of these interventions appear promising on paper but need to be supported by real-world effectiveness, which remains uncertain.
There is a long way to go, and we will likely continue to face many challenges with misinformation. If I had to summarize everything in a single piece of advice, it would be: Pay attention to the content you consume, the profiles you follow, and especially the information shared by people you trust. Generally, your critical sense is less alert when it comes from someone with similar opinions to yours. In times of rampant misinformation, questioning, and skepticism are good ways to avoid being misled.
[1] Quality included the reliability of the information, balance and impartial, citation of references, and transparency. Accuracy was the factual correctness of information, typically compared to scientific literature or recognized guidelines.
[2] The authors used an adapted version of The Academy of Nutrition and Dietetics Quality Criteria Checklist involving questions of relevance and validation. The authors adopted three questions: Is the review question clear and appropriate? Is the search strategy comprehensive and well-described? Are the processes of abstraction, synthesis, and analysis described and applied consistently? To be classified as positive, all three questions must have positive answers; otherwise, the rating will be neutral or negative.
[3] The criteria include those developed by the authors, DISCERN, “a brief questionnaire which provides users with a valid and reliable way of assessing the quality of written information on treatment choices for a health problem,” and JAMA’s Benchmarks, utilizing “four core standards to evaluate web sites: authorship, attribution, disclosure, and currency.
[4] Although I am generally wary of narrative reviews due to their potential for bias and their tendency to tell a story, I believe this work presents exciting ideas that could contribute to developing a more effective model for health information management.
Sources: Quality and accuracy of online nutrition-related information: a systematic review of content analysis studies. Public Health Nutrition. DOI: 10.1017/S1368980023000873
Beyond misinformation: developing a public health prevention framework for managing information ecosystems. The Lancet Public Health. DOI: 10.1016/S2468-2667(24)00031-8
The Challenge of Reforming Nutritional Epidemiologic Research. JAMA. DOI: 10.1001/jama.2018.11025.
