“Methodolatry”: the veneration of the RCT as the only valid method of investigation, dismissing or discounting valuable input from other sources. A cursory search of the Oxford English Dictionary reveals that methodolatry has not made it in, which is a shame.
In the biomedical model, the randomised, double-blind, and placebo-controlled trial is the “gold standard” trial design, and considered the ideal research methodology on which to base conclusions. Writing that sentence even sounds redundant given the unquestioned acceptance of this as the status quo.
But what is the purpose of any scientific inquiry? Surely a common-case answer from any researcher, regardless of discipline, would be: to bring us closer to a ‘truth’, understanding this is approximate, and use that information to inform practice.
In many of the conversations around evidence in nutrition, and how it is applied, the RCT has taken on a bizarre double-agent role. On the one hand, the limitations of the purely explanatory RCT design for nutrition as a subject of scientific inquiry have been acknowledged, and the subject of much discourse within the field. Outside the field, this is flipped on its head: the limitations of nutrition as a subject of scientific inquiry for the use of RCTs have been the focus of much frustration from biomedical purists arguing the field is all “noise” [see: Ionnidis, John].
This is a somewhat bizarre argument to make: that the issue with is with subject matter, not the method of investigating the subject. To insist this is the case clearly a manifestation of dogma. It is not the first time biomedical dogma has been projected onto other disciplines within health and biosciences. In the 1970’s, the disconnect between the biomedical model and psychiatric investigations led George L. Engel to publish his seminal paper (1) calling for the evolution of the bio-psycho-social model as an investigative paradigm for psychiatry, born out of what he saw as the limitations of the purely biomedical model to investigating conditions of the mind.
Engel characterised the biomedical model as follows:
“The biomedical model was devised by medical scientists for the study of disease…broadly defined, a model is nothing more than than a belief system utilised to explain natural phenomena…
The historical fact we have to face is that in modern Western society biomedicine not only has provided a basis for the scientific study of disease, it has also become our own culturally specific perspective about disease…
The biomedical model has thus become a cultural imperative, its limitations easily overlooked. In brief, it has now acquired the status of dogma. In science, a model is revised or abandoned when it fails to account adequately for all the data. A dogma, on the other hand, requires that discrepant data be forced to fit the model or be excluded.” [Emphasis added].
We can see Engel’s treatise evident in much of the issues in nutrition science today. The limitations of the biomedical model for investigating diet-disease relationships are often overlooked. It is assumed that this model is the cultural imperative for the study of all health sciences, including diet-disease relationships. Rather than modify the model to fit the data, biomedical purists call for the data to be forced to fit the model. The ‘cultural imperative’ for investigation is the RCT; other forms of evidence are discounted, it is often assumed an RCT automatically demonstrates causation, and by default any findings from an RCT must be correct, with the implication that if the finding of an RCT differs to an observational study, the latter must be ‘wrong’.
These assumptions are seldom challenged, although refreshingly there has been increasing challenges to limitations of reductionism and the methodological prejudice of excessive RCT veneration in nutrition science (2-4). While not exhaustive, let’s focus on a number of key issues that arise in this misunderstood model.
1) The biomedical RCT is explanatory
The biomedical model of investigation is based on the guiding principle of reductionism and the elucidation of molecular mechanisms to intervene at a targeted level. The emphasis on double-blinding, strict placebo-control, and randomisation, is predicated upon the presuppositions and assumptions that have to be met if a trial is to have strong internal validity, i.e., to be confident that the intervention caused the outcome. This reflects a primary purpose of the model: testing drug interventions. In this model, a compound is designed with a specific pharmacokinetic and pharmacodynamic profile to have a targeted effect on a specific aspect of physiological function, e.g. ACE-inhibitors inhibit the angiotensin-converting enzyme, in turn lowering blood pressure. A single compound, acting on a specific pathway, for a specific outcome.
The full gamut of characteristics for an RCT with strong internal validity is thought of as an absolute, when the reality is that randomised trials exist on a continuum from purely explanatory [the biomedical RCT] to pragmatic [used often in the bio-psycho-social model] (5). Pragmatic trials emphasis ‘external validity’, are by definition unblinded, and conducted under real-life conditions. The double-blinded, placebo-controlled randomised trial emphasis is internal validity, with the desire for strong adherence to these design conditions to result in more confident demonstrable causality: A causes B. This design does not always adequately translate the effect of a single nutrient on a single endpoint into an accurate reflection of the effect of a complex food matrix of multiple dietary constituents, and an overall diet pattern, on multifactorial disease processes.
Notwithstanding this complexity, the biomedical model has been superimposed onto nutrition. Often, this leads to RCTs examining an isolated compound on a specific biomarker of a disease or aspect of physiological function, to determine whether that relationship may be causal. And while this approach is certainly not redundant, it can lack context: both the wider context of diet patterns as a whole, and the context that modern non-communicable diseases are not diseases of isolated nutrient deficiency. The discovery of micronutrients and their association with specific disease states served public health nutrition well during periods where single-nutrient deficiency diseases with short latency periods were commonplace, helping to eradicate these conditions from the populace. However, there are now clear limitations to triumphing reductionism and the principle that all disease can be distilled down to a single primary source through molecular biology, as an investigative imperative for diet-disease relationships.
Issues with the subject of inquiry – drugs vs. foods – aside, other issues arise with internal validity assumptions and application to nutrition. One of the presuppositions of randomisation is that is balances known and unknown covariates between the intervention and control group. However, this is based on an assumption that no changes or other covariates are introduced after randomisation, i.e., the exchangeability of the two known and unknown variables between groups remains unchanged over the course of the intervention. If this is achieved, the true purpose of randomisation becomes manifest: providing a more accurate estimates of the mean treatment effect, statistical significance, and margins of error [i.e., confidence in the results]. The assumption that factors remain constant between groups, except the intervention agent being introduced, for the duration of the intervention is practically impossible in nutrition trials conducted under free-living conditions. This isn’t an intractable problem, because of the availability of pragmatic RCTs that by design compromise on a degree of internal validity to obtain results with greater applicability in the target population.
Another issue for preconditions of RCT validity is the relationship between research bias, and blinding. Double-blinding is another practical impossibility in most nutrition interventions, given studies require specific instructions to be given by the interventionists, followed by the participants, and then require adherence to the intervention. The diets of participants will change, including in the control group if the researchers don’t ensure the control group are unaware they are in a comparative study. An excellent example of addressing this issue in nutrition science is the Lyon Diet-Heart Study (6), where neither the intervention nor control group knew they were being compared to another group.
The final presupposition underpinning internal validity in an explanatory RCT is that the intervention is clearly defined compared to a placebo. This is easily achievable for drug trials, because the intervention is not a compound habitually consumed, and is easily compared to a zero exposure: a placebo. In nutrition, there is no ‘nutrient-free state’ (7). Even if a supplement trial compared a supplemental nutrient to a placebo of ‘nothing’, the placebo group would still likely have intake of that nutrient through diet alone to a level at least around the recommended daily intake. The reality is that there is no real placebo for food.
There are circumstances, of course, where high levels of internal validity can be achieved. Metabolic ward studies, or research questions focusing on a specific variable [for example, comparing different levels of macronutrients matched across diets]. However, the issues for research findings relative to explanatory biomedical RCTs should not be ignored. In particular, the contentions from certain commentators that because the study of nutrition does not always meet these criteria, that fault lies with the subject of inquiry rather than the applicability of the model, should be treated with skepticism. Recounting Engel: “A dogma…requires that discrepant data be forced to fit the model or be excluded.” Thus, be guarded against that dogma; what is required is a revising of our current models to account for the particular characteristics of the subject of inquiry.
2) The explanatory biomedical RCT better informs the hierarchy of medical evidence
In the biomedical model, because the effect of the compound is typically quite pronounced, trials can use smaller numbers of subjects and still demonstrate that the effect is not by chance. Because a drug – taken once a day – is easy to administer, intervention trials can also be conducted with thousands of participants over a number of years, yielding substantial statistical power. With one specific compound and a placebo, randomisation and blinding is straightforward. The scientific desire for reproducibility is easily achieved, and a large effect size can ultimately be deduced in a meta-analysis, with a confident conclusion on efficacy for the intervention.
Modern nutrition science is defined by chronic lifestyle diseases with long-latency periods, and dietary habits earlier in the lifespan may influence the natural history of a condition long before diagnosis. In this respect, long-term observational studies are a cornerstone investigative tool for the field, a logistical requirement that creates friction with biomedical skepticism over observational research.
However, in the realm of evidence-based medicine [EBM], the ‘gold standard’ dogma has incidentally spawned the second issue, that of the “methodolatry” referred to above. Many of the severe criticisms levelled at nutritional epidemiology are over-exaggerated; the question is whether the empirical model yields actionable conclusions. Although it is a difficult science, the objective reality is that nutritional epidemiology has provided numerous actionable findings that have resulted in improved population health. Critics of nutrition science engage much cognitive dissonance in ignoring the fact that there is less discord between nutritional epidemiology and intervention studies than one would be led to believe (8). Ironically, most of the severe critics of epidemiology also ignore the fact that direct head-to-head comparison between observational research and results from RCTs on various research questions have shown concordance in the findings (9). When combined with mechanistic studies and strong biological plausibility, RCTs are often not required to come to casual conclusions based on the totality of available: establishing the causal relationship between processed meat and colorectal cancer illustrates this point.
Explanatory RCTs form the ‘gold standard’ for the biomedical hierarchy because it is vital to elucidate the effect of an isolated compound on a specific biomarker of a disease or aspect of physiological function, to determine whether that relationship may be causative of the outcome. As stated earlier, applying this approach to nutrition can fail to capture the wider context of diet patterns as a whole, and the context of long-latency chronic conditions not influenced by a single nutrient deficiency, but a multiplicity of dietary factors. For intervention studies, the reality is that a whole-diet, food-based approach to intervention studies is by implication inconsistent with reductionism. Moreover, ethical issues preclude many diet-disease research questions from ever being examined in an RCT context. For example, in an explanatory RCT, assigning participants to a diet deficient in long-chain omega-3 fatty acids with cognitive function as an outcome would be unethical, given what we know about the role of these nutrients in cognitive health over the lifespan, much of which is derived from a combination of observational research and mechanistic studies on nutrient action.
The contribution of explanatory RCTs to conclusions on causation in nutrition science is much less clear cut than in the biomedical model. The top three tiers of the hierarchy of evidence – systematic reviews and meta-analyses, RCTs, and cohort studies, respectively – are much more fluid in nutrition science, rather than distinct cut-offs. Methodological quality is not a given, and in certain circumstances long-term prospective cohort studies form a greater body of evidence than RCTs for informing assessments of diet-disease relationships. In contrast, the ideal of the biomedical model is that multiple RCTs on a given research question are combined into a meta-analysis to yield a strong conclusion on the efficacy of an intervention. Applying this ideal to nutrition, however, often yields inconsistencies and invalid conclusions. Incorporating heterogenous studies, with differing methodology, dose of a nutrient or particular dietary intervention, duration of intervention, population characteristics, and data sources, has the opposite effect; weakened statistical power, and ’null’ findings, the opposite of what a meta-analysis is intended for in concept. While meta-analysis may sit atop the pyramid of evidence, the fact that nutrition studies often fail to satisfy explanatory RCT preconditions and assumptions mean that meta-analyses are often conducted with only a handful of studies and yield useless answers. Many Cochrane Reviews on nutrition questions fall into this criticism.
A meta-analysis in nutrition science.
They key point here is that the hierarchy of nutrition evidence is always a composite, a lens through which to view a total body of evidence, but without considering RCTs and meta-analyses of RCTs impeachable merely due to their historic position in the hierarchy of evidence. Assessment of evidence in nutrition is always dependent on multiple lines of evidence. In Part 2 of this essay, we’ll discuss the issue of incorrect assumptions that are commonly made in respect of RCTs and causation, and finish with the issue of favouring a type of design for evidence, over the true scientific process of evaluation that incorporates “interpretation, synthesis, and extrapolation, and to draw wide-ranging conclusions” (10).
_________________________________________________________________________________
References:
1. Engel G. The need for a new medical model: a challenge for biomedicine. Science. 1977;196(4286):129-136.
2. Messina M, Lampe J, Birt D, Appel L, Pivonka E, Berry B et al. Reductionism and the Narrowing Nutrition Perspective. Journal of the American Dietetic Association. 2001;101(12):1416-1419.
3. Hébert J, Frongillo E, Adams S, Turner-McGrievy G, Hurley T, Miller D et al. Perspective: Randomized Controlled Trials Are Not a Panacea for Diet-Related Research. Advances in Nutrition. 2016;7(3):423-432.
4. Heaney R. Nutrients, Endpoints, and the Problem of Proof. The Journal of Nutrition. 2008;138(9):1591-1595.
5. Tosh G, Soares-Weiser K, Adams C. Pragmatic vs explanatory trials: the Pragmascope tool to help measure differences in protocols of mental health randomized controlled trials. Dialogues Clin Neurosci. 2011;13(2):209-15.
6. de Lorgeril M, Salen P, Caillat-Vallet E, Hanauer M, Barthelemy J, Mamelle N. Control of bias in dietary trial to prevent coronary recurrences: The Lyon diet heart study. European Journal of Clinical Nutrition. 1997;51(2):116-122.
7. Blumberg J, Heaney R, Huncharek M, Scholl T, Stampfer M, Vieth R et al. Evidence-based criteria in the nutritional context. Nutrition Reviews. 2010;68(8):478-484.
8. Satija A, Stampfer M, Rimm E, Willett W, Hu F. Perspective: Are Large, Simple Trials the Solution for Nutrition Research?. Advances in Nutrition. 2018;9(4):378-387.
9. Concato J, Shah N, Horwitz R. Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs. New England Journal of Medicine. 2000;342(25):1887-1892.
10. Swales J. Evidence-based medicine and hypertension. Journal of Hypertension. 1999;17(11):1511-1516.