*Geek Box: Meta-Analysis

The term “meta-analysis” was coined in 1976 by a psychologist, Gene Glass, as “an analysis of analyses”. The conceptual basis was a means to address the sheer volume of studies published in medical and social sciences, and distill such large quantities of research into a summary estimate of the effect of an intervention or drug.

Conducting a meta-analysis involves:

(a) selecting the primary studies to be included in the meta-analysis;
(b) calculating the magnitude of the effect of the particular exposure/intervention for each study;
(c) assigning a statistical weight to each study, which is determined by the size and quality of the study [the weight is always out of 100%], and;
(d) calculating a summary average of the magnitude of effect from all studies.

This latter purpose has become the primary outcome of meta-analyses conducted by investigators, but meta-analysis can also be used to compare and contrast different studies for differences, and to identify patterns amongst the included studies. Epidemiologists have long argued that these latter two functions would be a much more effective use of meta-analysis for observational research, however, many investigators continue to remain myopically focused on attempting to synthesise the results into a “conclusive” summary estimate.

However, the summary estimate is only precise if the included studies are uniformly similar in methodological quality, and that is certainly not the case for nutrition science. As a result, summary estimates of nutrition studies are often untrustworthy. Meta-analysis in nutrition science can, consequently, be akin to throwing mud against a wall to see what sticks. Because they are considered the “platinum standard” of evidence, their results are often accepted unquestioningly, which is problematic when the results are misleading. Be cautious with meta-analysis; when it comes to nutrition science, their general status of “platinum standard” evidence is not warranted, and they take significant expertise in research methods, and in nutrition as a subject [which many investigators simply don’t have the first clue if we’re being honest], to do well.