*Geek Box: Food Frequency Questionnaires, Validation and Reliability
Dietary collection methods have been subject to significant recent criticism. While far from perfect, however, understanding their use can help distinguish the valid from excessive criticism. Food-frequency questionnaires are devised to be specific to the population in which they will be applied. It would make little sense, for example, to try and validate frequency of typical foods consumed in the US and apply that FFQ in India.
In developing a FFQ, therefore, researchers aim for the food items to reflect habitual consumption in the intended population. To validate a FFQ, the researchers administer the questionnaire to a representative sample of the population; this representative sample also records a 7-day weighted/measured food diary, where all food and beverages consumed are weighed before consumption. This 7-day weighted diary thus provides as accurate an assessment of intake in a free-living population as is currently available. However, recent studies have used multiple 24-hour recalls to validate an FFQ, and there appear to be similar estimates of intakes and degrees of error using this approach as with a 7-day food record. The FFQ is then compared to the other method [i.e., 7-day food record or 24HR] to assess how well the questionnaire correlated to intake assessed using the other instrument; this process is known as “validation”.
So, are FFQ reliable? It depends on the nutrient that is the outcome of interest, and certain nutrients have a stronger correlation to FFQ than others. For example, saturated fat has a stronger correlation with weighted food diaries than vitamin B3. In this sense, ‘reliability’ is relative. It appears much of the criticism of FFQ stems from an expectation they would yield 100% accuracy. Here’s the thing: the don’t have to. The exposure of interest is average intake over time.
Let’s assume you eat oats regularly: do you eat exactly 100g every time? Some days smaller bowl, other days bigger? Perhaps not eat oats on some days? But over time, maybe you average 100g. And perhaps if you filled out an FFQ, it might yield an average of 60g. While not completely accurate, it still may provide an estimate of the effect of comsuming oats over a 10-year period, but that would be an underestimation of the health effects. And this is important, because generally the effect of measurment error in an FFQ – known as “systematic error” is to underestimate intake, and therefore the relationship between a food and health outcome whether positive or negative.
The question, therefore, of whether FFQ are reliable is relative to the variable of interest they are designed to capture: average intake, over time. And while this may be lower than the true average, it still provides epidemiologists with a tool to broadly, and with an accepted margin of error, assess relationships between the variable of interest [a nutrient or diet pattern], and outcome of interest [disease].