How to measure health Medical sciences know two ways of studying health effects of dietary choices in human populations: observational studies and intervention studies. In observational studies, a study group is observed and relevant data are collected, but care is taken not to influence the normal behaviour of study subjects. For example, researchers could record food preferences (organic, conventional), dietary patterns, and health parameters in a study population. This can be done at one occasion (cross-sectional study) or several times (longitudinal study). In intervention studies, researchers control certain parameters. They could, for example, exchange conventional food for organic food in one study group, but not in a control group, and record health parameters before, during, and after the intervention. Both types of studies have their limitations: observational studies do not normally allow conclusions on causal relationships. And long-term intervention studies are expensive and difficult to design. It is easier to study health effects of organic food in laboratory animals. Most environmental factors in animal studies can be controlled, and it is also possible to conduct studies over several generations. It is not always straightforward, though, to draw conclusions for humans. Also, the animals in animal studies do not represent a natural human population with a variety of lifestyles.
Animal feeding trials Animal feeding studies are performed because it is much easier to control the food intake of animals than of humans over long periods. A recent important study in this area compares chicken that were fed organic or conventional feed over two generations8 . Three chicken lines, bread for different immune responsiveness, were used in this study. Two batches of feed were identically composed of ingredients obtained from organic and conventional pairs of neighbouring farms, and the feeds were comprehensively analysed for nutrients in order to avoid nutrient deficiencies. However, the feeds differed to some extent in their nutritional content. For example, the amount of proteins was about 10 percent higher in the conventional compared to the organic feed. No pesticide residues were detected in any of the feed ingredients. A variety of health parameters, many related to the development of the immune system, were measured in the chickens of the second generation. The most important observations, in the breeding line representing the general population, were: 1. chickens on conventional feed grew faster, 2. chickens on organic feed showed a higher immune responsiveness, as measured by the production of antibodies in response to a vaccine, and 3. after an immune challenge, induced by the injection of a protein foreign to the body, the growth rate of all chickens was reduced, but chickens on organic feed recovered their growth rate more quickly. The authors summarize: “The animals on organic feed showed an enhanced immune reactivity, a stronger reaction to the immune challenge as well as a slightly stronger ‘catch-up growth’ after the challenge.” Even other parameters such as feed intake, body weight and growth rate, as well as several immunological and physiological parameters differed between the groups on organic and conventional feed. These differences are not easily divided into positive or negative for the organism. Nonetheless, they cannot be explained by the small differences in organic and conventional feed composition that the authors found. Overall, the enhanced “catch-up growth” in chicken on organic feed is interpreted as a sign of health9 . Generally, in all organisms prioritization of resource allocation takes place all the time. The observations of the chicken study can be interpreted such that the source of feed (organic, conventional) affects prioritization towards growth (conventional feed) or immune system development (organic feed). To date, this has not been subject to any long-term study in humans.
In hundreds of studies, long-term health effects of pesticide exposure have been investigated (see chapter “Public health effects of low-level pesticide exposure”), but few studies directly address the health effects of the consumption of organic food. In the cross-sectional PARSIFAL study with 14 000 children from 5 European countries, children aged 5-13 years in families with an anthroposophic lifestyle, which comprises the preference of organic (or biodynamic) food, had fewer allergies than other children10. This is in line with other studies11, 12 of the anthroposophic lifestyle and allergies in children, but the allergy-protective effect of lifestyle cannot be attributed to the organic food consumption. In the longitudinal KOALA study, which followed about 2 700 babies through childhood, an association was found between the consumption of organic dairy products during pregnancy and infancy and a lower risk for eczema at 2 years of age. This was possibly mediated via a higher content of some ruminant fatty acids in organic milk (see chapter “Composition of animal foods” below)
The Nutrinet-Sainté study is a French-Belgian study on the relation between nutrition and health in a large population. In one sub-study with about 54 000 adult participants, researchers characterized sub-populations of consumers who did or did not prefer organic food with respect to food habits, socioeconomic factors, and body mass index (BMI). Regular consumers of organic food had a substantially lower risk of being overweight (women 28 and men 27 percent decreased risk) or obese (41 and 57 percent decreased risk) compared to the control group of consumers who were not interested in organic food. This association holds even after adjustment for age, physical activity, education, smoking status, energy intake, restrictive diet, and adherence to public nutritional guidelines. Also, participants with a strong preference of organic food did not differ in average household income from the group of participants who were not interested in organic food. Due to the nature of the study (observational, cross-sectional), it was not possible to draw conclusions on what caused the lower observed risk for overweight and obesity among people preferring organic food. The authors speculate, however, that long-term low-level exposure to pesticides could be the cause14. One recent study follows over 600000 middleaged women in the UK over 9.3 years and investigates associations between the intake of organic food (never, sometimes, usually/always) and the incidence of cancer. For all cancers, there was no association between the preference of organic food and cancer. There were, however, weak associations between organic food preference and non-Hodgkin lymphoma (21 percent decreased risk for consumers of organic compared to conventional food) and between organic food preference and breast cancer (9 percent increased risk for organic consumers)116. A small number of short-term dietary intervention studies with conventional and organic food have also been performed15, but with limited scope and without any conclusive differential health effects reported.
Composition of plant foods
Biology By practice and by regulation, fertilization differs between organic and conventional agriculture. Typically, in conventional agriculture, the soil is fertilized with mineral fertilizer containing the plant nutrients nitrogen (nitrate and ammonia), phosphate and potassium (among other minerals). In contrast, in organic agriculture, these nutrients are supplied to the soil mainly in the forms of farm manure, green manure, or other organic materials, while e.g. synthetic nitrogen mineral fertilizers are not allowed. Generally, mineral nutrients are water-soluble and readily available to the plant, while a large portion of the nutrients in organic fertilizers first needs to be decomposed (mineralized), before the nutrients are available to the plant. Furthermore, the total amounts of these nutrients used for fertilization per hectare per year are on average higher in conventional than in organic agriculture, by regulation and in practice. Accordingly, plants in conventional agriculture receive higher amounts of important plant nutrients in a more easily available form, compared to plants in organic agriculture. Are differences in plant nutrient amounts and availability of the fertilizer reflected in differences in the composition of the crops? This is what the theory says: Pro: Biologists sometimes break down plant metabolism into primary and secondary metabolism. Primary metabolism is responsible for basic plant functions such as growth and reproduction, while secondary metabolism is responsible for plant functional diversification, such as defence or appearance. Both the primary and the secondary metabolism are active at all times. Although the classification into primary and secondary metabolism is not clear-cut and represents a simplification, generally associated with primary metabolism are compounds like sugars, carbohydrates, lipids, and many vitamins. Secondary plant metabolites include compounds like phenols, flavonoids, and glucosinolates, among others.
The abundance of plant nutrients (nitrogen, phosphorus, potassium) can influence the balance between primary and secondary metabolism; higher plant nutrient abundance generally causes a shift towards the primary metabolism (sometimes referred to as growth-differentiation balance hypothesis16). This is one reason why conventional and organic crops can be expected to be different in their composition. Contra: Plants (as all living organisms) are homeostatic, i.e. they are able to maintain their functions over a range of environmental conditions. Both conventional and organic farmers strive for optimum growth and health in their crops, and within the range of environmental conditions (here: different fertilization regimes), plants develop equally in both production systems. This is one theoretical argument why conventional and organic crops can be expected to be similar or identical in their composition. Scientific experiments comparing organic and conventional crops are needed in order to test this reasoning.
Comparative studies: types
Three kinds of study designs are used in order to compare the composition of organic and conventional crops:
On one field site, the crop of interest is grown in several field plots with different agricultural practices. Often, there are randomized replicate plots in such field trials. The researchers have control over all agricultural practices used in the experiment, which is very valuable. However, such a field site does not necessarily reflect the diversity of realistic production conditions on farms.
- Farm-pairing studies
In a farm-pairing study, neighbouring farm pairs, one organic and the other one conventional but both producing the same crop, are identified. It is usually left to the farmers to make all necessary decisions during cultivation, e.g. when weeds should be controlled, if irrigation should be used, and so on. Sometimes, farmers are supplied with seeds; otherwise the choice of cultivar is left to the farmer. Such a study is more realistic than a field trial, but it is also more difficult to ensure that the comparisons are valid. If organic farms for example tend to use another cultivar than conventional farms, any observed differences in nutrient contents or other characteristics could be due to different farming practices or differences between cultivars.
- Market-basket studies
Here, samples of fresh produce or processed foods are taken at the consumer end of the distribution chain, for example at markets or supermarkets. In field trials and farm-pairing studies, normally some kind of “best practice” of agricultural management is ensured. In contrast, the range of products offered at a supermarket represents the actual agricultural practice and the distribution chain. This is a relevant perspective for the consumer, but it is very difficult to ensure the general validity of findings. For example, any changes in the supply chain (different farms of origin, changed means of transport, changed storage) may affect the final composition of the products, and are very difficult to control. Thus, the results need to be interpreted with caution. None of these three kinds of studies are able to provide the final answer to the question of the effect a production system has on crop composition. Moreover, depending on the details of the study design, they could lead to different answers to a similar research question. However, dramatic differences in crop composition due to the production system are likely to manifest themselves irrespective of the kind of study design.
A study could be designed to test the hypothesis “organic potatoes contain more vitamin C than conventional potatoes” in Sweden. A controlled field trial would compare potatoes of the same variety in one or several typical potato production areas under conventional and organic conditions, thereby directly measuring the influence of organic and conventional production regimes on this specific variety’s vitamin C content under the given climatic and soil conditions.
However, in Sweden, the most popular table potato variety is King Edward VII. King Edward VII is susceptible to the disease late blight (Phytophthora infestans) and therefore receives fungicide treatment frequently during the growing season. Consequently, King Edward is not well suited for organic cultivation in Sweden. Therefore, a farm-pairing study would most likely collect a different mix of potato varieties from the conventional and from the organic farms. A comparison of vitamin C contents would then measure a mix of the influences of production system and variety on the vitamin C level.
This may be more relevant to the consumer than a field trial, because the farm study ideally reflects the potato varieties available on the market. On the other hand, the popularity of potato varieties changes over time, and differs between countries and even regionally, so care needs to be taken when generalizing such results. Accordingly, the same question, answered using different study designs, may have different answers.
Comparative studies: overview
Minerals, vitamins, antioxidants* are all frequently compared in their concentrations in organic and conventional foods. Macronutrients (protein, total fat, carbohydrates) have generally attracted less interest in this context.
In excess of 150 studies have been published that investigate the content of various nutrients in a wide range of food crops in response to conventional and organic production. The results diverge between studies and it is not easy to draw straightforward conclusions of general validity such as “crops from production system A contain xy percent more of a certain vitamin”. Rather, careful statistical analysis is needed when summarizing all available data, in order to find consistent trends. Several review articles have been published in recent years, summarizing original research. Here, three such reviews are discussed (rather than discussing individual studies) in order to summarize the state of the science in this subject. Further below, the sources of variation between studies are discussed in more detail.
For each nutrient, the reviews report an effect size (i.e. a measure of the magnitude of the difference between production systems) and a statistical significance (i.e. the probability that the observed difference is due to chance)
Organic food crops are more nutritious
Brandt and co-workers published in 201117 a metaanalysis of all 102 available studies since 1992 comparing the content of seven (groups of) vitamins and secondary metabolites in organic and conventional food crops: Total phenolics, phenolic acids, other defence compounds (three groups of plant defense related compounds), as well as carotenoids, flavones and flavonols, other non-defense compounds, and vitamin C (four groups of not plant defence related compounds). According to Brandt’s analysis, plant defense related compounds were on average present in 16 percent higher concentrations in organic crops. Vitamin C was six percent higher, flavones and flavonols were eleven percent higher, and other non-defense compounds were eight percent higher in organic crops. There was no significant difference in carotenoid content between organic and conventional crops. The overall conclusion of the authors was that on average, organic crops contained twelve percent more vitamins and secondary metabolites than conventional crops.
Organic food crops are not more nutritious In a systematic review from 201218, Smith-Spangler and co-workers summarized 153 studies comparing nutrient content in organic and conventional grains, fruits and vegetables. 14 nutrients were included in the comparison. Only phosphorus and total phenols concentrations were significantly higher in organic crops. For most nutrients, Smith-Spangler reports a high statistical heterogeneity, which means that results of the original studies are inconsistent. The authors also raise concern about reporting and publication bias (tendencies to report statistically nonsignificant results incompletely, or to prefer publishing studies with significant findings) in some cases. The authors report the effect sizes as Standardised Mean Differences (SMD), which is common in medical sciences but has no direct intuitive interpretation. The overall conclusion of Smith-Spangler et al. is that “The published literature lacks strong evidence that organic foods are significantly more nutritious than conventional foods.”
Or are they? In a review from 2014, Barański19 and co-workers present the most comprehensive meta-analysis of compositional aspects of organic and conventional crops to date, comparing almost 120 nutrients, and other aspects of food quality from 343 original studies. The authors report a significantly higher content of a range of (groups of) antioxidants in organic food, ranging between 19 and 69 percent for phenolic acids and flavanones.
Organic crops also had a lower content of amino acids and proteins. Many other compounds and groups of compounds did not significantly differ in concentration between the production systems.
The authors provide a structured analysis of the overall reliability of their findings: the findings with good reliability were a small increase in antioxidant activity (measured as Trolox equivalent antioxidant capacity, TEAC), a higher content of flavones and flavonols (sum), and a higher content of flavonols (including single compounds in that group) in organic products. Barański reports, similar to Smith Spangler, indications for the presence of publication bias in the meta-analyses of many compounds. Furthermore, Baranski includes all available peerreviewed studies in the analyses without an evaluation of their quality, in contrast to Smith-Spangler and Brandt, who both apply (different) quality criteria for studies to be included. Data were analysed in two separate ways, in parallel to both Brandt’s and Smith-Spangler’s work, making a comparison with earlier meta-analyses easier.
Overall, this meta-analysis finds a higher systematic content of some groups of antioxidants and secondary metabolites as well as a lower protein, amino acid, nitrate, nitrite and total nitrogen content in organic crops. This is consistent with the principles discussed under “pro” in section “Biology” above, where a low nitrogen availability causes a shift towards the secondary metabolism. The data extracted from the 343 studies are freely available on the internet.
In summary, there is some evidence that organic crops contain higher amounts of vitamin C and some other beneficial compounds, but there is no final agreement. It is important to note that even if there was a systematically higher vitamin C content in organic fruits and vegetables, the difference due to the production system is small (6 percent higher in organic crops according to17), and the variation between cultivars, years, geographical growing locations, climatic conditions, ripeness at harvest etc. are much larger.
The two reviews of Brandt and Smith-Spangler are in apparent contradiction to each other, although it should be noted that they cover somewhat different selections of nutrients. A closer look at the statistical procedures reveals, however, that Smith-Spangler has applied a statistical (Sidak) correction for the large number of comparisons (14 nutrients and 8 contaminants), while Brandt has not.
The “multiple testing problem” is a well-known problem in statistics: the more comparisons of nutrient levels that are made, the higher is the risk of false differences (i.e. differences due to chance alone) being found. A correction can be applied to decrease this risk. This, however, increases the risk of obscuring real differences. If SmithSpangler et al. had not applied such a correction, they would have reported seven of the 14 compared nutrients (vitamin C, calcium, phosphorus, magnesium, quercetin, kaempferol, and total phenols, but not vitamins A and E, potassium, iron, protein, fibres, and total flavanols) in significantly higher concentrations in the organic crops, with the risk that approximately one of the detected differences was false.
Vitamin C is the only nutrient that both Brandt and Smith-Spangler report, and their divergent findings are here discussed in some detail. Brandt reports a statistically significant (p=0.006) six percent higher vitamin C content in organic crops, based on 86 comparisons from 30 published studies. Smith-Spangler reports no significant difference (p=0.48) after Sidak correction for multiple testing, but a significantly (p=0.029) higher vitamin C content in organic food without such a correction (calculated from data in18), based on 31 studies.
The magnitude of the difference is reported as SMD (SMD=0.5), which is not easily translated into a percentage difference. The discussion as to whether organic crops contain more vitamin C than conventional crops appears thus to boil down to a discussion on statistics, i.e. whether it is appropriate or not to apply a multiple testing correction in a meta-analysis of a range of nutrients. There is no final answer to this question, as the appropriateness in part depends on what kind of decisions are to be based on the results.
However, the Cochrane Collaboration, which is renowned for their systematic reviews in medical sciences, state in their guidelines: “Adjustments for multiple tests are not routinely used in systematic reviews, and we do not recommend their use in general”20. It should also be noted that the two meta-analyses of Brandt and Smith-Spangler, and earlier ones, differ in a number of other methodological aspects, including the definition of what kind of data that constitute a data pair for the meta-analysis21. The recent review by Barański allows for a direct comparison of both Brandt’s and Smith-Spangler’s results.
Barański19 finds a 29 percent (p=0.005) or 6 percent higher content of vitamin C in organic food, depending on if studies that have not reported the within-study variation of data are included or excluded. Barański also reports an SMD of 0.33 (p=0.018, without correction for multiple testing). This highlights that the recent meta-analyses indeed are to some extent consistent in their results, yet differences in the treatment of the multiple testing problem lead to different conclusions.
What is causing the variation between studies ?
Different studies may find very different results when measuring the same nutrient in conventional and organic crops. For example, in 113 comparisons of vitamin C in various crops, 23 found more vitamin C in organic and 12 found more vitamin C in conventional crops, while in the remaining comparisons, no significant difference was found18. The reason for this variation between studies lies in the differing study designs, climatic conditions, soils, production years, crops, crop varieties, ripeness at harvest etc., all of which may influence the nutrient content of a plant.
As one illustrative example, quercetin is a plant compound of the flavonoid group. Quercetin has antioxidant properties and is generally desirable as a food component. Figure 1 illustrates how the production system (organic vs. conventional), the production year (2003, 2004, 2005) and the tomato variety (Burbank and Ropreco) all influence the quercetin content in tomatoes. From these data alone, no general trend is apparent as to whether organic or conventional tomatoes have a systematically higher content of quercetin. If the results of many different studies are analyzed together (metaanalysis), such a trend may appear, but it is important to keep in mind that other factors (like the variety) may be equally or more important.
Relevance of comparing nutrient contents
In recent years, it has been increasingly questioned whether it is adequate to describe a food’s value by its content of vitamins, minerals and antioxidants in situations where malnutrition does not generally occur; as one researcher puts it: “Food, not nutrients, is the fundamental unit in nutrition”3 . Focusing on a few compounds neglects the “matrix” they exist in, the fact that any fruit or vegetable is composed of maybe 10000 small compounds, most of them probably with some interaction with the organism that eats it, and/or with other nutrients As an illustrative example, in a recent meta-analysis of 78 scientific studies of vitamins A, C, E, beta-carotene and selenium antioxidant supplements with in total 297000 participants, beta-carotene and vitamin E supplementation seem to slightly increase the mortality rate (number of deaths per 1000 individuals per year) compared to supplementation with a placebo, or no supplementation1 . In contrast, there is strong evidence that a high consumption of fruits and vegetables has positive health effects including a lower mortality rate2 .This is a quite drastic example of the fact that vitamins outside their natural matrix (i.e. our food) are not necessarily “good”. In this example, people received comparatively high doses of isolated vitamins, and it is unlikely that vitamins in their natural concentrations in food would have such an effect. Yet, it is questionable whether the vitamin content of a fruit or vegetable alone is a good indicator of food quality, especially in a setting where vitamin deficiencies are generally rare (such as Western Europe).
In the absence of drastic differences, it is therefore questionable if differences in, for example, vitamin contents between products from different production systems can be directly translated into health claims. As a fruit or vegetable is composed of thousands of compounds, studies of actual health effects are to be preferred over studies of a few nutrients and an extrapolation to health effects.
Overall plant composition Some scientists have measured the influence of the production system on the entire set of expressed genes, proteins, or metabolites, approaches known as “Omics” (transcriptomics, proteomics, metabolomics). Generally these studies have shown that the production system has some effect on the overall plant composition (e.g. 23–25), but there is no easy way of judging whether, or how, the observed effects are of relevance for human nutrition.
For example, in one study, researchers measured approximately 1 600 metabolites (small plant compounds) in organic and conventional white cabbage samples from two years from a controlled field trial26. The production system left a measurable imprint in the cabbage composition that was retained between production years. This imprint was successfully used to predict the production system of samples from one year using data of samples from the other year. However, at present no knowledge about which production system yields the healthier crops can be directly gained from such measurements, because it is difficult to chemically identify so many compounds, and because nutritional science is far from understanding the interplay of so many compounds with the human body.
Significance for adherence to dietary guidelines
In Sweden, the National Food Agency has adopted the Nordic Nutrition Recommendations27 as guidance for the intake of various nutrients in the general population. Recommendations include suggested intakes for macronutrients (carbohydrates, fats, proteins) and a number of vitamins and minerals. Of the ten vitamins and nine minerals for which a recommended intake is specified in the Nordic Nutrition Recommendations, the recent review by Barański19 presents comparisons of three vitamins (B1, C, E) and six minerals (Ca, Mg, Fe, Zn, Se, Cu) in organic and conventional foods. For vitamin B1, no differences were detected. For vitamin C, the content was as mentioned earlier higher in organic crops. Vitamin E, in contrast, was nine or 15 percent higher in conventional crops. Regarding the small potential systematic differences in nutrient composition, and the uncertainty in the meta-analyses (“overall reliability” for vitamin C and E is moderate), these potential differences do not clearly speak in favour of either organic or conventional crops, with respect to meeting dietary recommendations. For both magnesium (Mg) and zinc (Zn), a slightly (less than five percent) higher content in organic crops was found. Although a higher intake of these minerals is generally desirable, the authors argue that these differences are probably not important. For the other minerals, no differences were detected in the meta-analyses of this paper.
It is sometimes discussed that a higher intake of secondary metabolites (such as many antioxidants) in organic produce would increase the “effective intake” of fruit and vegetables, making it easier to meet or exceed the recommendation of eating five portions of fruit or vegetables per day with organic choices. This assumes that the content of secondary metabolites or antioxidants is responsible for the beneficial health effects of a high fruit and vegetable consumption.
However, as discussed above, there is still no general agreement that organic fruits and vegetables have systematically higher contents of such compounds. Also, the Nordic Nutrition Recommendations conclude that, apart from the general advice on fruit and vegetable intake, at present no recommendations towards antioxidant-rich fruits and vegetables (e.g. some berries) can be made27. That is, according to present knowledge, health benefits come with fruit and vegetable consumption, and not specifically with antioxidant-rich fruit and vegetable consumption.
Trends of plant food composition
A large number of studies have been performed that compare the content of a range of nutrients in a range of crops under a range of conventional and organic management practices. Summarizing these findings, if conventional and organic crops differ in the content of specific nutrients, then these differences are small. Sometimes, the belief is expressed that organic fruits or vegetables are “full of healthy stuff”, while conventional food is “empty”. There is no scientific base for this belief. If large differences existed under present farming practices, they would have been found by now One aspect that has received little attention so far is the choice of crop varieties (with their individual characteristics with respect to disease resistance and yield, and their individual ability of taking up trace minerals, or forming some phytochemicals) in the different farming systems. There is substantial evidence that the development of high-yielding varieties during the past half century has had an impact on the mineral content of crops (e.g. 20–30 percent lower concentrations of zinc, iron, copper and magnesium in high-yielding semi-dwarf wheat varieties compared to old varieties in a 160 year experiment, irrespective of fertilizing regime28). Generally, a too strong focus on yield may lead to a breeding of less nutritive varieties29, 30.
However, under current market conditions and facing a global population of 9 billion people in 2050, high crop yields are an important priority. One potential way of combining a high nutritive value and yield is the development of intercropping systems. Another potential way forward is the development of “nutritional yield” concepts30 and their introduction in plant breeding.
Vitamin A deficiency is a public health problem in parts of Africa, Asia, Latin America, and the Western Pacific. Associated with a change from traditional to processed and imported foods, the rates of vitamin A deficiency in Micronesia have increased from zero (before the 1970s) to over half of the children under 5 years being affected (year 2000). Notably, banana varieties have changed from traditional cultivars to Cavendish, which dominates the global banana trade. Research has shown that traditional Micronesian banana varieties have an up to 15 times higher carotene content than Cavendish, which had the lowest carotene levels of the investigated varieties. Local banana varieties, rather than vitamin A supplements, are now promoted for meeting vitamin A intake requirements31–33. With relevance for the present report, farming systems that make use of traditional varieties have the potential of producing more nutritious food.