Dear Editor,
We read with considerable interest the paper by Messmer et al recently published in your journal.1 The issue of the effects of PFAS on cancer incidence is a very important one and it is critical that the research and public health communities investigate the health effects of PFAS exposure. However, we are very concerned that the published paper led by Messmer contains a number of critical methodological flaws which make the results as reported of limited value at best and highly misleading at worst.
We identified the following major problems:
Standardization for age: The analyses presented do not adjust for the distribution of age. Age-standardization or adjustment is essential in most cancer studies because cancer is strongly related to age. Without adjustment for the difference in the distribution of age in the populations being compared, it is impossible to interpret the data/odds ratios presented.
Choice of comparator towns: The control or comparison towns to which Messmer compares Merrimack cancer numbers, were chosen as being similar to Merrimack other than being unexposed to PFAS. A quick look at Table 1 doesn’t support this: Merrimack residents are older than South Portland and Auburn; are more likely to have health insurance than all comparators except Colchester; have far higher median income, higher educational attainment, and are far more likely to own their homes. The Maine Department of Environmental Protection is currently investigating areas near all 3 of the comparison towns for potential PFAS contamination issues.2 Therefore, the comparison with these towns, quite apart from the lack of age standardization, is not likely to be meaningful. According to the protocol published by the Centers for Disease Control and Prevention,3 “the reference population could be the surrounding census tracts, other counties in the state, or the state as a whole (not including the community under study).” A more appropriate analysis would compare Merrimack with the rest of NH outside the immediate PFAS-affected region taking into account differences in age.
Comparison with whole of US: We agree with Messmer et al that there are challenges in selecting “unexposed” comparator communities when PFAS exposure is ubiquitous in the US. However, comparison with the whole of the US without adjustment for age and stratification for race/ethnicity is very problematic due to the known age-, race-, ethnicity-, and regional differences in cancer incidence. A further complexity arises due to likely confounding by arsenic exposure, and by other industrial pollution in the Northeast and elsewhere, which raises serious doubts over the value of a simple comparison of Merrimack with the US to make inferences about the effects of PFAS.
Apparent inconsistent results: Our attention was drawn to the 4-town pooled risk ratio presented in the first line of Table 3. The pooled-town risk ratio of 1.34 is higher than any of the risk ratios comparing Merrimack with the 4 individual towns. In a crude analysis, this must be an error because the pooled estimate cannot be greater than all of the constituent values (0.91, 0.92, 1.09, 1.14). We tried to reproduce the analyses in Table 3 but this was challenging as the paper does not report the case counts or incidence rates. We obtained data from Vermont and Maine as referenced in the paper and unfortunately then identified 2 further issues, described next.
“All-cause cancer”: The paper’s methods section defines all-cause cancer based on 24 of the 27 categories presented in the 2018 Merrimack report, apparently excluding Kaposi’s sarcoma (N < 5), gall bladder (N < 5), and “other” (N = 95) categories.4 In the wider cancer literature, the term “all-cause cancer” conventionally refers to all cancer types, not just 24 selected sites, and the use of the term “all cause” by Messmer et al may mislead readers and lead to incorrect use and/or interpretation of these data in the future. Further, it appears that the publicly available data for Vermont towns gives counts for only 7 (not 24) cancer types [although reference 25 does not provide Vermont data, which we believe are found at another URL5]. In the absence of clear methodology in the paper, we ask whether different definitions of “all-cause cancer” were used for Vermont towns and for the pooled 4-town variable, as shown in Tables 3 and 4? We cannot tell how the 4-town pooled analysis was conducted if Vermont town data included only 7 cancer sites and Maine town data included 24.
Colorectal cancer: We contacted the Maine Cancer Registry and obtained the same data used in the study which included counts for 26 major cancer types; however, counts for colorectal cancer were not provided; instead their data represented “colon excluding rectum.” Colon cancer data from Maine towns should not be compared with colorectal data from any comparator population. This difference in case classification invalidates the comparisons for colorectal cancer between Merrimack and Maine towns, and between Merrimack and the pooled 4-town variable, and by extension the all-cause analyses because they include colon/colorectal counts (Tables 3 and 5). We hope we have overlooked an explanation for this discrepancy in case definitions. If not, multiple results throughout the paper may be incorrect—for “all-cause” and colorectal cancer—as well as the related text in the results, discussion, and conclusions sections.
In addition, we note the following minor problems:
Page 7 (Merrimack v Sanford) “49% higher risk of thyroid cancer (RR = 2.5, 95% CI 1.45-4.32)”—this should say 150% higher risk.
On pages 9 and 11, Risk Ratios for thyroid cancer should refer to 3 towns, not 4, because thyroid data were not available in Vermont towns: Page 9 (Merrimack v pooled towns): “Residents of Merrimack, NH have a 34% higher risk of all cause cancer (RR = 1.34, 95% CI 1.25-1.43), 69% higher risk of thyroid cancer (RR = 1.69, 95% CI 1.19-2.39), 27% higher risk of colon cancer (RR = 1.27, 95% CI 1.02-1.57), and 36% higher risk of prostate cancer (RR = 1.36, 95% CI 1.15-1.6) compared to the pooled risk of residents in 4 unexposed communities (Table 3).” Page 11. “Results indicate that Merrimack residents have a 47% higher risk of thyroid cancer compared to the general US population (RR = 1.47, 95% CI 1.12-1.93) and a 69% higher risk than the pooled risk of residents in 4 unexposed towns (RR = 1.69, 95% CI 1.19-2.39).”
On page 21 below Figure 6, results that are actually from the US comparison are said to relate to the pooled 4-town variable: “Merrimack residents have a 45% increased risk for bladder cancer, 71% increased risk for esophageal cancer, and 141% increased risk for mesothelioma than the pooled risk for the 4 New England towns.”
On page 13, the excess risk of all-cause cancers reported as 34% in Table 3 is stated to be 14% (14% is the Colchester result): “Merrimack residents also experience a 14% higher risk of all-cause cancers, . . .. when compared to pooled data from 4 comparator New England towns.”
In view of the enormous importance of this topic and the need for the New Hampshire community and policy makers alike to have access to sound evidence to inform decision-making and actions, we strongly believe that the paper in its present state falls short; it does not provide the high quality research required to make evidence-based decisions or gain community trust, and several claims in the paper go well beyond the evidence provided.
