The United States Environmental Protection Agency mandates that community water systems (or water utilities) provide annual consumer confidence reports (CCRs) – water quality reports – to their consumers. These reports encapsulate information regarding sources of water, detected contaminants, regulatory compliance, and educational material. These reports have excellent potential for providing the public with accurate information on the safety of tap water, but there is a lack of research on the degree to which the information can be understood by a large proportion of the population. This study evaluated the readability of a nationally representative sample of 30 CCRs, released between 2011 and 2013. Readability (or ‘comprehension difficulty’) was evaluated using Flesch–Kincaid readability tests. The analysis revealed that CCRs were written at the 11th–14th grade level, which is well above the recommended 6th–7th grade level for public health communications. The CCR readability ease was found to be equivalent to that of the Harvard Law Review journal. These findings expose a wide chasm that exists between current water quality reports and their effectiveness toward being understandable to US residents. Suggestions for reorienting language and scientific information in CCRs to be easily comprehensible to the public are offered.
INTRODUCTION
The consumer confidence report (CCR) rule of 1998 (Federal Register 1998) requires all community water systems (i.e., water utilities) to provide annual water quality reports to their consumers. These reports contain information regarding water source, level of any detected contaminants, compliance with drinking water regulations, and relevant educational information. All water utilities and retailers that provide at least 15 service connections or regularly serve at least 25 residents year-round fall under the purview of this rule (US Environmental Protection Agency 1999). Their overarching purpose is to ‘improve public health protection by providing educational material to allow consumers to make educated decisions regarding any potential health risks pertaining to the quality, treatment, and management of their drinking water supply’ (US Environmental Protection Agency 2002).
The need for providing water quality information to consumers has been repeatedly emphasized (Odugbesan et al. 1998; Roper Starch Worldwide 1999; Benson et al. 2002; Means et al. 2002; Meyer-Emerick 2004; Blette 2008); this communication should also be transparent (Pene & Levi 2011). Consumers want more information about their tap water (Roper Starch Worldwide 1999), and have expressed concerns about its quality (Means et al. 2002). The CCR acts as an agent toward improving the public's confidence in their tap water and prompting increased water consumption.
Tap water in the USA is readily available, regulated and monitored for safety, calorie-free, and low cost. Yet, on a given day, half of the US population over the age of 2 years consumes sugar-sweetened beverages (SSB), which represents a significant source of daily calories (Ogden et al. 2011; Huth et al. 2013). ‘Drink water instead of sugary drinks’ is one of the seven key selected messages for consumers in the Dietary Guidelines for Americans (US Department of Agriculture & US Department of Health and Human Services 2010). Unfortunately, SSB manufacturers are not required to publish a beverage quality report while perceptions of water safety based on difficult to read CCRs could represent a barrier to promoting water as an alternative to SSB. The perception that local tap water is unsafe varies by region, and is more common among young adults, those with lower income levels, and among racial/ethnic minorities (Onufrak et al. 2014). Americans have been consuming less tap water while consuming more bottled water and other beverages due to perceived health and safety concerns (Azoulay et al. 2001; Hu et al. 2011). Moreover, the bottled water industry has had a negative influence on perceptions about drinking water quality that has been difficult for water utilities to counteract (Meyer-Emerick 2004).
Efforts to build trust in public water utilities to overcome tap water ‘avoidance’ and reliance on bottled water are needed (Scherzer et al. 2010). The public has to be an ‘informed partner’ if the utility wishes to incorporate new treatment techniques to meet new regulations, undertake expansions of the plant capacity, effectively handle water contamination events, and raise water rates to meet all of the above (Glicker 1992). ‘Persevering trust (or distrust), evocation of negative beliefs about risk from any discussions of ‘contaminants’, or personal experience’ are some factors that may dominate CCR reception by the public (Johnson 2003).
One solution would be for health officials and community leaders to assure the public that tap water supplies are safe (Hu et al. 2011). Implications associated with poor communication can be serious: economic impacts on low-income homeowners, for instance, who may seek more costly sources of drinking water in search of true or perceived safety if they do not have confidence in the safety of their tap water (Blette 2008). For a random sample of New Jersey residents, Johnson (2003) found that reading water quality reports did not shift customers' evaluations of water quality and utility performance from the evaluations of those in the control group, who did not see a report. Water utilities should, thus, actively ensure their consumers receive and understand their CCRs in order to positively impact public perception.
Presenting information that is understandable and meaningful to scientists, engineers, administrators, and to the general public is a challenge for the water quality personnel (Mackenthun 1969). Some important considerations that should be addressed while developing CCRs include:
(a) readability levels and health literacy of the public;
(b) language complexity and use of technical jargon/risk information;
(c) informational design and graphics;
(d) clarity in addressing behavioral recommendations and multiple main messages;
(e) public distrust of tap water based on media reports, prior contamination events, and public perceptions.
The United States Environmental Protection Agency (US EPA) encourages usage of its CCR iWriter software, available on the internet, for maintaining a standardized format for information delivery (US Environmental Protection Agency 2013). While this guarantees that consumers are receiving comparable information, it also limits the ability to educate diverse groups (Meyer-Emerick 2004) and does not include standards to improve the comprehension of CCR messaging. To our knowledge, the effectiveness of published CCRs and the mandated EPA language has not been evaluated. Thus, our objective was to assess the readability of CCRs to determine the degree to which the content is accessible to a broad cross-section of the population, and to compare results to those recommended for public health communications.
MATERIALS AND METHODS
Selection of CCRs
Three water utilities were selected for each of the 10 US EPA regions – one from each size category (Figure 1). Size categories for the utilities are based on the population size they serve: medium (3,301–10,000); large (10,001–100,000); and very large (100,000+). Their population size (and, thus, size category) were confirmed through the US Environmental Protection Agency's (2014b) and water utilities' websites and/or telephonic/email exchange with utility personnel. Consequently, a total of 30 water utilities materials were evaluated. The CCRs were selected from years 2011 to 2013 and obtained through the US Environmental Protection Agency's (2014b) and water utilities' websites. The content of these reports ranged from two-page text blocks in Microsoft Word to illustrated 10+ page reports elaborating information such as source water to plant expansion plans.
Assessing readability
The relationship between the reading ease, grade level, and examples of publication type is presented in Table 1. The FKR tests are reliable, widely used, and correlate well with comprehension measured using reading tests (Chall 1958; Klare 1963; DuBay 2007). With regard to recommendations, the National Institutes of Health (2013) advises that health communication materials be written at a ∼6–7th grade level, and health literacy experts recommend that materials be targeted to the 5th–6th grade reading level (Conrath et al. 1996; Weiss & Coyne 1997). The average American reads at the 7th–8th grade level (National Center for Education Statistics & Kutner 2006). The State of the Union addresses of recent US presidents including Bill Clinton, George W. Bush, and Barack Obama corroborate with these levels since they were written at 8th–10th grade levels as well (The Guardian 2013).
Style . | Flesch reading ease . | Average words/sentence . | Magazine type . | Example . | Estimated school grade completed . |
---|---|---|---|---|---|
Very easy | 90–100 | 8 or less | Comics | 4th | |
Easy | 80–90 | 11 | Pulp fiction | Harry Potter and the Sorcerer's Stone, chapter 2 | 5th |
Fairly easy | 70–80 | 14 | Slick fiction | 6th | |
Standard | 60–70 | 17 | Digests | Reader's Digest | 7th–8th |
Fairly difficult | 50–60 | 21 | Quality | US Department of Defense documents/manuals; life insurance policies in the state of Florida (ease >45) | Some high school |
Difficult | 30–50 | 25 | Academic | Harvard Law Review articles | High school/some college |
Very difficult | 0–30 | 29 or more | Scientific | College |
Style . | Flesch reading ease . | Average words/sentence . | Magazine type . | Example . | Estimated school grade completed . |
---|---|---|---|---|---|
Very easy | 90–100 | 8 or less | Comics | 4th | |
Easy | 80–90 | 11 | Pulp fiction | Harry Potter and the Sorcerer's Stone, chapter 2 | 5th |
Fairly easy | 70–80 | 14 | Slick fiction | 6th | |
Standard | 60–70 | 17 | Digests | Reader's Digest | 7th–8th |
Fairly difficult | 50–60 | 21 | Quality | US Department of Defense documents/manuals; life insurance policies in the state of Florida (ease >45) | Some high school |
Difficult | 30–50 | 25 | Academic | Harvard Law Review articles | High school/some college |
Very difficult | 0–30 | 29 or more | Scientific | College |
Additional data from Ressler (1993), US Department of Defense (1995), DuBay (2007), Kunz & Osbourne (2010), State of Florida (2013), and the website ReadabilityFormulas.com.
The CCRs for chosen utilities were available either as portable document format (PDF) or Microsoft Word (DOC) files. The PDF versions were converted to DOC file using select online tools: PDFOnline.com, FreePDFConvert.com, and PDFBurger.com. Documents were inspected to ensure continuity of all paragraphs and text blocks were intact before running the FKR tests using the readability statistics function in Microsoft Word 2013.
RESULTS AND DISCUSSION
The CCR reading ease ranged from 26.3 to 43.8 (median = 36.45), which is within the academic/scientific level (Figure 2). To provide context, the Harvard Law Review journal has a reading ease in the low 30s (Kunz & Osbourne 2010). Similarly, the CCR grade level ranged from 11.1 to 14.3 with a median value of 12.65 (Figure 3); this is substantially higher than the NIH's recommended 6th–7th grade level for health materials (National Institutes of Health 2013).
An analysis of the reading ease and grade level medians at the utility size level revealed no differences from those of the overall sample size values. The reading ease and grade level of all three utility sizes (medium, large and very large) ranged from 35.5 to 36.8 and from 12.4 to 12.85, respectively. While CCRs from the large and very large categories were generally more detailed, illustrated and visually pleasing, their FKR results were comparable to that of medium-sized utilities.
Such high scores could partly be attributed to mandated US EPA language seen in several CCRs. Below is an example of a typical CCR paragraph under the category ‘general information about your drinking water’:
All drinking water, including bottled water, may reasonably be expected to contain at least small amounts of some contaminants. The presence of contaminants does not necessarily indicate that the water poses a health risk. Some people may be more vulnerable to contaminants in drinking water than the general population. Immuno-compromised persons such as persons with cancer undergoing chemotherapy, persons who have undergone organ transplants, people with HIV/AIDS or other immune system disorders, some elderly, and infants can be particularly at risk of infections. These people should seek advice about drinking water from their health care providers. For more information about contaminants and potential health effects, or to receive a copy of the U.S. Environmental Protection Agency (EPA) and the US Centers for Disease Control (CDC) guidelines on appropriate means to lessen the risk of infection by Cryptosporidium and microbiological contaminants call the EPA safe drinking water hotline at 1-800-426-4791.
With an average of 24.8 words per sentence, this paragraph has a low reading ease of 12.9 and a high grade level of 17.6. Unfortunately, when text exceeds one's reading level, consumers usually stop reading (DuBay 2007). A study (Johnson 2003) on a random sample of New Jersey residents getting CCRs found that many respondents had trouble identifying presence/absence of substance amounts or violations, despite their seeming obviousness (e.g., in a ‘bottom line’ summary on the front page of each report). This seemed to suggest they were not processing the information carefully, although their response patterns were not substantially different from the group responding as a whole. Jordan (1998) advised usage of familiar units, explaining action levels and health effects, and using fewer acronyms and more graphical representations. Using bullet points, fewer words per sentence, simplifying content and using the CDC's clear communication index (CDC 2014) may help to develop clearer and more comprehensible water quality reports for the public.
To illustrate, the above paragraph has been modified with the above recommendations and presented below. The text before the table has a new reading ease score of 52.5 and a grade level of 7.9:
All drinking water has at least small amounts of some contaminants. Even bottled water. This does not necessarily mean the water poses a health risk. Some people are more vulnerable to these contaminants than the general population. They usually have compromised immune systems like
• Cancer patients undergoing chemotherapy.
• People with organ transplants.
• Patients with immune system disorders like HIV/AIDS.
• Older people and infants can get infections easier.
They should seek advice about drinking water from their health care providers. Otherwise, use the phone numbers below:
IMPORTANT PHONE NUMBERS . | |
---|---|
Call us for:
| (555) 555-5555 (business hours) OR (888) 888-8888 (emergency/after hours) |
EPA's safe drinking water hotline for information on
| (800) 426-4791 |
IMPORTANT PHONE NUMBERS . | |
---|---|
Call us for:
| (555) 555-5555 (business hours) OR (888) 888-8888 (emergency/after hours) |
EPA's safe drinking water hotline for information on
| (800) 426-4791 |
Bishop (2003) highlights the need to communicate ‘negative, unpleasant, or unwelcomed information’ to consumers when utilities fail to meet regulations or encounter emergencies (contamination events like the West Virginia chemical spill in January 2014). This is important both during emergencies and in their aftermath in the annual CCR. Being open, ethical, and comprehensible when failing to comply with federal regulations is also essential (Bishop 2003). This should be done while ensuring the readability and understanding are directly addressed using clear, unambiguous statements (e.g., ‘tap water not safe to drink’ and ‘do not drink tap water’). This has the potential to avoid many barriers in communicating risk, especially when CCRs present highly technical information with different possible health outcomes for different subgroups of the population (Berberich 1998). According to Nsiah-Kumi (2008), effective communication is audience-centered, and it is imperative that, in communicating public health messages to communities, we do not neglect vulnerable populations (which includes the linguistically vulnerable). In fact, low literacy is associated with poor health outcomes (Pignone & DeWalt 2006).
When technical information is not effectively rendered for the public, there can be undesirable effects. To illustrate, Wegner & Girasek (2003) found the readability of instruction manuals for child safety seat installation in cars from several manufacturers to be very high; improper use of safety seats is the single strongest risk factor for infant/toddler deaths in traffic accidents (Johnston et al. 1994) and their correct use significantly reduces fatal injury and hospitalization (Kahane 1986). The knowledge gap between experts and the public is wide. So, a good place for water utilities to start is emphasis on identifying what people know at the outset, correcting misinformation, and, subsequently, providing accurate information (Löfstedt & Frewer 1998).
One limitation of the FKR test is that it ignores vocabulary. While readability measurements have some general ability to broadly predict text difficulty, they are not precise, final measures (Pikulski 2002). No mathematical formula can truly measure understanding (Stockmeyer 2009). Future studies on this topic could extend this investigation to evaluate CCRs for cultural appropriateness, the placement of primary messages, and the use of images to reinforce written content using tools such as the simple measure of gobbledygook or SMOG (Wang et al. 2009) and the CDC's clear communication index and risk communication knowledge (World Health Organization 2001). These qualitative and quantitative assessments should be conducted in consultation with seasoned language experts (Oakland & Lane 2004). In addition, there is a need for future research on the benefits of water utility engagement of local residents to act as ‘informed partners’ in decision-making as pointed out by Glicker (1992).
CONCLUSIONS
Over 300 million residents receive water from a water utility mandated to provide a yearly CCR to its customers (US Environmental Protection Agency 2012) and, hence, the CCR has a large audience. To our knowledge, this is the first study to explicitly examine readability of water quality reports for the US population and it demonstrates that current CCRs are not meeting this standard for a large proportion of the population. Thus, a more holistic approach is required toward writing these reports, especially, as water utilities move toward electronic delivery (email/online availability) of CCRs (Carpenter & Roberson 2013).
A potential cost-effective suggestion for developing more understandable CCRs is to modify standard US EPA literature in the CCR iWriter software in conjunction with communication and health experts to the NIH recommended 6th–7th grade level. This would involve reducing words per sentence while maintaining text organization and coherence, breaking away from technical jargon, simplifying vocabulary, increasing reliance on pictures/multimedia, usage of active voice, increasing bulleted/numbered lists, using whole numbers for contaminant levels, explaining contaminant violations and health effects, simplifying messages throughout the report, and providing the underlying scientific information with clarity (‘water is safe to drink’, for instance, should be backed with ‘meets/exceeds all federal/state regulations but not necessarily safe for infants/immune-compromised population’).
The CCR is potentially a powerful resource for meeting several of the public's health information needs in terms of tap water quality and authentic communication in terms of readability and clarity. The public will have confidence in the safety of the tap water only if the water supplier is trusted (Shovlin & Tanaka 1990). Since consumers use their senses and ‘their personal opinions’ to assess drinking water (Dietrich 2006), a readable CCR would play a noteworthy role in increasing this trust. Associated goals like increasing awareness of water conservation can also possibly be better achieved. Finally, by addressing the ease of comprehension and reading level of CCRs, water utilities have an excellent chance of influencing population health through informed decision-making on the degree to which tap water can be consumed safely.
ACKNOWLEDGEMENTS
We acknowledge the support of the Virginia Tech Water INTERface Interdisciplinary Graduate Education Program (IGEP) and the Virginia Tech Institute for Critical Technology and Applied Science (ICTAS).