Water Safety Plans (WSPs) are a comprehensive risk assessment and management approach to water delivery that were internationally recommended in 2004. WSPs consist of five implementation steps, followed by evaluation. To date, approximately 90 countries have implemented WSPs; however widespread uptake is limited by lack of documented outcomes and impacts. We conducted a systematic review to collate outcomes, impacts, and lessons learned from WSPs developed in general, rural, and three case-study country contexts. Overall, 53 documents met inclusion criteria. In general contexts, the need for institutional support during WSP implementation was highlighted. In rural applications, the need to simplify the WSP process and provide community support was emphasized. In case-study countries, we found the WSP process was selectively adapted and integrated within existing programs. In outcome and impact evaluations, financial outcomes have the clearest evidence base, while operational outcomes are documented most frequently, particularly in relation to infrastructure improvements. However, evidence is lacking on institutional and policy outcomes and impacts of WSPs. To ensure WSPs reach their potential for improving water delivery and management, support should be provided to implementers, outcomes and impacts of urban, peri-urban, and rural WSP implementations should be evaluated, and adaptation of WSPs locally encouraged.

## ABBREVIATIONS

• CDC

United States Centers for Disease Control and Prevention

•
• DRC

Democratic Republic of the Congo

•
• eWQMS

Electronic Water Quality Management System

•
• HACCP

Hazard Analysis and Critical Control Points

•
• IWA

International Water Association

•
• QMRA

Quantitative Microbial Risk Assessment

•
• UNICEF

United Nations Children's Fund

•
• VWSS

Village Water Safety and Security Plan

•
• WASH

Water, Sanitation, and Hygiene

•
• WHO

World Health Organization

•
• WSP

Water Safety Plan

•
• WSeP

Water Security Plan

## INTRODUCTION

The goal of Water Safety Plans (WSPs) is to ensure drinking water safety by preventing or minimizing contamination. WSPs were laid out in the joint 2004 launch of the 3rd edition of the World Health Organization's Guidelines for Drinking Water Quality (WHO 2011) and the International Water Association's (IWA) Bonn Charter for Safe Drinking Water (Breach 2012). They have their roots in risk management practices such as the multiple barrier approach and Hazard Analysis and Critical Control Points (HACCP), applied in the food industry, and balance the effort of stakeholders involved in water supply between testing of point-of-use water quality and risk assessment/management of the supply. Uptake of WSPs has resulted in implementations in approximately 90 countries, with policy or regulation development in more than 60 countries globally (WHO & IWA 2015).

The WHO Guidelines state ‘the most effective means of consistently ensuring the safety of a drinking-water supply is through the use of a comprehensive risk assessment and risk management approach that encompasses all steps in water supply from catchment to consumer. In these Guidelines, such approaches are called water safety plans' (WHO 2011). Similarly, the Bonn Charter states its primary goal is the provision of ‘good safe drinking water, which has the trust of consumers’, and at the heart of this charter is the call for a risk based WSP from catchment to consumer (IWA 2004).

While the primary goal of a WSP is to protect public health, potential additional WSP benefits include: improved compliance with regulatory and other requirements; improved consumer trust; improved confidence of key stakeholders; cost effectiveness and investment planning; improved staff commitment; and pricing competition with peers (Breach 2012).

The five key steps in developing a WSP are: (1) Preparation; (2) System assessment; (3) Monitoring; (4) Management and communication; and (5) Feedback and improvement (Bartram et al. 2009). Two different models of WSP implementation exist, the full process comprised of 11 modules and the process for small systems comprising six tasks (Figure 1).
Figure 1

Steps of the WSP process and the associated modules and tasks (Bartram et al. 2009; WHO 2012).

Figure 1

Steps of the WSP process and the associated modules and tasks (Bartram et al. 2009; WHO 2012).

While validation and verification of the WSP occur in steps 2 and 3 of the process (Mudaliar 2012) outcome and impact evaluation of a WSP is a separate, external activity to determine the overall outcomes and impacts of the WSP on the water supply chain. To date, outcome and impact evaluation of WSP implementation is lacking, which leads to stakeholder uncertainty in the value of developing and utilizing a WSP; however, a framework has been suggested by the CDC and to date larger scale evaluations have been planned by WHO (Williams 2008; Gelting et al. 2012; Aquaya 2015).

Additionally, rural communities present a special challenge to WSP development, as there may not be a primary water utility managing the supply, the supply may be intermittent or from sources lacking infrastructure, and capacity for water quality monitoring may be low (WHO 2012; Rickert et al. 2014). WHO has noted there are knowledge gaps in the use and implementation of WSPs in rural settings and, in particular, community managed supplies (WHO 2012).

As part of a United Nations Children's Fund (UNICEF)–Tufts University WSP project, we conducted a systematic literature review on WSPs in general contexts, rural implementations, and in three countries/areas of interest to the project: India, the Democratic Republic of the Congo (DRC), and the Pacific Islands, specifically Fiji/Vanuatu. The goal of this systematic review was to collate outcomes and impacts from WSP evaluations and lessons learned from WSP implementations in general, rural, and country-specific contexts.

## METHODS

Documents were gathered via database searching, utilizing PubMed, Engineering Village, and Google Scholar. Key search phrases included: ‘WSPs’, ‘water utility management’, ‘water utility risk management’, ‘rural water supply risk management’, ‘evaluation of risk management water supply’, ‘water risk management policy’, and ‘evaluation WSP’. Due to the size of the Google Scholar database, further down selection was conducted during the search, including limiting years of document publication to 1995–2015 and controlling specifically for the phrase ‘WSP’. Additionally, the IWA/WHO Water Safety Portal was manually searched for documents and manuals, and publications were solicited directly from personal connections and the Household Water Treatment Google Group. Secondary literature searches were conducted by reference tracing in the identified literature. Lastly, UNICEF staff and local partners in the three focus countries provided information and documentation relevant to their country context.

The down selection process for documents returned from all search venues involved first evaluating the titles of potential papers, then reading abstracts, and finally reading the full-text of selected papers. At each stage, documents were evaluated based on the inclusion and exclusion criteria presented in Table 1.

Table 1

Inclusion and exclusion criteria used to identify relevant WSP documents

Inclusion criteriaExclusion criteria
Language: English, French, Spanish Language: any language other than English, French, or Spanish
Publication: 1995-Nov 1, 2015 Publication: pre-1995
General WSP manuals/guidelines or associated guideline for implementation or evaluation Specific WSP manuals/guidelines developed for non-generalizable situations (i.e. WSP in Buildings Manual, country specific policies)
Case studies of WSP implementation in general or rural setting with documented outcomes Case studies which presented no outcomes, no clear examples indicating WSP process was followed, or a lack of information to draw conclusions
Any document pertaining to WSPs in countries of interest secured by UNICEF Documents that were partial examples of a WSP, were Powerpoint presentations lacking detailed information, or were summaries of already published manuals
Inclusion criteriaExclusion criteria
Language: English, French, Spanish Language: any language other than English, French, or Spanish
Publication: 1995-Nov 1, 2015 Publication: pre-1995
General WSP manuals/guidelines or associated guideline for implementation or evaluation Specific WSP manuals/guidelines developed for non-generalizable situations (i.e. WSP in Buildings Manual, country specific policies)
Case studies of WSP implementation in general or rural setting with documented outcomes Case studies which presented no outcomes, no clear examples indicating WSP process was followed, or a lack of information to draw conclusions
Any document pertaining to WSPs in countries of interest secured by UNICEF Documents that were partial examples of a WSP, were Powerpoint presentations lacking detailed information, or were summaries of already published manuals

In full text review of down selected documents, documents including lessons learned or evaluation outcomes and impacts were identified, grouped into four categories, and analyzed. The four groups included: (1) all studies for general context lessons learned (further sub-grouped into the five steps of WSP implementation); (2) lessons learned from rural contexts; (3) lessons learned from case-study countries; and (4) outcomes and impacts from evaluations of WSPs. Documents could be included in more than one group.

## RESULTS

Key phrase searching of PubMed garnered 608 possible documents, of which 76 met the inclusion criteria, while the Engineering Village database had 8,646 hits, of which 114 met inclusion criteria. Google Scholar searching identified 1,330 documents that met the inclusion criteria after the more stringent search parameters were included. Of the documents on the IWA Water Safety Portal, 108 were identified that met inclusion criteria. Three documents were included from reference linking. In total, 1,627 documents met inclusion criteria. After full-text reviews, 53 documents were included, of which: 12 were included for general WSP lessons learned; 12 were pertinent to the rural implementation context; 11 pertained to WSPs in case-study countries; and 24 contained evaluation outcomes and impacts (Table 2).

Table 2

List of documents included in review

General WSP steps
Author (Date)Manual12345Rural WSPCase studyEval
Barrington et al. (2013)        ✓
Bartram et al. (2009)   ✓ ✓ ✓ ✓ ✓
Breach (2012)  ✓ ✓       ✓
Author unknown (2015a)         ✓
Davison & Deere (2007)          ✓
de Souza et al. (2011)      ✓ ✓   ✓
Dyck et al. (2007)          ✓
Environmental Science and Research (2015)         ✓
Gelting et al. (2012)          ✓
Author unknown (2015b)         ✓
Greaves & Simmons (2011)  ✓      ✓
Gunnarsdottir et al. (2008)          ✓
Gunnarsdottir et al. (2012a)          ✓
Gunnarsdottir et al. (2012b)          ✓
Gunnarsdottir et al. (2015)          ✓
Hasan & Gerber (2010)          ✓
Hasan et al. (2011)        ✓
Howard et al. (2005)          ✓
Hubbard et al. (2013)   ✓    ✓   ✓
IWA (2004)  ✓
Jalba et al. (2010)   ✓
Kabir & Gedam (2012)         ✓
Keirle & Hayes (2007)    ✓
Kooy & Bailey (2012)         ✓
FMoWIE (2015)  ✓      ✓
Lockhart et al. (2014)  ✓        ✓
Mahmud et al. (2007)      ✓  ✓  ✓
Mälzer (2010)          ✓
Mudaliar (2012)          ✓
Mudaliar et al. (n.d.)  ✓      ✓ ✓
Nath (2013)         ✓
NEERI (2013)         ✓
Ncube & Pawandiwa (2013)      ✓
Perrier et al. (2014)        ✓
WSNH (2009)  ✓      ✓
Rickert et al. (2014)  ✓      ✓
Rinehold et al. (2011)   ✓ ✓ ✓ ✓ ✓   ✓
Rizak et al. (2003)          ✓
Saltori (2013)         ✓
Singleton (2014)         ✓
Smeets et al. (2010)          ✓
Summerill et al. (2010a)   ✓   ✓
Summerill et al. (2010b)      ✓    ✓
Timilsina (2012)        ✓
UNICEF & GWP (2014)  ✓      ✓
Vieira (2011)          ✓
WSP (2010)         ✓
Williams (2008)          ✓
WHO (2011)  ✓
WHO (2012)  ✓      ✓
WHO (2014)          ✓
WHO & IWA (2015)          ✓
Zimmer & Hinkfuss (2007)      ✓
General WSP steps
Author (Date)Manual12345Rural WSPCase studyEval
Barrington et al. (2013)        ✓
Bartram et al. (2009)   ✓ ✓ ✓ ✓ ✓
Breach (2012)  ✓ ✓       ✓
Author unknown (2015a)         ✓
Davison & Deere (2007)          ✓
de Souza et al. (2011)      ✓ ✓   ✓
Dyck et al. (2007)          ✓
Environmental Science and Research (2015)         ✓
Gelting et al. (2012)          ✓
Author unknown (2015b)         ✓
Greaves & Simmons (2011)  ✓      ✓
Gunnarsdottir et al. (2008)          ✓
Gunnarsdottir et al. (2012a)          ✓
Gunnarsdottir et al. (2012b)          ✓
Gunnarsdottir et al. (2015)          ✓
Hasan & Gerber (2010)          ✓
Hasan et al. (2011)        ✓
Howard et al. (2005)          ✓
Hubbard et al. (2013)   ✓    ✓   ✓
IWA (2004)  ✓
Jalba et al. (2010)   ✓
Kabir & Gedam (2012)         ✓
Keirle & Hayes (2007)    ✓
Kooy & Bailey (2012)         ✓
FMoWIE (2015)  ✓      ✓
Lockhart et al. (2014)  ✓        ✓
Mahmud et al. (2007)      ✓  ✓  ✓
Mälzer (2010)          ✓
Mudaliar (2012)          ✓
Mudaliar et al. (n.d.)  ✓      ✓ ✓
Nath (2013)         ✓
NEERI (2013)         ✓
Ncube & Pawandiwa (2013)      ✓
Perrier et al. (2014)        ✓
WSNH (2009)  ✓      ✓
Rickert et al. (2014)  ✓      ✓
Rinehold et al. (2011)   ✓ ✓ ✓ ✓ ✓   ✓
Rizak et al. (2003)          ✓
Saltori (2013)         ✓
Singleton (2014)         ✓
Smeets et al. (2010)          ✓
Summerill et al. (2010a)   ✓   ✓
Summerill et al. (2010b)      ✓    ✓
Timilsina (2012)        ✓
UNICEF & GWP (2014)  ✓      ✓
Vieira (2011)          ✓
WSP (2010)         ✓
Williams (2008)          ✓
WHO (2011)  ✓
WHO (2012)  ✓      ✓
WHO (2014)          ✓
WHO & IWA (2015)          ✓
Zimmer & Hinkfuss (2007)      ✓

## GENERAL LESSONS LEARNED

Of the 12 documents that generated general WSP lessons learned, six provided lessons for Step 1: Preparation, three for Step 2: System assessment, three for Step 3: Monitoring, eight for Step 4: Management and communication, and four for Step 5: Feedback and improvement.

### Step 1: Preparation

In order to successfully prepare for a WSP process, multiple documents highlighted the need to: (1) assemble a well-supported interagency team led by the water utility (Bartram et al. 2009; Summerill et al. 2010a; Rinehold et al. 2011; Breach 2012; Hubbard et al. 2013); and (2) include the public health department in the process (Jalba et al. 2010; Summerill et al. 2010a; Hubbard et al. 2013).

Additionally, authors noted that carefully choosing the WSP coordinator, valuing the opinion of WSP team members with fresh perspectives (Bartram et al. 2009), having a team committed to long-term implementation of the WSP (Rinehold et al. 2011), and recognizing a WSP's value as a capital improvement-planning and money-saving tool (Bartram et al. 2009; Rinehold et al. 2011) led to successful WSP planning.

### Step 2: System assessment

The process of assessing a water supply system encompasses Modules 2–5 or Tasks 2–4, including describing the water supply system, identifying hazards and hazardous events and their associated risks, determining and validating control measures, prioritizing risks, and developing, implementing, and maintaining an improvement or upgrade plan.

Keirle and Hayes called for broader stakeholder involvement in water supply after identifying that the catchment area is often beyond the jurisdiction of water companies in England and Wales (Keirle & Hayes 2007).

Bartram et al. noted through several case studies that utilizing existing data to determine the current water quality was critical to establishing discrepancies in perceived and actual quality, leading to better established control measures (Bartram et al. 2009). They also noted the importance of reaching agreement upon which water quality standards to use for validation of control measures and that validation of each control measure was required unless there was a confirmed robustness of existing data on certain controls. Furthermore, the importance of including hazards that occur on the consumers’ premises in the risk assessment was documented.

Rinehold et al. corroborates this by noting that point-of-use assessment is needed, in addition to testing of the quality of the supply, for consumers that store and treat water at home (Rinehold et al. 2011). They further noted that the water quality standards should be clearly defined and locally appropriate, and in some cases a qualitative approach to prioritizing risks should be considered.

### Step 3: Monitoring

Monitoring includes both defining the operational monitoring of the control measures outlined in Step 2 and verification that those measures continue to be both functional and relevant, through an audit or assessment. Monitoring parameters chosen for a particular WSP will be dependent upon existing regulations, the availability of test equipment and expertise, the type of supply system, and the financial cost of utilizing a measure.

Bartram et al. noted that monitoring is an integral part of the WSP approach as it informs whether public health goals are being met (Bartram et al. 2009). They further observed a benefit that clearly defined operational monitoring plans helped to eliminate conducting irrelevant tests. Rinehold et al. noted monitoring not only ensures compliance but also offers a basis for identifying needed improvements (Rinehold et al. 2011). Furthermore, they observed that monitoring water quality can be difficult in resource-constrained systems, and this step of WSP implementation can be used to strengthen monitoring and improve water safety.

To aid the WSP community in verification of WSP effectiveness, the WHO and IWA published a practical guide to auditing WSPs (WHO & IWA 2015). Divided into four audit categories, internal or external and formal or informal, the guide contains suggestions for why each type of audit would be performed and who would conduct it. The goal of a WSP audit, ideally carried out independent of the WSP team, is to: provide feedback on WSP implementation and maintenance; provide a critical assessment of the effectiveness of the WSP; and to confirm compliance of the WSP with any regulation. Additionally, the guidelines call for the development of training tools and certification processes for auditors. Ultimately, the results of an audit, in conjunction with regular review of the WSP by the implementer, help to inform and strengthen the overall WSP process.

Bartram et al. observed that informal audits conducted in one case study helped to identify lapses in both data collection of monitoring parameters and in dissemination of information to key stakeholders (Bartram et al. 2009). In another case study they noted that the regulator, acting as an external auditor, planned to audit certain components of a WSP regularly for compliance and the WSP in its entirety on occasion.

### Step 4: Management and communication

The management and communication step of the WSP process calls for the development of a standard operating procedure for the water supply system as well as the development of support programs, such as staff training and preventative maintenance.

Summerill et al. recognized that organizational culture has an influence on the effectiveness of WSP implementation and that leaders are key in not only developing this mindset but also in challenging the beliefs and attitudes of staff (Summerill et al. 2010b). Zimmer and Hinkfuss noted resistance to implementing WSPs may result from uncertainty and misconceptions surrounding the process (Zimmer & Hinkfuss 2007). Yet others acknowledged when staff are encouraged to collaborate they have been found to share their knowledge with each other and with colleagues not on the WSP team (Mahmud et al. 2007; Ncube & Pawandiwa 2013).

Several authors noted the importance of the WSP team having adequate experience and expertise in communication (Bartram et al. 2009; de Souza et al. 2011), while Rinehold et al. highlighted the need for thorough operator training (Rinehold et al. 2011). Proper training allows a team to prioritize and put in place an improvement plan when significant risks are identified, which helps a team avoid complacency and neglect of long-term safety improvements (Bartram et al. 2009; Summerill et al. 2010a).

### Step 5: Feedback and improvement

In addition to regular monitoring of the system, the WSP needs long-term feedback and continual improvement via reviews and assessments by the WSP team because of the iterative nature of the process.

Rinehold et al. indicated the WSP implementation team needs to determine how WSP success will be measured and what metrics will be used to evaluate it, while Bartram et al. noted that it is imperative the assessment methodology be clear to allow for consistency (Bartram et al. 2009; Rinehold et al. 2011). Hubbard et al. noted that an infrequent reliance on benchmarks and indicators to monitor improvements and provide valuation is a hindrance to scaling up of WSPs (Hubbard et al. 2013). Bartram et al. also documented that paper-based WSPs were difficult to keep updated as improvements to the system were implemented, and found in another case utilities maintaining WSPs on an intranet with version-control allowed for faster updating (Bartram et al. 2009). Furthermore, they observed from case studies that a well-defined plan for responding to different types of incidents allowed teams to respond quickly and provided a process for WSP revision if necessary.

The conclusions drawn from lessons pertaining to each step of the WSP process highlight the need for institutional support and collaboration, clear communication of goals, knowledge, and data amongst WSP implementers, and well thought-out organization of the WSP team in order to accomplish WSP implementation and upkeep. While many conclusions relate to organizational culture of the WSP team and development of the WSP document, there are also important conclusions that can be made around the choice of indicators for system assessment, monitoring, and auditing.

## RURAL LESSONS LEARNED

Several guidance manuals have adapted WSPs to community-managed water supplies, such as those found in rural settings (Mudaliar et al. n.d.; Espinzoa et al. 2009; Greaves & Simmons 2011; WHO 2012; Rickert et al. 2014; UNICEF & Global Water Partnership 2014; Federal Ministry of Water Irrigation and Energy of Ethiopia 2015). WHO modified the eleven modules to six tasks and provided example programming geared towards professionals working towards implementation of WSPs in small communities (WHO 2012). This document was expanded upon in 2014 to include simplified templates and guidance for community members introducing WSPs in community-managed supplies (Rickert et al. 2014). Implementers in Honduras refocused WSP requirements for community management and simplified the language of standard WSP manuals to meet the needs of rural communities (Espinoza et al. 2009). Global Water Partnership and UNICEF modified the WSP for small communities to a WSP-Plus, which included sub-tasks to assess the risk of environmental and climate change hazards (UNICEF & Global Water Partnership 2014). The Ministry of Water, Irrigation, and the Environment in Ethiopia developed detailed guidelines for the implementation of Climate Resilient WSPs in community-managed supplies, tailored to existing community and local governance structures (Federal Ministry of Water Irrigation and Energy of Ethiopia 2015). The Drinking Water Safety Planning guide for Pacific Island countries provides guidance in navigating not only the creation of WSPs but also in establishing national support processes and frameworks for rural WSPs (Mudaliar et al. n.d.).

Four case studies provided insight into WSP implementation in rural community supplies. Early adopters of WSPs in rural water systems in Alberta, Canada found ‘interpersonal relationships’ and ‘communication dynamics’ between operators and community or provincial officials were managed more effectively by the WSP process (Perrier et al. 2014).

In a study on WSP implementation in remote regions in the Pacific Islands, it was suggested that trained local facilitators assume the role of monitoring in place of the national government. WSP simplification was needed and it was suggested that modified sanitary inspections and hydrogen sulfide presence/absence tests be used in place of more formal monitoring (Hasan et al. 2011). Barrington et al. corroborated the need for minimal water quality testing, a focus on hazard identification, and control measure implementation in a study on pilot WSP implementations in Nepal (Barrington et al. 2013). While Mahmud et al. found the use of sanitary inspections to be a useful indicator of improved source water condition, with a decrease from 8% to 1% of water sources in the high-risk category and an increase from 69% to 89% of sources in the low-risk category in pilot projects (Mahmud et al. 2007). In comparison, microbiological data indicated that 20% of supplies were reduced from a high or very high-risk rating.

Several authors also noted the need for interactive workshops and tools, such as flow diagram mapping, hazard identification, and the development of pictorial tools for community monitoring, coupled with water, sanitation, and hygiene (WASH) education in community-based management (Mahmud et al. 2007; Hasan et al. 2011; Timilsina 2012; Barrington et al. 2013). During microbiological testing, Mahmud et al. found 20% increase in household water samples in the low- to no-risk category and 15% decrease of household samples in the high- to very-high-risk category after hygiene promotion under WSP pilot implementation (Mahmud et al. 2007).

## CASE STUDY LESSONS LEARNED

Twelve documents have described the state of WSPs in the four case study countries: five in India, two in the DRC, and four in the Pacific Islands.

### India

In India, individual states develop water and sanitation service delivery strategies. A primary concern is water supply, with many states focusing on the development of Water Security Plans (WSePs) (National Environmental Engineering Research Institute 2013), as distinct from WSPs. The goal of a WSeP is to ensure that every rural person has enough safe water to meet their drinking, cooking, and domestic use needs at all times and in all situations.

UNICEF operates in 14 states in India, of which five have developed WSePs/WSPs for piloting (2015). One program that UNICEF has been involved with is the Bio Village Project in 110 villages (approximately 24,000 families) in Maharashtra to develop a Village Water Safety Security Plan (VWSS) (Kabir & Gedam 2012). The VWSS is divided into activities in support of source management, water system management, and water quality monitoring and surveillance. After a pilot period, it was found that: 85% of villages had conducted their own water supply repairs; nearly 100% had appointed a water manager and conducted operation and monitoring training; the water tax recovery increased in the first district by 28% and in the second district by 8%; and 98% of villages were consistently chlorinating water supply; but only 56% maintained regular logs. Key recommendations were made for taking this initiative to scale, including: (1) triggering community behavior change through participatory approaches; (2) obtaining external support, especially for the training and capacity building to the districts; and (3) developing process guidelines and appropriate tools for conducting training of trainers. VWSS Plans in Chandrapur District operate under a larger District WSP that provide a water quality monitoring framework, laboratories, and guidelines for infrastructure planning and funding allocation to villages (Author unknown 2015a).

The Sulabh International Academy of Environmental Sanitation and Public Health (SIAES) and WHO India have developed specific guidelines for different types of water supply systems in India, which are integrated into broader Government of India guidelines on WSePs (Nath 2013). Additionally, the Water and Sanitation Program has produced a detailed report on policy issues related to implementation of WSPs in rural India (Water and Sanitation Program 2010). Key policy issues that have been identified are: adopting WSPs; establishing roles and responsibilities; demonstrating that WSPs are a basis for investment; setting performance targets to reflect health objectives; and developing policies on interventions. They also identified the need for reporting and auditing, district planning coordination, and baseline surveys of water quality as critical functions to support WSP implementation.

### DRC

The DRC government-operated national WASH development program, Village Assaini, was established in 2006 by the Ministry of Health and Ministry of Education to contribute to achieving the Millennium Development Goals, with a goal of reaching 2,850 villages and 1,000 schools by 2012. The primary support for the Village Assaini program came from UNICEF, with UNICEF acting as a trainer of public servants across the country and a supporter of non-government organizations (NGOs) to implement WASH activities alongside government agencies in villages. They also provided institutional support of the National Water and Sanitation Committee, the Action on Water Supply and Sanitation, and the Provincial Department for Health by paying salaries, equipment, and administrative costs (Kooy & Bailey 2012).

The WSP process was integrated into the existing Village Assaini program through the Pas a Pas Manual, by evaluating overlap points between the two programs (Saltori 2013). As a result, only four new activities were added to the Village Assaini program: visit the water sources; develop WSP operation plans; develop an emergency-plan; and develop an auditing plan. A challenge in implementing WSPs in the DRC is the lack of capacity for water quality analysis, especially in the rural areas. Therefore, validation and verification of WSPs are based on proxy indicators of quality, like turbidity, experience with the water supply, and local knowledge of the source. In his report on the experience of introducing WSPs to the Village Assaini program, Saltori also developed KAP surveys and community worksheets for this context (Saltori 2013). The first Village Assaini communities with integrated WSPs are currently in pilot.

### Pacific Islands

Water Safety and Security Planning has been actively pursued in the Pacific for a number of years, with development on several islands, including Fiji and Vanuatu. Management of freshwater supply in the Pacific is threatened by climate change, fragile groundwater supply, and remoteness of many islands. A lessons-learned guide on setting up national support processes and implementing the WSP process was written to share pilot experiences in Tonga, Cook Islands, Palau, and Vanuatu (Mudaliar et al. n.d.). At a national level, in Tonga a new steering committee was elected to manage WSPs, whereas in Cook Islands and Vanuatu, old committees were renamed and tasked with developing safe drinking water programs. Part of the national strategy was to then develop legislation and policies on drinking water supply and create a coordination plan for sector actors.

In Kiribati, for instance, where there is a shallow freshwater lens in porous coral soil, the importance of sanitation cannot be overstressed. However, due to the fragility of the lens, traditional ‘improved’ toilets can prove just as harmful as ‘unimproved’ ones because of infiltration (Environmental Science and Research 2015). To combat these issues, a toolkit for WASH safety planning in schools has been developed, with an aim of broadening the risk assessment and management approach to better align a multitude of connected activities (Environmental Science and Research 2015).

In Fiji, 15 communities have had WSPs implemented, with another 20 planned by the Ministry of Health in the coming year. In Vanuatu, 20 WSPs have been created with community implementation partners and the Department of Geology, Mining, and Water Resources. Further development in the Pacific has focused on the creation of monitoring and verification templates to be used by implementing partners in assessing the WSPs in communities. At this time, monitoring activities have been carried out in four communities in Fiji and have not been formalized in Vanuatu (Singleton 2014). Three of the four communities monitored were showing enough progress to advance further in their programs.

Overall, the key lesson from focus countries we investigated is that they have heavily modified WSPs by adopting them to the local context and integrating them into other existing programs. In India this included the national government allowing specific policy directives to be determined by each state based on local water quantity and quality issues. In the DRC, WSPs were integrated into a nationally supported and recognized community WASH program to avoid the creation of parallel structures. In the Pacific Islands, each country approached the process according to quality needs, some in the context of climate resilience, and according to local government structure.

## EVALUATION OUTCOMES AND IMPACTS

The need for evaluation of WSPs has been extensively noted. A significant issue observed at the 2008 Lisbon Water Safety Conference was the lack of an evidence base in support of WSPs (Williams 2008). Stemming from this conference was an increase in the call for health-based targets to be utilized as evidence in support of WSPs (Dyck et al. 2007). In Honduras, for instance, the efforts to integrate WSP methodology into national drinking-water legislation fell short when there was not enough evidence-base to support the health impact of a WSP (Hubbard et al. 2013). Davison and Deere, suggested that key public health indicators, when used in conjunction with operational targets, were a more comprehensive way of monitoring a water supply in Pacific Island countries (Davison & Deere 2007). Viera called for both global monitoring of WSP development and global benchmarking with utilities all utilizing the same set of tools and performance indicators (Vieira 2011).

The United States Centers for Disease Control and Prevention (CDC) developed a conceptual framework for WSP outcome and impact evaluation as a complement to other WSP implementation manuals (Gelting et al. 2012). The primary goal of the framework is to provide a common terminology for defining WSP outcomes and impacts and the indicators upon which they are measured to illustrate the benefits of WSP implementation and establish an evidence base for WSP effectiveness. To achieve these tasks, a logic model containing inputs (human, financial, organizational, and community resources), activities and outputs of WSP document creation, outcomes (institutional, operational, financial, and policy change and regulation), and impacts related to health, supply, and operational targets was utilized (Gelting et al. 2012).

Mudaliar argued that the CDC's conceptual framework did not provide direct evidence of WSP benefits, and further states that most WSP evaluation is focused on the stages of document development (Mudaliar 2012). He suggested the development and use of key performance indicators (KPI) to provide conclusive evidence of WSP effectiveness extended to water quality, operational performance, institutional performance, and to create separate indicators for small community-led supplies. Lockhart et al. operationalized the CDC's framework into 25 specific KPIs that can be used to evaluate a WSP across institutional, operational, financial, and policy change outcomes (Lockhart et al. 2014).

Only two studies set out to conduct an impact evaluation of WSPs (Gunnarsdottir et al. 2012a, 2012b). A total of 13 other studies presented data associated with the implementation of WSPs. Although they were not conducted as impact evaluations, they presented evaluation outcomes and lessons learned, which we have arranged by the CDC's outcome indicators to present examples of the type of information that is currently available. Of these documents, three presented institutional lessons learned, eight presented operational lessons, two presented financial lessons, and five presented policy/regulation lessons. In the next section, we present these studies organized by outcome category and summarize the limited impact evaluation results (even if the studies themselves were published prior to development of the CDC's evaluation framework).

### Institutional outcomes

In lower income countries, the value of investing limited resources in a WSP may not be seen by water suppliers where day-to-day demand already exceeds both their personnel and fiscal capacities (Rinehold et al. 2011). Summerill noted the extensive influence of organizational culture on both the attitudes of the water utility staff and on the long-term commitment of the WSP team in implementation (Summerill et al. 2010b). Furthermore, Rizak et al. found management practices, such as communicating changes in water treatment, could have a positive effect on consumer satisfaction (Rizak et al. 2003).

### Operational outcomes

Infrastructure improvements after WSP implementation, such as source repair and decontamination, were noted in several case studies (Howard et al. 2005; Mahmud et al. 2007; Gelting et al. 2012).

Gunnarsdottir and Gissurarson evaluated ten years of Icelandic use of HACCP for water supplies (Gunnarsdóttir & Gissurarson 2008). They found an improvement from 88% to 99% water quality compliance in one city and 94% to 99% compliance in a second after the implementation of HACCP. These improvements came about after an initial documentation of corrective actions and system failures for each water supply was undertaken.

One follow-up study by Gunnarsdoittir et al. found that the WSP implementation process improved utility culture, but that lack of audits and communication with the public were still operational shortcomings of the process (Gunnarsdottir et al. 2012a). Another follow-up study by Gunnarsdoittir et al. that tracked water suppliers before and after WSP implementation, found detection of E. coli decreased from 1.28% to 0.32% after WSP implementation (Gunnarsdottir et al. 2012b).

Smeets et al. evaluated the use of quantitative microbial risk assessment (QMRA) in monitoring the safety of a water supply managed by a WSP; they utilized treatment data, estimated the number of failures, and tested treatment efficacy to determine the probability of detecting a failure event (Smeets et al. 2010). Factors that complicated this assessment included the occurrence of ‘special events’ in 10% of datasets, such as when human error led to high chemical concentrations. A process-monitoring log was suggested for water suppliers to test water quality and an example of setting critical limits based on the QMRA model was developed.

A second study of microbial risk in a water supply managed by a WSP was conducted by de Souza et al. on a municipal water supply in South Africa, where the Department of Water Affairs introduced an incentive-based regulatory program and associated regulatory drinking water quality information system, to water suppliers in the country (de Souza et al. 2011). The electronic Water Quality Management System (eWQMS) platform allows municipalities to load drinking water quality data to a database and track the performance of key management functions and was integrated with additional information critical to successful operation of a supply via a WSP approach. While the WSP tool implemented in eWQMS is available to utilities in the system, it was found only 20% of systems assessed had a WSP in place.

### Financial outcomes

In Uganda, there was a 30% reduction in costs of water control activities when monitoring was done at intermittent points along the supply chain instead of only verifying the final product. This was attributed to earlier identification of problems in the supply (Howard et al. 2005). In Palau, Hasan and Gerber found that for every $1 USD invested in a WSP there was a$6 USD return on investment when comparing the capital cost of WSP investment to the long-term savings of reduced health costs, fewer purchases of water from other sources, and leakage reduction (Hasan & Gerber 2010).

### Policy/regulation outcomes

Policy incentives alone may represent too large of a regulatory workload in countries that are already trying to enforce existing regulations, requiring a critical balance between top-down policy directives and bottom-up support of WSPs (Rinehold et al. 2011).

Several years ago, Germany adapted the WSP approach to fit into their technical guidelines to support water utilities’ internal management schemes, particularly in for those without existing Technical Risk Management plans (Mälzer et al. 2010). The WHO noted extensive plans for policy and regulatory development from 2015–2020 in eastern European, Caucasus, and central Asian countries through the Protocol on Water and Health, although definitive policy documents are still in development for many countries (WHO 2014).

An update to the study that evaluated the national framework for safe drinking water found that enforcement to regulation is particularly lacking, but mandatory requirements for WSPs have been beneficial for improving compliance to regulation (Gunnarsdottir et al. 2015).

### Summary of outcomes and impacts

Overall, the outcomes of WSP implementation are still mixed and the dataset is weak; a stronger evidence base that tracks both the outcomes and impacts of WSP implementation is needed. From the case studies that were reviewed, it was found that financial outcome evaluations have been clearest, while the evidence from operational outcome evaluations that pertain to improvement of infrastructure have been stronger than those related to the improvement of procedures at water suppliers. Currently, there is a lot of promising development in policy outcomes, although follow-through on how policies are carried out and regulated is needed. The weakest evaluations to date have been for institutional outcomes.

Because the case studies were mostly limited in scope, the degree to which the observed outcomes of the WSPs are directly attributable to implementation of the WSPs is unknown. However, of the impacts noted in the framework, most outcomes may point to water supply improvements, as seen by results already noted elsewhere in this review (Mahmud et al. 2007; Kabir & Gedam 2012; Gunnarsdottir et al. 2012b). For example, Gunnarsdottir et al. noted that mean non-compliance to Icelandic Drinking Water Regulation decreased by 80% after WSP implementation. It was further documented that there was also a decrease in non-compliance in water sample collection at both the source and in the distribution network (Gunnarsdottir et al. 2012b).

Outcomes that may preliminarily point to health improvement impacts are extremely limited. Gunnarsdottir et al. found that both mean and median rates of diarrhea were statistically significantly lower and that the 95th percentile was reduced by half when WSPs were implemented than without a WSP (Gunnarsdottir et al. 2012b). They also documented a 14% reduction in incidence of diarrhea in areas with a WSP, but they did not find any statistically significant correlation between lower incidence and a utility scoring better on WSP implementation. While in a German hospital environment, Dyck et al. found that neonatal sepsis and hospital acquired infections were reduced after implementing a WSP (Dyck et al. 2007). These limited results point to the continued need for evidence-based, documented impacts to both water supply and health after WSP implementation.

## DISCUSSION

WSPs are a comprehensive risk assessment and management approach to water delivery. The goal of this review was to collate lessons learned and evaluation results by general contexts, rural implementations, and focus country case studies. Of the 53 documents included in this review, 12 were included for general WSP lessons learned, 12 were pertinent to the rural implementation context, 11 pertained to WSPs in case study countries, and 24 contained information related to evaluation of outcomes and impacts. In general lessons learned, a majority of studies provided qualitative insight, but it is seen from the evaluations that there is a disparity in the number of studies that have actually conducted an impact assessment. This highlights the need for continued operationalization of the framework for evaluation to build the evidence base for WSP usage. From rural context applications, we have seen the need to modify the WSP framework and provide support to community managed supplies. It is apparent from lessons learned on the case study countries that stakeholders need to adapt the WSP to their local context. Overall, we found three themes in the lessons learned identified in this paper: (1) support; (2) need for evaluation of outcomes and impacts; and (3) adaptation.

Support for WSP implementation could not be overstated. The need for external WSP expertise was highlighted frequently in the steps of implementation, particularly in Step 1 when assembling the WSP team and stakeholders (Howard et al. 2005; Bartram et al. 2009; Summerill et al. 2010a). Case studies on WSP implementation demonstrated the importance of support from implementing partners external to the water supplier, such as the government, interagency committees, or NGOs (Gunnarsdóttir & Gissurarson 2008; Bartram et al. 2009; Jalba et al. 2010; Rinehold et al. 2011; Breach 2012; Kabir & Gedam 2012; Kooy & Bailey 2012). It was also noted that there is a lack of practical tools for education, surveillance, monitoring, and research to support implementation (de Souza et al. 2011; Hubbard et al. 2013). Continued support of WSPs through further publication of case studies, lessons-learned documents, and manuals/guidelines will help to drive the conversation around implementation forward.

In general, outcome and impact evaluation data demonstrating WSP value remains weak. The new CDC framework for evaluation has provided common metrics against which to assess WSP across institutional, operational, financial, and policy outcomes and the impact categories of improvements in water supply and health. Of the weak evidence base, evidence on financial outcomes of WSPs is the clearest, with evidence that a WSP provides fiscal value to water suppliers and consumers (Howard et al. 2005; Hasan & Gerber 2010). The largest body of evaluation evidence of WSPs is on operational outcomes, however, results remain mixed as evidence of procedural improvement is weaker than that in support of infrastructure improvements (Rizak et al. 2003; Mahmud et al. 2007; Gunnarsdóttir & Gissurarson 2008; Smeets et al. 2010; de Souza et al. 2011; Gelting et al. 2012). Evidence on institutional outcomes are varied, with stronger evidence in relation to organizational culture of the WSP team, but weaker evidence in relation to long-term activities of the team (Zimmer & Hinkfuss 2007; Bartram et al. 2009; Summerill et al. 2010a; Rinehold et al. 2011; Hubbard et al. 2013). To date policy outcomes have a lesser evidence base despite clear momentum, which could simply be due to the fact that policies and regulations have not yet been documented on a global scale. Lastly, there is minimal evidence from impact evaluations of WSPs, with indications that there may be more documented data in relation to water supply improvements, but only preliminary data on health improvements. At this time, the rigorousness of studies does not yet support drawing clear links from outcomes of WSPs to attributable impacts.

WSP adaptation has been highlighted particularly in relation to small community-managed supplies, like those found predominantly in rural areas (Espinoza et al. 2009; WHO 2012; UNICEF & Global Water Partnership 2014). Adaptation to the local context provides implementers with language and monitoring techniques common to their experience (Espinoza et al. 2009; Hasan et al. 2011). WSP modification, as seen in India with the implementation of VWSS and WSePs, leads to plans that are more aligned to the existing requirements of local government agencies in regards to water delivery (Water and Sanitation Program 2010; Kabir & Gedam 2012). Furthermore, adaptation of WSPs to integrate with existing programs, such as the incorporation with Village Assaini in the DRC, can lead to a more streamlined implementation and less overhead work by stakeholders (Kooy & Bailey 2012; Saltori 2013).

The limitations of this study included: securing documents that pertain to case study WSP implementations that were not published; limiting the inclusion language to English, French, and Spanish; and securing documents that pertained to work in the case-study countries of interest to the UNICEF program.

Further research is needed on the evaluation of outcomes and impacts of WSP implementations and on WSP implementations in rural communities. There is active, ongoing work to address both of these topics. The UNICEF work that motivated this review will attempt to provide more insight into documenting rural WSP implementations by assessing pilot programs over the next two years. A recent call for proposals to conduct evaluations in 10 countries by WHO highlights their interest in pursuing WSP evaluation metrics and in building an evidence base for WSP usage (Aquaya 2015).

## CONCLUSIONS

WSPs have shown promise not only as a risk mitigation tool, but also as a cost effective venture for water suppliers, despite challenges in on-the-ground implementation. To ensure WSPs reach their potential for improving water delivery and management, this review found: (1) support should be provided to implementers; (2) outcomes and impacts of urban, peri-urban, and rural WSP implementations should be evaluated; and (3) local adaptation of WSPs encouraged.

## ACKNOWLEDGEMENTS

The authors would like to thank Rick Gelting for his review and comments on this paper. Further comments and suggestions from the WSP Reference Group were appreciated on early drafts of this document. The authors would also like to extend a thank you to UNICEF for funding support to this work.

## REFERENCES

REFERENCES
Author unknown
2015a
Chandrapur District Water Safety Plan, February 7 2015. Available from corresponding author
.
Author unknown
2015b
No Golden Solution–ODF is easy but sustainability is hard in Kiribati's challenging environment. Case Study. Draft. Available from corresponding author
.
Aquaya
2015
Presentation on Planned WSP Evaluation for WHO
.
UNC Water and Health: Where Science Meets Policy
,
Chapel Hill, NC
,
USA
. ).
Barrington
D.
Fuller
K.
McMillan
A.
2013
.
J. WASH Dev.
3
(
3
),
392
401
.
Bartram
J.
Corrales
L.
Davison
A.
Deere
D.
Drury
D.
Gordon
B.
Howard
G.
Rinehold
A.
Stevens
M.
2009
Water Safety Plan Manual: Step-by-Step Risk Management for Drinking-Water Suppliers
.
WHO
,
Geneva
,
Switzerland
. ).
Breach
B.
(ed.)
2012
Drinking Water Quality Management from Catchment to Consumer: A Practical Guide for Utilities based on Water Safety Plans
.
IWA Publishing
,
London
,
UK
.
Davison
A.
Deere
D.
2007
Water Safety Plan Workbook for Drinking-water: Materials for Training of Trainer
.
Water Futures for World Health Organization Western Pacific Regional Office
,
Sydney
,
Australia
. ).
de Souza
P. F.
Burgess
J. E.
Swart
M.
Naidoo
V.
Blanckenberg
A.
2011
.
WST: Water Supply
11
(
5
),
568
577
.
Dyck
A.
Exner
M.
Kramer
A.
2007
Experimental based experiences with the introduction of a water safety plan for a multi-located university clinic and its efficacy according to WHO recommendations
.
BMC Public Health
7
,
34
47
.
Environmental Science and Research
2015
Water, Sanitation and Hygiene (WASH) Safety Planning Technical Toolkit for Kiribati Schools: Framework. Draft. Available from corresponding author
.
Espinoza
A. M.
Rodriguez
S.
Figueroa
D. S.
2009
Guia para la implementacion de Planes de Seguridad de Agua en el Sector Rural de Honduras: Metodologia basada en la gestion de riesgos (Guide for the Implementation of Water Safety Plans in the Rural Sector of Honduras: methodology based on risk management)
.
RAS-HON
,
Honduras
. ).
Federal Ministry of Water Irrigation and Energy of Ethiopia
2015
Climate Resilient Water Safety Plan Implementation: Guidelines for Community-Managed Rural Drinking Water Supplies
.
FMWIE
,
.
Available from corresponding author
.
Gelting
R. J.
Delea
K.
Medlin
E.
2012
.
J. WASH Dev.
2
(
2
),
103
111
.
Greaves
F.
Simmons
C.
2011
Water Safety Plans for communities: Guidance for adoption of Water Safety Plans at Community Level
.
Tearfund
,
Teddington, UK
. .
Gunnarsdottir
M. J.
Gissurarson
L. R.
2008
.
J. Water Health
6
(
3
),
377
.
Gunnarsdottir
M. J.
S. M.
Bartram
J.
2012a
.
Water Sci. Technol.
65
(
2
),
277
288
.
Gunnarsdottir
M. J.
S. M.
Elliott
M.
Sigmundsdottir
G.
Bartram
J.
2012b
.
Environ. Sci. Technol.
46
(
14
),
7782
7789
.
Gunnarsdottir
M. J.
S. M.
Bartram
J.
2015
.
Int. J. Hyg. Environ. Health
218
(
2
),
196
202
.
Hasan
T. J.
Gerber
F.
2010
Economics of Drinking Water Safety Planning: An Advocacy Tool
,
Secretariat S.
,
Suva, Fiji
, (
accessed 15 January 2016
)
Hasan
T. J.
Hicking
A.
David
J.
2011
.
WST: Water Supply
11
(
3
),
309
317
.
Howard
G.
Godfrey
S.
Tibatemwa
S.
Niwagaba
C.
2005
.
Urban Water
2
(
3
),
161
170
.
Hubbard
B.
Gelting
R. J.
del Carmen Portillo
M.
Williams
T.
Torres
R.
2013
.
J. WASH Dev.
3
(
4
),
541
548
.
International Water Association
2004
The Bonn Charter for Safe Drinking Water
.
IWA
. ).
Jalba
D. I.
Cromar
N. J.
Pollard
S. J. T.
Charrois
J. W.
R.
Hrudey
S. E.
2010
.
Environ. Int.
36
,
51
59
.
Kabir
Y.
Gedam
P.
2012
Village Water Safety Security & Environmental Sanitation: Biovillage Project in Maharashtra
.
UNICEF
,
Maharashtra
,
India
.
Available from corresponding author
.
Keirle
R.
Hayes
C.
2007
.
Water Environ J.
21
(
3
),
208
216
.
Kooy
M.
Bailey
S.
2012
Tearfund WASH service delivery in the Democratic Republic of Congo: Contributions to Peace-Building and State-Building
.
ODI
,
London, UK
. ).
Lockhart
G.
Oswald
W. E.
Hubbard
B.
Medlin
E.
Gelting
R. J.
2014
.
J. WASH Dev.
4
(
1
),
171
181
.
Mahmud
S. G.
Shamsuddin
S. A. J.
Ahmed
M. F.
Davison
A.
Deere
D.
Howard
G.
2007
.
J. Water Health
5
(
4
),
585
597
.
Mälzer
H. J.
Staben
N.
Hein
A.
Merkel
W.
2010
.
Water Sci. Technol.
61
(
5
),
1307
1315
.
Mudaliar
M. M.
2012
.
WST: Water Supply
12
(
1
),
109
116
.
Mudaliar
M. M.
Bergin
C.
MacLeod
K.
(n.d.)
Drinking Water Safety Planning: A practical Guide for Pacific Island Communities
.
WHO and Pacific Islands Applied Geoscience Commission
,
Suva, Fiji
. ).
Nath
K. J.
2013
Development of guideline for implementation of water safety plan for the rural water supply systems in India
.
IPHE Journal
2012–13
(
4
),
33
42
.
National Environmental Engineering Research Institute
2013
Water Security Plan for select villages in Rajnandgaon district, Chhattisgarh
.
NEERI
,
Nagpur
.
Available from corresponding author
.
Ncube
M.
Pawandiwa
M. N.
2013
.
J. WASH Dev.
3
(
4
),
557
563
.
Perrier
E.
Kot
M.
Castleden
H.
Gagnon
G.
2014
.
Water Policy
16
(
6
),
1140
1154
.
Rickert
B.
Schmoll
O.
Rinehold
A.
Barrenberg
E.
2014
Water Safety Plan: A Field Guide to Improving Drinking-Water Safety in Small Communities
.
WHO
,
Geneva
,
Switzerland
. ).
Rinehold
A.
Corrales
L.
Medlin
E.
Gelting
R. J.
2011
.
WST: Water Supply
11
(
3
),
297
308
.
Rizak
S.
Cunliffe
D.
Sinclair
M.
Vulcano
R.
Howard
J.
Hrudey
S.
Callan
P.
2003
Drinking water quality management: a holistic approach
.
Water Sci. Technol.
47
(
9
),
31
36
.
Saltori
R.
2013
Water Safety Plans for ‘Village Assainis’
.
Final Report, Bangkok
.
Available from corresponding author
.
Singleton
R.
2014
Monitoring Report–Water Safety Planning in Rural Fiji. Available from corresponding author
.
Smeets
P. W. M. H.
Rietveld
L. C.
van Dijk
J. C.
Medema
G. J.
2010
.
Water Sci. Technol.
61
(
6
),
1561
1568
.
Summerill
C.
Smith
J.
Webster
J.
Pollard
S.
2010a
.
J. Water Health
8
(
2
),
387
398
.
Summerill
C.
Pollard
S. J. T.
Smith
J. A.
2010b
.
Sci. Tot. Environ.
408
(
20
),
4319
4327
.
Timilsina
B.
2012
Water Safety Plans in Community Managed Water Supplies in Nigeria
.
IWA Water Safety Conference
,
Kampala, Uganda
. ).
UNICEF & Global Water Partnership
2014
WASH Climate Resilient Development: Local Participatory Water Supply and Climate Change Risk Assessment –Modified Water Safety Plans
.
UNICEF
,
New York City, NY
,
USA
. ).
Vieira
J. M. P.
2011
.
J. Water Health
9
(
1
),
107
116
.
Water and Sanitation Program
2010
Water Safety Plans for Rural Water Supply in India: Policy Issues and Institutional Arrangements
,
World Bank
,
New Delhi, India
. ).
Williams
T.
ed.
2008
Water safety in 2010: a Brazilian perspective
.
Drinking Water Safety International: Newsletter
.
IWA Publishing
,
London
,
UK
, pp.
1
2
.
World Health Organization
2011
Water safety plans
. In:
Guidelines for Drinking Water Quality
, 4th edn.
WHO
,
Geneva
,
Switzerland
. ).
World Health Organization
2012
Water Safety Planning for Small Community Water Supplies: Step-by-Step Risk Management Guidance for Drinking-Water Supplies in Small Communities
.
WHO
,
Geneva
,
Switzerland
. .
World Health Organization
2014
Water Safety Plans in Eastern Europe, the Caucasus and Central Asia: Summary of a workshop on building capacities for the development of water safety plans 24–25 June 2014, Bishkek, Kyrgyzstan
,
WHO
,
Copenhagen
,
Denmark
. (
accessed 15 January 2016
).
World Health Organization, International Water Association
2015
A Practical Guide to Auditing Water Safety Plans
.
WHO and IWA
,
Thailand
. ).
Zimmer
H.
Hinkfuss
S.
2007
Global Water Supplier Survey: Synthesis of Main Trends
.
IWA, London
,
UK
.
Available from corresponding author
.