Drinking water quality management progress in Ontario, two decades after Walkerton

Over the past two decades, the Canadian province of Ontario has developed and implemented significant regulatory reforms in its municipal drinking water sector. A key reform has been an attempt to shift the traditional emphasis on end-product testing and compliance of numerical drinking water quality limits towards a more comprehensive and preventive risk-based framework that places greater focus on how safe drinking water is produced and delivered. This quality management approach was developed in parallel with similar efforts elsewhere, most notably the Water Safety Plan (WSP) framework advocated by the World Health Organization (WHO) since 2004 and endorsed by the International Water Association under the Bonn Charter (IWA 2004; WHO 2004, 2023). The objectives of this work were to (i) understand how Ontario's Drinking Water Management Standard (DWQMS) aligns with the WHO's WSP framework and (ii) review the readily available data for evidence that the Ontario framework has improved drinking water safety and promoted a more preventive approach through risk-based quality management.

The impetus for the significant regulatory shift was the tragic waterborne outbreak (May, 2000) of pathogenic bacteria E. coli O157:H7 and Campylobacter jejuni in the small community of Walkerton, Ontario (O'Connor 2002a). The direct economic cost of the outbreak was estimated to be $65 million (Livernois 2002). Importantly, a substantial loss of public confidence in this essential service also occurred (Perkel 2016). The subsequent public inquiry, presided over by Justice Dennis O'Connor, was conducted in two parts, the first concerned with the events that led to the outbreak itself and the second with the systemic issues faced by municipal drinking water supplies across Ontario (O'Connor 2002a, 2002b).

The inquiry findings revealed fundamental and widespread problems throughout Ontario at multiple levels, from the competence of local operators to the regulatory oversight structure (O'Connor 2002a, 2002b; Hrudey & Hrudey 2014). Ninety-three recommendations were made to address source protection, standards, treatment, distribution, monitoring, laboratories, the role of municipalities, quality management, operator training, provincial government oversight, small systems, and First Nations communities (O'Connor 2002b). In response to the Inquiry recommendations, Ontario eventually passed 3 statutes, 12 regulations, and a number of guidelines and policy tools (Tovilla & Webb 2017).

Justice O'Connor specifically recommended the adoption of a quality management system (QMS) as a framework for elevating organizational management and operating practices to the ‘…highest standards possible’ (O'Connor 2002b, p. 335). O'Connor proposed expanding such a framework to include the people responsible for managing and operating the system and noted the benefits to include:

• The adoption of best practices and continuous improvement;

• Application of ‘real-time’ process control wherever feasible;

• The effective operation of robust multiple barriers to protect public health;

• Preventive rather than strictly reactive strategies to manage risks to public health;

• Effective leadership

Following the recommendations, quality management obligations were incorporated among the licensing requirements of municipal drinking water system owners, under section 44 of the SDWA, in the form of accredited operating authorities and approved operational plans (Abouchar & Vince 2012). Municipal drinking water system owners/operators were required to obtain accreditation by developing an organizational QMS conforming to a prescribed standard, as verified by third-party independent audit. Operational plans were to document the QMS implemented conformed to the standard's requirements while also reflecting the drinking water system's operations and maintenance activities and would be subject to review and approval by the province. In this context, conformity refers to the fulfillment of the requirements of a prescribed standard and compliance refers to the fulfillment of regulatory requirements (MOE 2007). The DWQMS was developed, and on 4 May 2007, the Licensing of Municipal Drinking-Water Systems regulation (O. Reg. 188/07) came into force under the SDWA, requiring owners of municipal drinking water systems to apply for a licence by a specified date and upon completion of the requirements including those described above. Implementation was phased based on system size over an 18-month period spanning 2009–2010, with submissions from the 12 largest systems due by 1 January 2009. The first licences were issued on 24 February 2011 (Abouchar & Vince 2012).

The DWQMS describes a drinking water-specific framework drawing on the generic ISO 9001 quality management systems, the Hazard Analysis and Critical Control Point (HACCP) approach used by the food industry, and some aspects of the ISO 14001 standard for environmental management systems (Tovilla 2020). The DWQMS includes 21 elements organized around a plan-do-check-improve cycle (MOE 2017) and is listed in Table S1 in the Supplementary Materials. The Ontario framework was strongly influenced by the Australian Framework for Management of Drinking Water Quality (McRae et al. 2001; NHMRC 2001; O'Connor 2002b). Furthermore, the WHO's WSP framework, first published in 2004, itself was directly based on the one first developed in Australia (Bursill 2022). Given their common roots, not surprisingly, the DWQMS and WHO's WSP framework bear a strong resemblance.

In Canada, the regulation of municipal drinking water systems falls within the jurisdiction of the provinces and territories, and most of these rely heavily on more reactive regulatory approaches (Hrudey 2011). Ontario's DWQMS suggests a more proactive approach in a regulated context. As the use of WSPs and other industry-specific methods of quality management advance in the drinking water sector internationally (Tovilla 2020), it would be useful to better understand if Ontario's framework has demonstrated a meaningful benefit in terms of municipal drinking water safety and public health protection and if it represents an aspirational model for other jurisdictions considering or engaged in similar reforms.

Risk-based quality management of drinking water systems, in some form, is now common in multiple jurisdictions internationally, including Iceland (Gunnarsdóttir et al. 2012), the UK (Hasan et al. 2020), Australia (NHMRC 2011), and New Zealand (Taumata Arowai n.d.). Most recently, the revised European Drinking Water Directive (2020/2184) requires all EU member states to implement a risk-based quality management framework in national drinking water regulations consistent with the WHO WSP framework and the standard EN 15975-2 (Dettori et al. 2022).

While there is generally a consensus that risk management is integral to achieving safe drinking water through continuous operations and management improvements, it is less clear how a WSP-type framework should be regulated, reviewed, and enforced. In New Zealand, the preparation of WSPs has been required for all drinking water supplies since being legislated by the Health Act in 2007. However, the 2016 Havelock North Campylobacter outbreak tragically revealed the challenges involved with such a requirement in practice. The subsequent Havelock North Inquiry examined the failings of the WSP framework to prevent the outbreak. Contributing factors identified included resource and competency limitations in preparing and maintaining WSPs, the over-reliance on external consultants for WSP preparation and implementation, an uncritical and compliance-focused view towards their preparation, a lack of understanding of their purpose and need for acting on their findings, a lack of appreciation by top management of priority risks, and the need to allocate resources accordingly, the failure to incorporate risk mitigation measures into daily operations and long-term planning activities, and a lack of oversight resources to adequately audit and review WSPs and meaningfully address deficiencies (Government Inquiry into Havelock North Drinking Water 2017). The Havelock North outbreak served as a reminder that even progressive government policies should be periodically reviewed and evaluated to assess their effectiveness in practice and revise them as needed. As such, this research attempts to review some of the relevant aspects of the regime currently in place in Ontario in the hopes of gleaning insights into its improvement as well as guidance for possible implementation elsewhere.

Ontario's DWQMS has not been examined in the open literature since its implementation first began in 2007. There is limited information available regarding the extent of DWQMS conformity across municipal water systems or whether the tangible benefits to drinking water safety in general have been realized. Furthermore, understanding the alignment between the DWQMS and WSP frameworks may provide lessons for implementation in other jurisdictions. To address knowledge gaps, we have attempted to evaluate policy, public data and system performance information available from Ontario's Ministry of the Environment, Conservation and Parks (MECP).

This paper consists of two principal analyses: (1) a comparison of core elements in the WSP and Ontario DWQMS based on available documentation, and (2) an examination of publicly available data from the Ontario MECP and municipal system owners. The procedures for each of these analyses are outlined below. Analyses were verified through discussions with several Ontario water utility representatives according to an exempt Ethics Review process.

We compared the recently published second edition of the WSP Manual (WHO 2023), with two documents published by Ontario's Ministry of the Environment: (i) the second edition of the MOE (2017), and (ii) the ‘Implementing Quality Management: A Guide for Ontario's Drinking Water Systems’ (referred to in this documentation as the Implementation Guide) (Ontario Ministry of the Environment 2007) which was published to ‘assist owners and operating authorities in developing, implementing, and maintaining a quality management system.’ All documents were reviewed first in isolation to identify the steps, and key actions, outlined for each risk management system. These ‘elements’ are referred to as modules and key actions in the WSP manual and elements in Ontario's DWQMS. Ontario's DWQMS has 21 elements which are outlined in the Supplemental Materials Table S1. The WSP has 10 steps with a total of 23 key actions which are outlined in the Supplemental Materials Table S2.

Our comparison focuses on how closely the 21 DWQMS elements cover the 10 modules of the WSP. We selected the WSP as a comparator because it is a widely implemented and studied risk management framework (WHO 2017). We identified the DWQMS elements that individually or combined cover the key actions outlined in each module in the WSP. We deductively coded the extent to which each Ontario document covered the WSP key actions using the following codes: evidence of inclusion, partial evidence of inclusion, and absent. Table 1 defines and demonstrates how the evidence of inclusion and partial evidence of inclusion were used. Absent was used when the WSP key action was completely absent from the Ontario document or could not be verified conclusively. We also included elements in the DWQMS that go beyond any modules in the WSP evidence of inclusion.

There is an abundance of publicly available data and information for Ontario's municipal drinking water systems. The data sets, materials, and resources used in this work are detailed below.

Ontario Ministry of Environment, Conservation and Parks (MECP) maintains an open-access repository of a range of water quality data and system performance information available to the public for the years 2014–2022 (Ontario Open Data Catalogue 2023). Although annual reporting and public communication were required by each municipal water system, the source data catalogue was not initiated until 2014. It is likely that there were significant improvements in drinking water quality and operational performance between 2007 and 2014; however, these were not discernable from the available data set. Data sets used in this work include:

• the number and type of registered systems

• drinking water quality test results

• adverse water quality incidents (AWQIs)

• enforcement activities related to inspections

• provincial orders and convictions

For the purpose of this work, only large municipal residential systems (LMRS) regulated under O. Reg 170/3 (Government of Ontario 2022a, 2022b) were evaluated (LMRS serve > 100 people). Data were retrieved from this catalogue and compiled into comma-separated values (CSV) files in Excel for data analysis in the R software coding language (Version 4.3.0/21 April 2023).

LMRS are required to immediately report AWQIs to the Ministry and the local Public Health unit under the SDWA, Section 18(1) and Schedule 16. The statute details categories including key water quality parameters such as a maximum acceptable concentration (MAC) exceedance, presence of Total Coliform Bacteria, E. coli bacteria, chlorine residual deviations, filter effluent turbidity, improperly disinfected water, etc. We examined the AWQI reports from 2014 to 2022 to investigate incident details.

Inspection and water quality ratings generated by annual MECP inspection reports were extracted and used to examine trends and rates of compliance for microbiological and chemical water quality parameters. A total of 646 LMRS were evaluated; systems not continuously registered between 2014 and 2022 were excluded. A list of questions included in the annual inspection process is available in the Supplemental Material (Table S3). It is important to note that microbiological and chemical test results were only reported as individual categories in the data post-2019. See Supplemental Materials for additional sentiment analysis.

The SDWA, 2002, under O. Reg 170/03, Schedule 22 requires all registered municipal systems to issue three annual reports: Annual Water Quality Reports, Summary Reports, and DWQMS Management Review Reports (issued to Council). Samples of these documents were reviewed to ground-truth the metadata in Ontario's Data Catalogue and provide context for AWQIs, inspector ratings, water quality ratings, and DWQMS conformity.

The Chief Drinking Water Inspector Annual Report summarizes the performance of all registered systems in the province, including initiatives, incidents, deficiencies, and convictions. The annual reports were reviewed to provide confirmation of the data obtained from the Open Catalogue and provide details on compliance with regulatory requirements and incidents in LRMS.

The SDWA, Part IV, Section 21 requires all drinking water systems to develop a QMS that conforms to the DWQMS as verified by an external certified auditor. Currently, MECP has certified two auditing firms, NSF International Strategic Registration (NSF-ISR) and Intertek SAI Global. The accreditation protocol developed by MECP requires a third-party desktop audit each year with an extensive on-site audit every 36 months. Auditors note areas of nonconformity and oversee a corrective action process before issuing accreditation. The authors were unable to find a repository for audit reports; however, some municipalities published audit reports on their websites. When available, select audit reports were evaluated for this study to understand the issuance of minor and major non-conformities, as well as ‘opportunities for improvement’. This work focused on mechanisms of continuous improvement, not individual systems. System names and reports were anonymized.

There are several areas where the DWQMS goes beyond the tasks outlined in the WSP: Elements 10, 11, and 13 (competencies, personnel coverage, and essential supplies and services) detail approaches to ensure that utilities are consistently staffed with adequately trained personnel. The ‘essential supplies and services’ element outlines ways to ensure high-quality supply of needed resources and materials even in the case of staff turnover or supply chain issues. Element 20, management review ensures that utilities reflect on the implementation of the DWQMS to ensure continuous incremental improvement in the system performance and water safety.

The WSP manual is promoted globally for adaptation in a range of contexts. There is less specificity in the WSP framework regarding the timing of review activities and follow-up actions since resources for such may vary widely and need to align with local regulations. The enforceability of the DWQMS as a part of regulatory compliance may fill gaps in the WSP framework. For example, while Module 2, Key action 1 in the WSP (‘Confirm the accuracy of the system description, using desktop reviews, interviews and field visits’) has partial evidence in both the DWQMS documents, discrepancies in the accuracy of the system description are checked during annual regulatory inspections in accordance with MECP procedures (Table S3). While a text comparison of Ontario's DWQMS documents with the WSP manual provides insight into the alignment of these approaches, water safety is most impacted by the extent to which documented practices are implemented at utilities. The annual management review and third-party audits required for operating authority accreditation provide a mechanism to promote continuous improvement.

Ontario's DWQMS aligns with the majority of the 23 key actions of the 10 step WSP framework. Ontario's framework is one of the first examples in North America of the integration of risk management into enforceable drinking water regulations. Some jurisdictions in Ontario have extended the QMS approach to other areas of public works, including wastewater services (Toronto Water 2022). Continued evaluation of Ontario's DWQMS and WSP implementation globally may be used to inform improvements to implementation, assessment, and future refinement of the Standard.

Key DWQMS elements requiring improvement are primarily related to communication. Where the DWQMS only partially aligned with the WSP framework relates to the forming of a team and verifying descriptions of risks in a system. The drinking water industry has a high turnover rate, especially in utility and government positions (Dickerson & Butler 2018); one of the noted concerns with drinking water management systems in the past has been the loss of institutional knowledge in a system when subject-matter expert staff leave the system (Dickerson & Butler 2018). The WSP is designed to be a ‘living document’ that is updated and improved over time (WHO 2023); ideally one person does not possess all the important knowledge about the system and knowledge is transferred amongst team members. While the DWQMS provides evidence of strong supporting programmes and management review, an improvement could be made in framework documentation of the assembly of a diverse team to ensure the continuity of the DWQMS.

Figure 2 may be interpreted as a possible improvement of water quality due to a decline in incidents; however, the data available do not allow for this level of evaluation. Correlation between AWQIs and QMS reviews or auditing outcomes is limited, so no significant insight regarding the role of the DWQMS in system improvements can be determined. However, some operating authorities reviewed by the authors do include AWQIs with corrective actions and outcomes in their QMS Management Review Reports (District of Muskoka 2022). If this practice were commonplace and included with AWQI data sets, the relationship between compliance and conformity could be better assessed. Available MECP data exclusively characterize regulatory compliance and not conformity with the DWQMS as verified by audit.

System A serves approximately 80,000 people and has multiple years of documented adverse secondary disinfection events in the distribution system. In 2018, the system reported over 90 AWQIs of low combined chlorine residual. System A published Annual Water Quality Reports, and the most recent Annual Inspection, DWQMS management review summary reports, and DWQMS Audit reports on the government website. These documents revealed System A is working with engineers and industry experts to understand and address the persistent secondary disinfection issues and take corrective actions (i.e., system flushing) to re-establish combined chlorine residual. The 2019 DWQMS audit found major nonconformity issues, with 14 ‘opportunities for improvement’ across 8 of the 21 DWQMS Elements and one major nonconformity report. The auditor found the operating authority was not implementing an effective preventive action process for continual improvement. This was further reflected in the review of Annual DWQMS management review summary reports which did not clearly map how improvements needed in 1 year were consequentially addressed in the following year. Despite these noted operational challenges, Inspector and water quality compliance ratings ranged from 94.9 to 100%, respectively. While the rating scores are an accessible high-level metric that is reported publicly, they may not be a valuable score to share with the public to reflect the state of drinking water systems.

System B serves approximately 12,000 people and experienced multiple years of boil water advisory issuance, with additional trends related to watermain breaks and pressure loss (that cannot be conclusively correlated to boil water advisories). System B publishes a DWQMS annual plan on the government website, but all details are not included on the website. Annual water quality and water treatment plant summary reports are available on the government website and detail the multiple instances of watermain breaks and pressure events but provide no additional context of the cause or preventive measures. The annual water treatment plant summary report notes that the 2022 DWMQS audit completed by SAI Global found all 21 elements to be in conformity with the standard, but the audit report itself is not available. A review of public-facing news articles related to this system found that the watermain breaks are a product of ageing infrastructure and the municipality is working on obtaining funding to address this serious issue, and notes that the system is near failure. Despite these noted infrastructure challenges, Inspector and water quality compliance ratings ranged from 94.1 to 100%, respectively. While Systems A and B presented here are exceptional cases in Ontario, they highlight how there may be value to further evaluation of Inspector ratings and water quality compliance ratings as meaningful metrics.

Table 2 shows the mean, median, maximum, and minimum for all LMRS to allow for the comparison of ratings. Note that microbiological and chemical compliance ratings (water quality ratings) were only available post-2019. Systems are required to complete differing numbers of microbiological tests based on population size served (Government of Ontario 2022a, 2022b). Annual inspection ratings are generated based on multi-day site visits, interviews, and document reviews and rely on risk-based assessments to generate the percentage score.

A 99.9% compliance rating equates to 1 adverse event out of the 1,000 sampling events. However, a 99.0% compliance rating equates to 10 adverse events out of the 1,000 sampling events. While a 99.0% compliance rating seems positive most water treatment operators, managers, and stakeholders would agree that 10 adverse water quality events in a year is a serious cause for concern and may be indicative of a larger water quality issue that needs to be addressed. This result shows that, while water quality compliance ratings are an important component of the SDWA and supporting regulations, using this mathematical representation of water quality outcomes obscures the 10 adverse incidents and could lead to a false sense of security in the system performance. Compliance ratings as used currently provide little real value to decision-making, particularly to the prioritization of systems at the greatest risk of repeated adverse water quality events, a step which is critical in a preventive management cycle.

A high volume of data is being collected as shown by the variety of reports available to the public, which provides a significant level of transparency – a key recommendation of the Walkerton Inquiry (O'Connor 2002b). However, an over reliance on reporting simple aggregated percentages does not help differentiate between systems with a high risk or low risk of failure. Further, end-of-pipe water quality data do not explicitly illuminate operational or system improvements, which is a key component of both the DWQMS and WSP. Details regarding improvements are mentioned in some Annual Reports, DWQMS Management Reviews and Inspection details made public by the operating authorities, but not in all examples reviewed for this work. However, Ontario's recent Asset Management Planning regulation (O. Reg. 588/17) now provides a link for capital improvements with the Municipal Drinking Water Licensing program's requirement for sustainable Financial Plans and may help strengthen the reporting on investments and drinking water quality (OWWA 2021). Both WSPs and DWQMS include the review of incidents and near misses as a central component to continuous improvement and preventive management. Some evidence of review and improvement can be found in Annual DWQMS Reports; however, report quality and depth vary across the small sample reviewed. To demonstrate the value and benefit of the DWQMS more fully in providing safe drinking water, Ontario can extend its open-data repository to include documentation and data sets characterizing key information on DWQMS conformity provided in DWQMS Management Reports. This would provide information on the relationship between regulatory compliance and DWQMS conformity and conditions for safe drinking water could be evaluated beyond endpoint, regulatory monitoring.

The analysis of MECP open data sets and review of annual reports, in combination with ground truthing through discussions with water utility representatives produced the following recommendations to improve the current DWQMS framework:

• Ensure continuity of DWQMS through the formation of QMS teams – to ensure that institutional knowledge and oversight remain over the long-term despite staff turnover and retirement.

• Develop technical review processes for verification of identified hazardous events – to ensure hazardous and associated critical control points are identified appropriately. Do the control measures provide adequate protection of water safety? Possible processes could include Ministry technical review, utility peer review, engineering consultants.

• Standardize reporting of DWQMS performance and audit findings – to facilitate review and trend analysis of conformity across Ontario municipal water systems.

• Develop key metrics to evaluate whether continual improvement is occurring – to ensure teams are regularly reviewing incidents, conducting root cause analysis, and implementing corrective and preventive actions. Key metrics are needed to identify evidence that routine team reviews are effectively learning from treatment upsets, equipment failures, critical control limit deviations, unexpected operational events, adverse conditions, etc. with corrective actions where necessary to improve prevention of similar issues in the future.

• Share ideas and best practices for DWQMS – to share experiences, new ideas, success stories, challenges, and best practices amongst municipal water systems. Use close calls as opportunities for training and improvement. Many water utilities report that their DWQMS is developed and operated in isolation from the Ministry and other municipalities. Annual or bi-annual seminars or trade conferences could be mediated by the province, or sessions could be developed within existing conferences to enhance cross-utility dialogue, training, and learning.

A core tenet of any preventive management approach is the value of reflection and learning from past experiences. In this spirit, we reviewed historical data and system performance information to understand how Ontario's legislative response to the Walkerton tragedy has impacted municipal drinking water systems.

Ontario's framework is the first and, to the authors' knowledge and understanding, the most robust example of a risk-based QMS implemented into drinking water regulations in North America. Direct evidence for the effectiveness of the framework remains unclear based upon analysis of publicly available data. Potential benefits of the QMS requirements are suggested based on the content of some of annual management review reports. However, heterogeneity in management review report depth and quality, limited availability of corresponding external audit reports, and limited information on nonconformity details restricted our ability to characterize the performance of the framework. Continual evaluation of Ontario's accreditation process and improved tracking of key DWQMS conformity trends, similar to what is done with regulatory compliance data, would identify opportunities for improvement while providing guidance for other jurisdictions considering a risk-based management approach to drinking water safety.

We acknowledge the support of Ontario MECP staff for assisting in the collection and interpretation of key information.

All relevant data are available from an online repository or repositories: https://data.ontario.ca/dataset/drinking-water-quality-and-enforcement.

The authors declare there is no conflict.