Abstract

This paper builds on previous research to address the question of whether there is practical value for a made-in-Ontario municipal management system standard (MSS) for wastewater and stormwater related activities, in addition to the Drinking Water Quality Management System Standard (DWQMS) that is already statutorily required. This research specifically addressed the questions: is there value in a mandatory or voluntary MSS; are there neutral, positive, or negative effects of having an MSS; and what standard is more adequate? Through a focus group method, this research finds evidence in support of and wide recognition of the practical value an MSS in assisting municipalities in meeting their environmental objectives, addressing property damage risks, providing an additional mechanism of public accountability, and improving alignment with the legal structure. It was also apparent that there is no political appetite in the provincial government to embark on a mandated MSS, so the preferred option at this time appears to be a provincially endorsed, voluntary, sector-specific standard for wastewater and for stormwater, which could constitute a catalyst to boost voluntary uptake of MSS by small to medium municipalities (as is already occurring with large municipalities). This standard could be based on a customized variation of the ISO 14001 and DWQMS.

HIGHLIGHTS

  • In part as a response to conference papers presented by the author on this research at regional and national conferences, both the Canadian Standard Association (CSA Group) and the Standard Council of Canada (SCC) are in the process of developing a new standard for stormwater water management.

  • The CSA Group is currently seeking funding for another standard for wastewater management, endorsed by the Ontario Ministry of the Environment, Conservation and Parks (MECP), and the Water Environment Association of Ontario (WEAO).

INTRODUCTION

In a broader context, the expectation of modern society is that governments would ensure the provision of safe drinking water, the collection and treatment of wastewater, as well as the collection and treatment of stormwater runoff. All these are expected to be done in a responsible way ensuring adherence to environmental, economic, health, and safety objectives. In achieving these objectives, all utility delivery models, either public, private, or hybrid, are also expected to have mechanisms of public accountability, a sound management structure with adequate checks and balances, and a clear regulatory framework. A representation of such societal goals is well captured in the 2016 UN Sustainable Development Goals (SDG), in keeping with the global trends concerning environmental, social, and economic responsibility (ESE) (UN 2016).

Generally, governments in the developed world have an inherent level of transparency and accountability, but the private sector is less prone to such levels of scrutiny. The initial reflections of societal expectations of SDG date back to the United Nations 1972 Conference on the Human Environment in Stockholm, and the 1987 World Commission on the Environment and Development, which let to the first globally accepted standardization of ‘sustainable development’ (Tovilla 2020a). In parallel, the term ‘quality’ came onto the global standardization agenda in the 1970s when manufacturing and defence industries began exploring total quality management and total productive maintenance (Franceschini et al. 2006; Sivaram & Devadasan 2012; Heras-Saizarbitoria & Boiral 2013). The focus on quality management introduced the idea that standardizing management systems and quality manuals could enhance confidence in a product.

In the Canadian context, but with global implications, the 1991 case of R. v. Bata Industries marked the first time that the directors of a large corporation were exposed to liability for environmental pollution. R. v. Bata Industries was a legally significant and highly influential case in relation to due diligence, criminal prosecution, and directors’ liability for environmental violations (Greenbaum & Wellington 2010; Tovilla & Webb 2017). Scholars have linked the origins of ISO 14001 to the Bata case as an important example of judicial recognition of the value of Environmental Management Systems (EMS) (Greenbaum & Wellington 2010). The defence of due diligence failed and the case raised awareness among corporations, insurance companies, regulators, civil society, and NGOs of the importance of an effective EMS standard.

The importance of the R. v. Bata Industries case was noted by Tovilla & Webb (2017), in that it shone a spotlight on the value of an EMS as part of an organization's due diligence in complying with Ontario environmental legislation. In effect, voluntary standards and audits performed by third-party organizations became a useful tool to assist firms in meeting their due diligence requirements under Ontario environmental law.

In general, management system standards (MSS) address inter-organizational and inter-jurisdictional harmonization to reduce inefficiencies (Franceschini et al. 2006; Heras-Saizarbitoria & Boiral 2013). Although primarily used by profit-driven organizations, there is increasing evidence of management standards being used by all three levels of government to supplement government regulation (Tovilla 2020a) in order to demonstrate accountability and maintain higher ESE levels of responsibility. As an example, from the establishment of global standards by the International Standards Organization (ISO) on quality management in 1987 (ISO 9001) and on environmental management in 1996 (ISO 14001), now ISO has developed over 23,428 international standards (ISO 2020). (The ISO is an independent, global non-governmental organization made up of members from the national standards bodies of 162 countries and is the world's largest developer of voluntary international standards. By definition, all ISO standards are voluntary in the sense that ISO is a non-state body and therefore is incapable of requiring an organization to comply with its standards the way governments can require and enforce adherence to laws.) ISO's most widely used MSS, ISO 9001 for quality management (QMS) and ISO 14001 for environmental management (EMS), remain popular with 1,033,936 and 319,324 certificates, respectively (ISO 2017). An estimated 1% of these world-wide certifications are associated with ‘water supply’ entities and/or ‘public administration’ (ISO 2017). This supports the idea that there is an increasing uptake of non-state MSSs by state environmental regulations, noting that while state rule instruments provide minimum standards, non-state standards are developed and updated more frequently and may be more stringent than government criteria (Fulponi 2006; Delmas & Montiel 2008).

This study applies the term MSSs as normative documents that articulate acceptable versus unacceptable behavior. A review of the literature on MSSs provides insights into their original uses, strengths, and weaknesses, leading to an exponential development of new sector-specific standards for various sectors. Scholars point to an increasing uptake of specific non-state standards by state environmental regulations, noting that while state rule instruments provide minimum standards, non-state standards are developed and updated more frequently and may be more stringent than government criteria (Fulponi 2006; Delmas & Montiel 2008; Tovilla 2020b). The academic literature on MSSs has centerd on the motivations for their adoption, the value of the business and critical factors for successfully implementing them (Fulponi 2006; Searcy et al. 2012). Other literature suggests that standards are developed in response to public concerns, human tragedy, and environmental crisis or as precautionary measures (Kollman & Prakash 2002; Delmas & Montiel 2008). This intersects with the sources leading to policy innovation, e.g. new policy goals, uncertainty, stricter policies, new scientific insight, disasters, disease, tragedies, and catastrophic events (see Figure 2 in Tovilla 2020b).

In the Ontario context, the Walkerton Tragedy (2000), when an estimated 2,300 people became seriously ill and seven people died from exposure to microbially contaminated drinking water in Walkerton, Ontario, a town of approximately 5,000 people located northwest of Toronto, was a trigger for regulatory innovation for the drinking water sector (Tovilla 2020b). In a span of 8 years following this tragedy, and after an independent public inquiry (O'Connor 2002a, 2002b), the drinking water governance framework was redesigned with the new Safe Drinking Water Act (2002), the Nutrient Management Act (2002), the Clean Water Act (2006), the Drinking Water Quality Management Standard (DWQMS) (2007), and a dozen regulations and several guidelines (Tovilla & Webb 2017).

The post-Walkerton drinking water regulatory structure in Ontario resulted in a risk-based management approach for source water protection and includes a legislatively mandated QMS developed for drinking water (DWQMS). The wastewater and stormwater sectors were not similarly adjusted, remaining with basically the same conventional, minimalistic governance structure established in the 1950s (Tovilla 2020b).

The DWQMS draws on the non-state ISO 9001 standard for QMSs, as well as the Hazard Analysis and Critical Control Points (HACCP) for the food industry, and also considered elements of ISO 14001 standard for Environmental Management Systems (City of London 2008; O'Connor 2002a, 2002b). The most used MSS for human health and environmental protection for municipal water utilities include ISO 9001, ISO 14001 the HACCP (Tovilla 2020a), and the ISO 45001 for health and safety. The focus in this study involves the first three.

The HACCP is an MSS that addresses food safety through the analysis and control of biological, chemical, and physical hazards occurring throughout the food production process, from raw material production, procurement, and handling, to manufacturing, distribution, and consumption of the finished product (USFDA 2015). The standard's origins date back to 1960, as the National Aeronautics Space Administration (NASA) partnered with the Pillsbury Company (a food processing company) and the US Army Laboratories to develop safe food for upcoming space expeditions (SFA 2019). This program was designed to ensure pathogen-free food for their space program. An outbreak of botulism in 1972 from commercially canned potato soup prompted the US Food and Drug Administration to promulgate regulations drawing on many HACCP concepts, which by 1997 were adopted as mandatory by the USFDA (PSFS 2005).

There is evidence that MSS, and in particular the ISO 9001, ISO 14001, and HACCP, have been used by governments to address water quality risks and in direct response to water quality tragedies. In addition to the Walkerton tragedy (2000), the cases in Milwaukee, Wisconsin (1993), and Flint, Michigan (2014), both states now established voluntary programs referencing ISO 14001 for their wastewater systems (Naumova et al. 2003; Behm 2013; WDNR 2014; Rhoads et al. 2017; Michigan State 2018).

In the Canadian context, the growing federal–provincial–municipal government recognition of the value of drawing on non-state MSS as part of human health and environmental protection efforts is evidenced in Canadian court decisions or court hearings to impose EMS standards based on ISO 14001 on water utilities (Tovilla & Webb 2017; Tovilla 2020b).

While the post-Walkerton risk-based approach for drinking water systems seems to be accomplishing its policy goals of human health protection and drinking water quality, there is not a parallel approach in place for wastewater and stormwater utilities in Ontario. After the regulatory modernization of the Ontario drinking water sector from 2002 to 2007, it would be reasonable to ask how safe Ontario's drinking water is today. In their 2011 report, the environmental non-governmental organization called Canadian Environmental Law Association (CELA) noted that:

[b]ased on monitoring results collected by the Ontario MECP, it appears that municipally treated drinking water usually meets Ontario's drinking water quality standards. High levels of contaminants are rarely found. The 2011 annual MECP report on municipal drinking water in Ontario found that the ministry's stringent drinking water standards were met in:

  • 99.88% of drinking water tests from municipal residential systems,

  • 99.51% of drinking water tests from non-municipal year-round residential systems, such as mobile home parks, and

  • 99.49% of drinking water tests from nonresidential and seasonal residential systems serving designated facilities such as day nurseries, schools, and health centers (CELA 2011, page 2).

More recent information from the Ministry of the Environment, Conservation and Parks (MECP) Chief Drinking Water Inspector Annual Reports from 2012 to 2015 indicate that province-wide 99.8% of all drinking water test results comply with Ontario's drinking water quality standards, and there is 98–99% compliance of non-municipal systems in the periods from 2012 to 2015 (MECP 2016a, 2016b). This level of transparency of monitoring of utilities does not exist in the wastewater/stormwater utilities. While there are requirements for monitoring and reporting wastewater treatment plants' overflows, and compliance with water quality criteria prior to discharge to the environment, there are no performance-based requirements for collection system overflows and for stormwater utilities performance (MECP 2018).

In Ontario, in the broader context noted above, after 10 years of experience with the provincially required DWQMS, municipalities have generated a body of knowledge about the value of MSSs, and this knowledge is now proactively being expanded from its use in drinking water management to wastewater and stormwater (Tovilla & Webb 2017). Research for this article suggests that there is a slow but steady transfer of knowledge concerning MSS approaches from the municipal drinking water to wastewater and stormwater sectors. In part, the transfer involves a community of officials in municipal water utilities that are growing more knowledgeable about, benefiting from and drawing on MSSs to address environmental performance in their operations.

Figure 1 shows a map of the study area, which depicts the Canadian Province of Ontario and zooms in to illustrate Greater Golden Horseshoe, an area encompassing the larger urban centers (e.g. Toronto, Peel, York, Durham, Halton, Hamilton, Niagara, and Barrie).

Figure 1

Province of Ontario, Canada, and the Greater Golden Horseshow (FAO.org 2020).

Figure 1

Province of Ontario, Canada, and the Greater Golden Horseshow (FAO.org 2020).

The proposition explored in this article is that ISO QMS and EMS standards appear to be forming a conceptual ‘bridge’ between state and non-state actors (Tovilla 2020a, 2020b; Weiss 2000), as innovative forms of regulation. There is an increasing reference to such standards in legislation and related government documents and in court decisions with respect to the value and use of MSS in support of environmental protection objectives (Tovilla & Webb 2017). The conventional command and control regulatory responses to water regulation have been characterized by commentators as an often costly approach to regulation, that tends to be inefficient, with a stifling effect on innovation, enforcement difficulties, an excessive focus on ‘end-of-pipe’ solutions, and a propensity for creating ‘adversarial relationships’ (Sinclair 1997; Tovilla 2020a).

Figure 2 illustrates the typical components in scope and out of scope for a typical municipal wastewater and stormwater system. This figure was developed by the author as an adaptation from work done by Tovilla and the Region of Peel staff for implementing an EMS for Peel Region's Wastewater system (at the time the author was the manager of the Peel Region wastewater system).

Figure 2

Scope of a municipal wastewater and stormwater management standard (adaptation from work done by Tovilla and the Region of Peel staff).

Figure 2

Scope of a municipal wastewater and stormwater management standard (adaptation from work done by Tovilla and the Region of Peel staff).

In this paper, the term ‘value’ means the practical utility of municipalities applying MSS to assist in achieving societal expectations on human health and the environment. Practical utility is here considered to include: possible assistance in meeting legal requirements and thereby decreasing the likelihood of regulatory violations taking place, assistance in demonstrating due diligence when violations occur, increasing consistency in operations and assistance in demonstrating accountability, transparency, and good governance to the public and stakeholders (Tovilla 2020b).

METHODS

For this study, a non-experimental method was used, applying descriptive and qualitative research components in the form of a focus group method. A preliminary literature review for this study and a survey in the form of semi-structured interview method was the subject of other papers published by the author (Tovilla & Webb 2017; Tovilla 2020a). This paper makes reference to that literature review and the survey as appropriate.

The focus group method objective was to build on the preliminary results of the literature review (Tovilla & Webb 2017), the semi-structured interviews (Tovilla 2020b), and the case studies. The case study method is not subject to this paper and it will be addressed in a separate article. The focus group method assisted the author in corroborating and triangulating the research process, particularly the main research question. For the focus group, the author moderated the session and the PhD Supervisor acted as a participant observer. Two graduate students were recruited as recorders of conversations and to facilitate the planned breakout sessions. This multi-methods approach allowed for what Cresswell et al., referencing Hossler and Vesper, refer to as a ‘concurrent triangulation method design’ (2007: 162), indicating a triangulation of data collection, separate data analysis, and the integration of databases at the interpretation or discussion stage of the report. This study summarizes the focus group method but also coalesces with the results of two different data collection techniques: literature review and the survey in the form of semi-structured interviews. The focus group was an additional survey method to help tie findings together and obtain feedback and criticism from industry experts, which in turn helped corroborate and triangulate the process (Stemler 2001).

The focus group method was undertaken with a view to addressing the following research sub-questions:

  • (1)

    Is there value in creating a provincially mandated municipal wastewater environmental MSS and a stormwater environmental MSS?

  • (2)

    Are MSSs likely to have neutral, positive, or negative effects on the performance of Ontario municipal water management activity?

  • (3)

    What management standard is more suited to the needs of municipal wastewater and stormwater?

These questions were explored with a group of professional experts on 15 February 2018, in Toronto. A representative balance of participants was sought to ensure a broad representation of state and non-state actors in the governance structure of water systems. The focus group had 14 participants including representatives from Ontario and international experts on environmental regulations, EMS, small and large water utilities (municipal and non-municipal), and the private sector. The focus group participants comprised:

  • provincial government regulators from the MECP directly involved with all municipal water, wastewater, and stormwater systems (four participants);

  • large municipalities (1+ million residents) with EMS systems in place or in the process of being implemented (three participants);

  • medium to small municipalities, including representation of the Ontario Clean Water Agency (OCWA) (three participants);

  • international private sector regulators and registrars, with representation of the CSA Group, including the two registrars designated to conduct DWQMS external audits for all municipal water systems in Ontario (three participants); and

  • a consultant with ample experience in ISO management systems and with the operation and compliance of municipal water systems (one participant).

One challenge was to ensure meaningful representation of small municipalities (i.e. there are 444 municipalities in Ontario, who own and/or operate 630 licensed drinking water systems with a similar distribution for wastewater systems). A total of 10 municipalities cover 75% of the population in Ontario, most of these 10 municipalities are located in what is referred to as the Greater Golden Horseshoe (see Figure 1). So, to address this challenge, the focus group included three senior officials from the OCWA. OCWA manages and operates 180 municipal, non-municipal and First Nation water and wastewater systems across Ontario, most of which are in municipalities with fewer than 50,000 people (OCWA 2017). These officials provided significant insights from smaller municipalities in Ontario.

All participants were based in Ontario, except for one consultant/registrar who is based in Quebec. The focus group session was organized by the author, with the support of Dr Webb (PhD Supervisor), and the Ryerson University – Ted Rogers School of Management, and in accordance with the Ryerson University Research Ethics Board protocols. All participants received the research questions in advance, along with brief contextual information based on Tovilla & Webb (2017) and Tovilla (2020a). The key research questions were provided in advance.

The points for which there appeared to be consensus were summarized and noted at the half-way point in the focus group session, and an opportunity was provided for focus group participants to confirm their veracity, to corroborate, elaborate, or disagree, after which the remaining questions were discussed. When the focus group session concluded, the author reviewed and consolidated the notes taken by the two recorders. The summary of transcripts was shared with each individual participant to obtain their feedback. All participants were given the opportunity to review the summary to clarify, comment, or confirm their insights. Based on the revised transcripts, all recorded discussions were reviewed and classified. The confirmed transcripts of the 14 participants were coded and then analyzed. A draft report of the conclusions was prepared and provided to participants for comment. Although consensus was not sought by the author, there was a certain degree of agreement expressed by the focus group participants. By reviewing the transcripts, a coding framework with topics based on the key research question, and sub-questions and categories was developed. The term consensus in this study means overall agreement on a topic by the majority of participants while having no one expressing opposing views.

RESULTS AND DISCUSSION

The participants appeared to have a common understanding that the governance structure for the wastewater and stormwater sector relies on a provincial regulatory framework but also on proactive action by municipalities in adopting best management practices (BMPs) derived from non-state actors that have developed useful tools such as non-state MSSs and protocols (e.g. ISO and Canadian Standard Association (CSA) Group standards, FCM 2005 Protocols). The data from the focus group generated the following topics:

  • (1)

    Perceptions of the effects of EMS on managing municipal wastewater and stormwater systems (positive, negative, and neutral);

  • (2)

    Role of the provincial government in adopting standards for EMS (legislated/ mandatory versus voluntary adoption of EMS);

  • (3)

    Role of municipalities in adopting non-state standards for EMS (peer-to-peer support and lessons learned);

  • (4)

    Role of non-state actors supplementing state-based regulations (risk assessment and standards development processes); and

  • (5)

    Drivers for EMS in the municipal wastewater sector (motivators and barriers).

Table 1 presents the frequency and distribution of each of the five topics across the five participant categories. The data indicate that all five participant classes had a good knowledge and understanding of the subject, its challenges, risks, and opportunities. It also notes that both state and non-state regulators generated and received arguments associated with their role as an agent of change and call for action (topics 2 and 4) with the greatest frequency.

Table 1

Focus group participant categories and argument counts by topic

Participant categoryParticipants’ topic count
#Topic 1 (+/−)Topic 2 (Prov. role)Topic 3 (Mpal. role)Topic 4 (Privt. role)Topic 5 (Drivers)
Provincial regulator 11 
Large municipality 
Small/medium municipality 12 
Private regulator 
Consultant 
Total 14 33 33 15 27 17 
Participant categoryParticipants’ topic count
#Topic 1 (+/−)Topic 2 (Prov. role)Topic 3 (Mpal. role)Topic 4 (Privt. role)Topic 5 (Drivers)
Provincial regulator 11 
Large municipality 
Small/medium municipality 12 
Private regulator 
Consultant 
Total 14 33 33 15 27 17 

Topic 1 – Perceptions of MSS effects on managing wastewater systems

There seemed to be agreement that MSS would have positive environmental and regulatory effects on Ontario's municipal wastewater and stormwater sectors. Participants referenced the success of the QMS for the drinking water systems, but highlighted how challenging it was to implement it for smaller municipalities:

[t]he Walkerton Inquiry identified major governance deficiencies with respect to water treatment, which led to the formalization and standardization of water operations (Fp1).

The phased-in approach set out in O. Reg. 188/07 allowed municipalities classified as small municipalities an 18-month window to comply with the QMS requirement. A non-state regulator emphasized that an EMS is ‘a well-established way of demonstrating due diligence in case of risks being underestimated’ (Fp2). In a concluding remark, a provincial official remarked ‘Do we need ISO or an EMS for wastewater management? Absolutely’ (Fp3). There was no disagreement with this assertion from other focus group participants. This particular topic was noted by the research team to confirm what is taken to be a consensus on the positive effects of the EMS in wastewater systems, where all participants nodded their assent, or added additional arguments:

[….] having the ISO 14001 certification for our wastewater systems translates into municipal council's confidence in our operations, building credibility, legitimacy, and reputation (Fp6).

[….] implementing an EMS is a good information ‘cleanup’ exercise that can help turn operational data into meaningful information (Fp7).

Representatives from all groups expressed the idea that MSS can provide a standardized and systematic approach that enables consistent compliance with regulatory requirements on a long-term basis. Some divergence of opinion emerged on the effectiveness of an MSS, given that management system effectiveness would be dependent on many factors that would vary on a case-by-case basis; nevertheless, EMS was generally seen as having positive effects:

[….] in a worst-case scenario, our EMS has provided staff with baseline literacy around environmental issues that they may not otherwise have (Fp8).

[….] implementing an EMS is an opportunity to ‘remove cobwebs’ and to focus on what makes an organization successful (Fp12).

These arguments were commented on by several members of different representative groups up to a point that it appeared to be a consensus in the understanding of benefits of the MSSs.

Topic 2 – Role of the provincial government in adopting standards for EMS

For this topic, there appeared to be a consensus among focus group participants that EMSs would have to be a provincially regulated requirement in order to ensure that municipalities would adopt it. There appeared to be a consensus that medium to large municipalities would likely have the resources to develop and implement an EMS on their own, but for smaller municipalities, compliance would be more of a challenge. Medium to large municipalities represent approximately 70% of Ontario's population.

All participants recognized the lead role the province would need to play in making such a standard a mandatory requirement. One provincial regulator noted that in the context of the drinking water regulatory overhaul following the Walkerton Inquiry, ‘the QMS requirement is part of an ecosystem of regulations that the ministry established’ (Fp1). It was further noted that:

[….] we would need to change legislation to make EMS mandatory for wastewater systems (Fp1).

[….] the [municipalities] that need help typically will not speak up, which is why a management standard should be made mandatory (Fp4).

The term ‘ecosystem of regulations’ was used to describe the array of regulations introduced after the Walkerton Inquiry, which formed the multi-barrier approach (i.e. regulations for: source water protection, disinfection, management of the water treatment-distribution systems, and monitoring of drinking water) with multiple regulations for drinking water.

There also seemed to be agreement among the focus group participants that legislative/regulatory requirements would not be immediately feasible because they would require an understanding of the different operating contexts and necessitate significant consultation efforts before the appropriate regulations were crafted and then moved through the legislative process. A distinction was also noted between ‘legislated’ and ‘mandatory’ requirements. A legislated process begins with a bill introduced to the Legislative Assembly and concludes with Royal Assent, when it typically comes into force (Kaye 2011), with significant timelines involved. Conversely, a mandatory requirement could be accomplished through provincial administrative tools such as Ministry Abatement Orders, Minister's conditions for approval, site-specific approvals, or even by referencing specific rule instruments (e.g. guidelines and standards) on permits, licenses, or orders. A mandatory requirement can thus be imposed in a fraction of the time needed for a legislative reform. As noted by another provincial official: ‘I can go back to my office and push a button [in order to require an EMS]’ (Fp3).

One point that emerged on multiple occasions in the focus group was the idea of having a risk-based approach by developing a standard that would be scalable, flexible, and with gradual or phased implementation. It was recognized by one of the non-state regulators that ‘in the municipal sector, if it is not required, the uptake is generally poor’ (Fp11), a position that was supported by the medium and small municipal representatives.

There appeared to be a consensus among focus group participants in favor of an EMS type of requirement needed to accommodate municipalities that have limited resources, by having a scalable, adaptable, and risk-based standard that could be implemented gradually, drawing on the lessons learned from the DWQMS experience regarding drinking water and its phased approach through O. Reg. 188/07. It was suggested that: ‘… an economic analysis (costs versus benefits) needs to be conducted for the application and implementation of a wastewater/stormwater EMS’ (Fp2).

In terms of risk assessment, there appeared to be recognition among focus group participants in relation to the DWQMS that the risk assessment element was addressed by having: (1) the required Critical Control Points (for water quality verification) embedded into the Act and regulations and (2) risk assessments undertaken pursuant to O. Reg. 284/07 (for source water protection areas) and O. Reg. 288/07 (for source water protection committees). To address the risk aspect of EMS for wastewater and stormwater systems, focus group participants proposed a similar approach to those for source water protection:

[….] by having a similar approach of pre-identifying risks – with identified control measures to mitigate these risks – and using the existing source water protection policy tools, we could potentially avoid smaller municipalities having to undertake costly risk assessments (Fp14).

[….] addressing wastewater and stormwater in one single standard would be complex since for the most part risks are different and have different drivers [….] there are [also] some cross-over issues between wastewater and stormwater that it would be necessary to address [….] in a different management standard (Fp11).

In discussions of the need for a wastewater and stormwater management standard, there appeared to be a consensus that smaller municipalities do not necessarily have less risk, and that any risk assessment involves equating magnitude of consequences (or severity) with the likelihood of the occurrence of the risk event. Given that one of the purposes of regulations is to mitigate risks, two alternatives were suggested by one provincial regulator:

[….] [an EMS] could be required for all municipalities; or on a case-by-case basis, if some municipalities are not performing well in the area of wastewater management, then the province could require them to implement an EMS as a compliance response (Fp2).

The idea that a ‘one-size-fits-all’ standard may not be adequate for the wastewater and stormwater sectors was shared by the focus group participants representing municipalities. Provincial officials observed that wastewater and stormwater management is more complex than drinking water management, as site considerations affect the quality of influent wastewater quality, and lakes and rivers have different assimilative capacities, that in turn determine pollutant loadings and corresponding technology. It was also argued that a gradual approach offered advantages for sound implementation:

[….] any path forward for EMS in municipal wastewater needs to follow a gradual/phased, risk-based approach that has to allow for contextual factors such as available resources, risk, size, geography, industrial waste output, etc., and options for scalable and adaptable EMS scope (Fp3).

Assuming a standard was established, additional roles were suggested for the provincial regulator. Some of these roles included using ISO 14001 as a model to develop a sector-specific standard and/or guidelines for the municipal wastewater sector. This should reduce the work of the municipalities, allowing them simply to complete the details and implement it. Finally, it was suggested that either provincial or private regulators should have a shared role in regularly maintaining and updating the standard to reflect changes to approaches in technology, new expectations and new concerns:

[….] it is not enough to develop and implement a standard. It must be regularly maintained and updated; reflecting process updates and changes and change management must be built into a standard (Fp11).

Topic 3 – Role of municipalities adopting non-state standards for EMS

There was a consensus among participants that a relatively informal peer-to-peer learning network of municipalities to share best practices would be useful and could enhance performance. As one non-state registrar noted:

[….] assuming there is provincial authority behind a standard's implementation, a steering committee composed of representatives from interested parties would be beneficial … identifying subject matter experts and sharing that expertise will eventually lead to a collective rise in performance (Fp12).

At the focus group session, the role of municipalities in voluntarily adopting an EMS for wastewater was pointed out by the author. It was observed that Richmond Hill, York Region, and Durham Region, which implemented an EMS approach certified to ISO 14001, are now expanding the scope of their management system to other public works areas. Participants from larger municipalities noted the value of the decision to keep expanding the EMS to their wastewater systems:

[….] an EMS approach for municipal wastewater systems could be developed in consultation with municipalities to ensure core requirements are captured (Fp3).

[….] compatibility and consistency to manage both water and wastewater systems would strengthen the case for a voluntary wastewater standard (Fp5).

[….] [municipalities] would benefit by having the DWQMS synchronized with the ISO family of standards (Fp7).

[….] this means that the DWQMS would be significantly improved if the methodology closely followed that of the ISO management standards (Fp8).

Nevertheless, focus group participants for municipalities remarked that obtaining council support to fund voluntary programs is challenging. Having guidance and clear direction from the provincial government would facilitate efforts to secure municipal funding:

[….] a mandatory EMS would make it easier for municipalities to obtain council support and funding. If left voluntary, then [funding] would be dependent on leadership (Fp5).

From the perspective of small municipalities and operators, participants noted the challenges not only in securing funding, but also regarding capacity-building, and long-term sustainability:

[….] in order to determine whether a wastewater EMS should be mandatory, a conversation around funding must occur. Small and remote townships and municipalities in particular face challenges:

  • they do not have the same manpower as larger ones,

  • they face [a] lack of resources at the local level,

  • they have a knowledge challenge (Fp9).

[….] it is possible for a wastewater management standard (potentially based on ISO 14001) to align with DWQMS so that there are fewer new requirements for municipalities to tackle, as most municipalities have transferred generic requirements of the DWQMS to their wastewater system operations (Fp10).

Another important observation noted by focus group participants representing municipalities experienced in ISO 14001 was that with an EMS, the investigative role of municipalities, when conducting their own root cause analysis, can reduce the need for provincial resources for investigation and abatement:

[….] when a municipality identifies an issue, [it] completes its own investigation and advises the provincial regulator. In some cases the investigative role of the MECP can be reduced (Fp6).

[….] there are benefits with the annual inspections required by Management Systems, which complement the provincial inspections every five years (Fp5).

The focus group's municipal representatives identified the EMS as a management tool for the risk assessment process: ‘We are identifying areas that we inaccurately predicted were low risk that in fact turned out to be higher risk’ (Fp5) noted one municipal official. It was pointed out that an ISO 14001 does not have built-in Critical Control Points the same way the DWQMS has for drinking water, but ‘that can be addressed with customization’ (Fp6). Finally, as pointed out by a non-state regulator, ‘municipalities should not wait for a disaster to happen before taking necessary measures to demonstrate due diligence in order to avoid penalties, cleanup costs, and legal action’ (Fp6).

While discussing the proactive approach of municipalities in adopting an MSS for wastewater and stormwater systems, with reference to the experience on the drinking water side with the DWQMS, a provincial representative noted that ‘[t]here was a balancing act [….] We gave municipalities the keys of the car, but we required them to follow a robust driving manual [DWQMS], street maps, driving test, monitoring and reporting’ (Fp4).

Topic 4 – Role of non-state actors supplementing state-based regulations

A point made by several participants was that standards are a well-established way of demonstrating due diligence. A non-state regulator noted that:

[….] increasingly, standards are being developed for particular sectors. More and more, they are not using ISO 14001, they are using something based on ISO 14001 that is sector-specific (Fp11).

[….] another advantage is that the non-government standards [such as CSA/ISO standards] have a mandatory review every three to five years, so they stay up to date and relevant (Fp11).

It was also suggested that either provincial or private regulators should share in the task of developing a sector-specific standard:

[….] we could develop an ‘upper and lower tier’ of the standard for larger, and smaller and/or less risky municipalities to adopt (Fp11).

[….] if, like in Walkerton, you cannot show due diligence, then you are exposing yourself to significant operational and legal risk. In the event that a risk has been underestimated, standards are [a] useful way of showing that you've done your due diligence (Fp11).

[….] it is not necessary to have another Walkerton-type incident to provide the motivation for change. With an EMS, municipalities and businesses have the opportunity to identify smaller scale issues themselves before they become too big to manage effectively. These issues can then be brought to the ministry to be addressed, rather than waiting for the ministry to investigate when a serious risk arises; this is a risk-based approach vs. preventive approach (Fp6).

A clear role identified for non-state actors was the third-party auditing process. Large municipality representatives had divergent points of view with regard to the quality of the third-party audits between those conducted by the MECP's accredited registrars, and those conducted under their voluntary ISO 14001. One participant noted that they dropped ISO 9001, as it was duplicative of the DWQMS, and after maintaining it for several years, it was found to provide minimal benefits or added value for continual improvement. However, the business perspective associated with the ISO-type audit was also noted, as ‘it covers different aspects than those covered by the ministry’ (Fp12). Finally, it was pointed out that non-state regulators have placed an emphasis on integrating standards and making them compatible in order to facilitate their implementation:

[….] there needs to be an integrated approach to drafting, budgeting, implementing and running any management system. In addition, given that approximately 40% of ISO 9000 and 14000 requirements are similar, audits need to be integrated to reduce duplication (Fp13).

[….] based on the 2015 new versions of the ISO management system standards, similarities [in the structure of the standard itself, which would assist adoption of multiple standards] for integration [of multiple standards] may be more than 70% [of the standard table of contents] (Fp14).

[….] municipalities we work with, they would be willing to implement [a management system standard for municipal wastewater] right away (Fp13).

Topic 5 – Drivers for EMS in the municipal wastewater sector

Participants discussed EMS motivators and barriers in wastewater and stormwater, both if adopted on a voluntary basis and if mandated by the provincial regulator. One relevant question raised in the discussion was: ‘do we have a clear understanding of what problem [it] is necessary to solve with MSS? The management of the system, or the infrastructure design principles of the infrastructure, or investing in capacity size increase – this is particularly relevant as the design of municipal wastewater and stormwater systems is based on BMPs and guidelines, with no minimum requirements. The question is what we need to fix first’ (Fp6).

This topic became one of the most debated points by the focus group participants during the second half of the session. As one municipal representative put it, ‘before thinking about making EMS a provincial requirement, we need to define what problem we are trying to solve, and how solving the problem will benefit the province’ (Fp7). A provincial regulator also pointed out that:

[….] phasing in a new requirement such as an EMS is relatively straightforward, we did that with the DWQMS. But what is more challenging is getting to the stage where you are ready for making it a requirement (Fp1).

There was agreement that the three problems noted above – management of the system, the infrastructure design principles, and investing in increasing capacity – need to be addressed, but there was no agreement as to which to look into first. There was a general understanding that MSS and design standards would assist in achieving the ultimate goal: to improve the levels of service to citizens and the community by reducing floods, spills and CSOs, flooding of people's basement, inflow/infiltration, and environmental degradation. Having management standards and adequate design standards would certainly help to achieve this objective. However, in terms of economic resources the question is where to spend tax-payer money: e.g. on an MSS or on increasing infrastructure capacity.

Another important driver noted was that the existing wastewater and stormwater regulations are very old, with the regulatory framework for the stormwater sector being particularly out of date. Specifically, it was pointed out that:

[….] policies and procedures [for wastewater and stormwater] can be quite old, with some still using the same ones they did in the 1950s and 70s (Fp8).

[….] the obsolete state of stormwater regulatory framework is more dramatic, in that it relies only on guidelines, and there are no legislative requirements (Fp3).

[….] [since Walkerton,] municipalities have been exploring the applicability, fit, and transferability of the drinking water management standard for the purposes of wastewater management (Fp4).

Toward the end of the focus group session, and considering the consensus reached on the positive environmental and regulatory effects of an EMS on Ontario's municipal wastewater and stormwater sectors, the debate concentrated on how to persuade politicians, council members, the public, and the medium to small municipalities of the need to implement EMS. To answer this question, several participants remarked that hopefully it would not be necessary to have another Walkerton-type incident to provide the motivation for change. Building on the need to manage risk, it was noted that an EMS approach can assist in demonstrating due diligence, which is beneficial in cases of environmental violations, and acts as an incentive for general managers and owners to adopt EMS: ‘…. a precautionary approach should be followed instead of regulation by disaster’ (Fp3). Moreover, recent regulations on asset management (O. Reg. 588/17) and infrastructure planning (e.g. Infrastructure for Jobs and Prosperity Act, 2015) were perceived as the catalyst for the MECP to take intermediate steps:

[….] this can support the development of targeted operational guidelines for user class, risk assessments, and use of mandatory requirements with built-in flexibility for adoption and implementation (Fp3).

The key research question underlying this study was also addressed by participants, i.e. is there value in creating a provincially mandated municipal wastewater MSS and a stormwater MSS? Based on literature, laws and court decisions, and the focus group, there appears to be a good basis of support for the establishment of a provincially required Wastewater and Stormwater Environmental MSS aligned with the ISO 14001 EMS standard and some aspects of the DWQMS.

It was also apparent that there is no political appetite in the provincial government to embark on a mandated MSS, so the preferred option at this time appears to take the form of a provincially endorsed, voluntary sector-specific standard for wastewater and also for stormwater, which could constitute a catalyst to boost voluntary uptake of MSSs by small to medium municipalities (as is already occurring with large municipalities).

SCOPE FOR MUNICIPAL WASTEWATER AND STORMWATER MSS

This section considers the basis for selecting the appropriate MSS for the municipal drinking water, wastewater, and stormwater sectors, followed by a discussion of the appropriate scope of the management standard for municipal wastewater and stormwater.

Based on the risks and hazards associated with the provision of water/wastewater/stormwater services can be grouped as follows:

  • Risks to human health, i.e. waterborne disease.

  • Environmental risks, such as contamination of soil, rivers and lakes, e.g. combined sewer overflow (CSO) (from combined sewer systems (CSS)) or sewer system overflows (SSO) (from separate sewer systems (SSS)). (Combined sewer overflow (CSO) refers to a discharge to the environment from a combined sewer system that usually occurs as a result of a rainfall event when the capacity of sewers are exceeded (MECP 2019); combined sewer system (CSS) refers to a collection system that contains sanitary sewers and stormwater sewers within the same sewer collection system (adapted from MECP 2019); sewer system overflow (SSO) refers to an overflow when additional flows enter the sewer system due to rainwater, groundwater rise or emergency situations (blockage such as grease build-up, tree roots in pipes, or accumulation of ‘unflushables’) overloading the system capacity (sewer pipes) (WEAO 2019); separate sewer system (SSS) refers to collection systems that contain separate sanitary sewers, and separate stormwater, making them two separate collection systems (adapted from MECP 2019).)

  • Harm to public or private property, such as sewer back-ups, flooding.

In response to the Walkerton Inquiry (2002), the Ontario government connected the dots to address the health hazards associated with the provision of drinking water, designing the DWQMS to reduce the likelihood of health hazards affecting Ontario's residents (Tovilla & Webb 2017).

Figure 3 represents an attempt to illustrate the QMS approach with controlled inputs and outputs, similar to a manufacturing facility, where an organization can control all inputs and have a quality product (outputs).

Figure 3

DWQMS single inputs and outputs (controls: QMS process).

Figure 3

DWQMS single inputs and outputs (controls: QMS process).

A QMS is designed to ensure that a product is manufactured at the same level of quality regardless of where it is manufactured. Manufacturing a beverage like drinking water requires a QMS/HACCP approach to ensure it is delivered (output) with the same level of quality regardless of the water source used (lake, river or wells). This is controlled through strict standards for design, as well as manufacturing (or construction of the infrastructure that will distribute water). In the drinking water context, the MECP accomplished an effective DWQMS with the support of regulated design standards and source water protection to ensure continual water pressure, testing, and disinfection.

In contrast, an EMS standard is intended to ensure that there are procedures in place to reduce impacts on the environment, including addressing the impacts of the broader infrastructure in both design capacity and performance. Figure 4 illustrates the EMS approach with uncontrolled inputs and outputs. The objective here is to minimize undesired outputs considering the uncertainty of multiple inputs to the system.

Figure 4

EMS wastewater – multiple inputs and outputs (controls: EMS process).

Figure 4

EMS wastewater – multiple inputs and outputs (controls: EMS process).

The primary objective of an EMS is to have systems in place that would reduce the risk of impacts on the environment, taking into consideration inputs (i.e. inputs are heavily dependent on weather, rain, inflow and infiltration, multiple non-controlled catchbasins, and cross-connections) and fewer opportunities to ensure a quality in the ‘manufacturing’ process.

In terms of wastewater and stormwater infrastructure, the key question in selecting between a QMS and an EMS is how far the organization is prepared to go to ensure that ‘inputs’ and ‘outputs’ are adequately controlled. This means selecting between ‘ensuring’ and ‘influencing’ the design and construction of infrastructure to certain minimum criteria.

While the QMS requires ensuring infrastructure is designed, built, and delivered, the EMS requires influencing the delivery of such quality infrastructure. Influencing is easier to document and achieve than ensuring. The EMS objective of environmental protection at a minimum should involve ensuring that the wastewater/stormwater infrastructure in place is well operated and well maintained, and that when disasters do occur municipalities have effective response procedures to reduce risks of environmental and property damage.

Involving design and construction of wastewater and stormwater infrastructure brings two separate dimensions, and is likely the reason why most entities (public or private) adopting ISO 14001 focus only on operations rather than design and construction. The two complexities if involving design and construction in the scope of the MSS include: the application of appropriate treatment technologies and design options (i.e. equipment selection, capacities, and redundancies) to achieve desirable environmental goals; and the use of appropriate methods for construction to reduce likelihood of environmental risks during construction.

As shown in Table 2, the risks and hazards are different for drinking water, wastewater, and stormwater, and as a result, an EMS standard would seem to be a more appropriate approach to address the environmental and public/private property damage risks. Moreover, EMS is widely accepted by Canadian courts and legislation as the most suitable MSS for municipal wastewater and stormwater activities (Tovilla & Webb 2017).

Table 2

Prominent risks/hazards versus most adequate MSS

Municipal water sectorsProminent risks/hazards
Most suitable MSS likely to address risk
Human healthEnvironmentalProperty
Drinking water Significant Low Low QMS, HACCP 
Wastewater Medium Significant Significant EMS 
Stormwater Low Significant Significant EMS 
Municipal water sectorsProminent risks/hazards
Most suitable MSS likely to address risk
Human healthEnvironmentalProperty
Drinking water Significant Low Low QMS, HACCP 
Wastewater Medium Significant Significant EMS 
Stormwater Low Significant Significant EMS 

Furthermore, the current trend is to develop sector-specific standards to meet sector-specific needs. Most such standards are developed by editing the main ISO standards. This development is relevant given the fact that a sector-specific QMS was created to meet the needs of the Ontario government for drinking water systems in 2006 in the form of the DWQMS. The DWQMS is an Ontario-made standard based on ISO 9001 and HACCP (Tovilla & Webb 2017).

The scope of the MSS was also discussed with the focus group participants. While it was acknowledged that the DWQMS is a standard that covered planning, design, construction, commissioning, and operations, it was noted that such a wide scope for a wastewater and stormwater management standard would be too onerous and close to impossible to implement without designated funding a strong legislative authority. Focus group participants noted that the practical approach for a voluntary MSS would be one that brings awareness to the planning, design, and construction, but focuses on the operational processes inherent in the collection and treatment.

The operational processes and business support processes align with many elements of the DWQMS such as the annual planning, management review, audit, continuous improvement, sampling, and document control. Moreover, many municipalities have already expanded some of those business processes from their drinking water operations to their wastewater operations. On the other hand, high-level elements such as master planning, watershed assessments, design, and construction, are important elements that an MSS should be aware of, but not included as a requirement. In Ontario, such high-level aspects follow established and legislated protocols under the Planning Act, the Drainage Act, and the Environmental Protection Act.

CONCLUSIONS

This study found widespread support among the state and non-state focus group participants for selecting the QMS as the best approach to address water quality objectives; and the EMS as the more adequate approach that could have positive environmental and regulatory effects on wastewater and stormwater sectors. The concept of ‘standardization of operations’ was generally acknowledged as one of the primary benefits of the new array of regulations for drinking water. An EMS-based approach was also noted as a safety net in the event of environmental risks being underestimated and as an enabler for consistent compliance with regulatory requirements and making municipalities more efficient (in removing cobwebs).

There was also consensus that an EMS approach would best be achieved through a provincially regulated requirement. There was consensus that medium to large municipalities would likely have the resources to develop and implement EMS, but that there would be significant implementation challenges for small to medium municipalities (i.e. lack of resources for development and operationalization, complexity, and capacity). A scalable, adaptable, and risk-based approach for gradual implementation was seen as preferable.

There appeared to be general agreement that addressing wastewater and stormwater environmental management in a single standard would be complex since for the most part risks are different and have different drivers. It was also apparent that there is no political appetite in the provincial government to embark on a mandated MSS for wastewater and stormwater, so the preferred option at this time appears to take the form of a provincially endorsed, voluntary sector-specific standard for wastewater and also for stormwater, which could constitute a catalyst to boost voluntary uptake of MSSs by small to medium municipalities (as is already occurring with large municipalities). This standard could be based on a customized variation of the ISO 14001, the DWQMS and other standards, and its scope should cover the operational processes and support businesses processes but does not include the planning, design and construction, which would make it less feasible for municipalities to implement and adopt. Figure 2 illustrates the typical components in scope and out of scope for a typical wastewater and stormwater system.

A clear opportunity was identified for the provincial regulator and non-state actors to work together, similarly to what occurred for the development of the DWQMS for drinking water, to develop and endorse a sector-specific standard for wastewater, and another for stormwater. As for having the EMS as a risk management tool, it was noted that for smaller municipalities the transfer of the concepts of pre-determined critical control points would assist them to avoid resource-intensive risk assessments and apply their scarce resources more efficiently.

Resulting from presentations of 2019 preliminary findings of this study to the Ontario, Ministry of the Environment (MECP) and the Canadian Standards Association Group (CSA Group), the CSA Group is taking the initiative to develop two separate standards: the CSA-W211 Management Standard for Stormwater Systems (Barber & Zupancic 2020) and the CSA, Made-in-Ontario Wastewater Management Standard.

A limitation of this article is that it focuses on water governance of municipalities in Ontario. Future research could examine the practical utility of examining water governance to other contexts (e.g. water governance for First Nations, small communities of <500 people, and in non-Canadian jurisdictions).

DATA AVAILABILITY STATEMENT

Data cannot be made publicly available; readers should contact the corresponding author for details.

REFERENCES

Barber
L.
Zupancic
B.
2020
Setting a New Management Standard for Stormwater Systems, Water Canada, September 21, 2020
. .
Behm
D.
2013
Milwaukee Marks 20 Years Since Cryptosporidium Outbreak
. .
CELA
2011
Ontario Safe Drinking Water Act, 2002 & Its Regulations: FAQs, Canadian Environmental Law Association, November 25, 2011
.
Available from: http://www.cela.ca/sites/cela.ca/files/Water-FAQs-Eng.pdf (accessed 12 October 2020)
.
City of London
2008
Regional Water Boards Receive Ontario Public Works Association Awards - Staff Report of January 28, 2008, [Online]. Available from: http://council.london.ca/meetings/Archives/Agendas/Environment%20and%20Transportation%20Committee%20Agendas/ETC%20Agendas%202008/2008-01-28%20Agenda/item%201.pdf (accessed 15 Nov 2015).
Cresswell
J. W.
Plano Clark
V. L.
Gutmann
M. L.
Hanson
W. E.
2007
An Expanded Typology for Classifying Mixed Methods Research Into Designs, Chapter 7 – Advanced Mixed Methods Research Designs
.
Sage Publications
,
Thousand Oaks
.
Delmas
M. A.
Montiel
I.
2008
The diffusion of voluntary international management standards: Responsible care, ISO 9000, and ISO 14001 in the chemical industry
.
Policy Studies Journal
36
(
1
),
65
93
.
FCM/NRC
2005
Environmental Management Systems for Municipal Infrastructure, Federation of Canadian Municipalities and National Research Council
. .
Franceschini
F.
Gelatto
M.
Cecconi
P.
2006
A worldwide analysis of ISO 9000 standard diffusion: considerations and future development
.
Benchmarking in Total Supply Management
13
(
4
),
523
541
.
Greenbaum
A.
Wellington
A.
2010
Environmental Law and Policy in the Canadian Context
.
Captus Press
,
Concord, Ontario
.
Heras-Saizarbitoria
I.
Boiral
O.
2013
ISO 9001 and ISO 14001: towards a research agenda on management system standards
.
International Journal on Management Reviews
15
,
47
65
.
ISO
2017
The ISO Survey of Management System Standards Certifications 2016 – ISO 14001 by Industrial Sector
.
Available from: https://www.iso.org/the-iso-survey.html (accessed 3 August 2018)
.
ISO
2020
ISO Standards Catalogue
.
Available from: https://www.iso.org/standards-catalogue/browse-by-ics.html (accessed 12 October 2020)
.
Kaye
P.
2011
Legislative Research Service – When Do Ontario Acts and Regulations Come Into Force?
.
MECP
2016a
2015–2016 Chief Drinking Water Inspector Annual Report, Nov 2016
:
Ontario MECP
,
Ontario
, .
MECP
2016b
Chief Drinking Water Inspector Annual Report 2014–2015
. .
MECP
2018
A Made-in-Ontario Environmental Plan, Ministry of the Environment
.
Conservation and Parks
,
Ontario
,
2018, EBR 013-4208, Available from: https://prod-environmental-registry.s3.amazonaws.com/2018-11/EnvironmentPlan.pdf (accessed 12 October 2020)
.
MECP
2019
Environmental Compliance Approval Template, Ontario MECP
.
Water Environment Association of Ontario – Practitioners Group
,
Available from: www.weao.org/wastewater_practitioners_group (accessed 24 March 2019)
.
Michigan State
2018
The Michigan Guide to Environmental, Health and Safety Regulations
. .
Naumova
E. N.
Egorov
A. I.
Morris
R. D.
Griffiths
J. K.
2003
The elderly and waterborne Cryptosporidium infection: Gastroenteritis hospitalizations before and during the 1993 Milwaukee outbreak
.
Emerging Infectious Diseases
9
,
418
425
.
O'Connor, D. R. (The Honorable Dennis R. O'Connor)
2002a
Part One of the Report of the Walkerton Inquiry, Ministry of the Attorney General
. .
O'Connor, D. R. (The Honorable Dennis R. O'Connor)
2002b
Part Two of the Report of the Walkerton Inquiry – A Strategy for Safe Drinking Water, Ministry of the Attorney General
. .
OCWA
2017
Annual Report
.
Available from: www.ocwa.com/sites/default/files/ocwa-ar2018-eng-tagged.pdf (accessed 15 September 2019)
.
Penn State Food Science-PSFS
2005
HACCP Historical Timeline – Food Safety & Sanitation
. .
Rhoads
W. J.
Garner
E.
Ji
P.
Zhu
N.
Parks
J.
Schwake
D. O.
Pruden
A.
Edwards
M. A.
2017
Distribution efficiencies operational deficiencies coincide with reported Legionnaire disease clusters in Flint, Michigan
.
Environmental Science & Technology
51
,
11986
11995
.
Safe Food Alliance
2019
The History of HACCP
.
Available from: https://safefoodalliance.com/haccp/the-history-of-haccp/ (accessed 12 October 2020)
.
Searcy
C.
Morali
O.
Karapetrovic
S.
2012
An analysis of ISO 14001 and suggested improvements
.
Journal of Global Responsibility
2
(
2
),
278
293
.
Sinclair
D.
1997
Self-regulation versus command and control? Beyond false dichotomies
.
Law and Policy – University of Denver
19
(
4
),
529
559
.
Sivaram
N. M.
Devadasan
S. R.
2012
A literature review on the integration of total productive maintenance elements with ISO 9001 standard
.
International Journal of Productivity and Quality Management
9
(
3
),
281
308
.
Stemler
S.
2001
An overview of content analysis
.
Practical Assessment, Research & Evaluation
7
(
17
).
Available from: http://pareonline.net/getvn.asp?v=7&n=17 (accessed 21 September 2013)
.
Tovilla
E.
2020a
Examining the Evolving Environmental Protection Policy Convergence in the Ontario Municipal Water, Wastewater, and Stormwater Sectors: Analysis of State/Non-State Governance and Value of an Environmental Management System Standard for the Wastewater and Stormwater Sectors, Ryerson University, 2020
.
USFDA 2015 U.S. Food and Drug Administration - Hazard analysis critical control point - HACCP Principles and Application Guidelines, FDA Website. Available from: www.fda.gov/Food/GuidanceRegulation/HACCP/ (accessed 18 Nov 2015).
WDNR
2014
Green Tier – Biennial Report
.
Wisconsin Department of Natural Resources
,
Available from: http://dnr.wi.gov/files/PDF/pubs/co/CO574.pdf (accessed 18 November 2015)
.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying, adaptation and redistribution, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/).