Quantitative microbial risk assessment (QMRA), the assessment of microbial risks when model inputs and estimated health impacts are explicitly quantified, is a valuable tool to support water safety plans (WSP). In this paper, research studies undertaken on the application of QMRA in drinking water systems were reviewed, highlighting their relevance for WSP. The important elements for practical implementation include: the data requirements to achieve sufficient certainty to support decision-making; level of expertise necessary to undertake the required analysis; and the accessibility of tools to support wider implementation, hence these aspects were the focus of the review. Recommendations to support the continued and growing application of QMRA to support risk management in the water sector are provided.

INTRODUCTION

Compliance monitoring of fecal indicator bacteria is inadequate for the provision of consistently safe drinking water – as seen from numerous waterborne outbreaks worldwide (Hrudey & Hrudey 2004, 2014). Sampling is too infrequent and too little water is sampled to identify short-term periods of sub-optimal system performance during which the overall system integrity can be compromised (Medema et al. 2006; Signor & Ashbolt 2006; Smeets et al. 2010). The WHO guidelines for safe drinking water (WHO 2011b) recommend the implementation of water safety plans (WSP) which require a comprehensive risk-based approach to managing hazards from source to tap (WHO 2009).

Quantitative microbial risk assessment (QMRA), the assessment of microbial risks when model inputs and health impacts are explicitly quantified, is a valuable tool to support water safety planning and specific control point parameters for pathogen management (WHO 2016). The role of QMRA within the WSP context was previously explored through the EU-MicroRisk project (2003–2006, www.microrisk.com) within 10 case study drinking water systems in Europe and Australia. At that time, practical implementation of QMRA by regulators and water utilities within the WSP process was limited. Despite clear theoretical advantages, data limitations and the associated uncertainty for decision making were identified as the key challenges to widespread adoption. Since the MicroRisk project, the role of QMRA for water safety management has continued and expanded with many peer reviewed journal publications, and large multi-partnered EU funded projects including HiWATE (Nieuwenhuijsen et al. 2009), TECHNEAU (www.techneau.org) and VISK (www.visk.nu).

Here we review research undertaken on the application of QMRA for drinking water systems to support the introduction and on-going development of drinking WSP by Environment and Sustainable Resource Development (as of 2015 renamed, Environment and Parks), Government of Alberta. Firstly, the application of QMRA for the development of national regulations was reviewed, as these regulations set the framework and criteria within which the WSP may be developed. Secondly, for each of five target areas where QMRA may be used to support WSP, cases from the literature that provide practical examples were identified, software tools that provide support for calculations were reviewed, and strengths and limitations of QMRA within each context are discussed. The important elements for practical implementation include: the data requirements to achieve sufficient certainty to support decision-making; level of expertise necessary to undertake the required analysis; and the accessibility of tools to support wider implementation. These factors are therefore the ultimate focus of this review.

REGULATION AND HEALTH-BASED TREATMENT TARGETS

While it is widely acknowledged that water of poorer quality requires a higher level of treatment to achieve safety, quantitatively specifying how much treatment is required, in terms of technologies or required Log10 pathogen reduction, is a challenge. The QMRA framework provides a systematic approach for defining treatment targets in relation to source water quality (US-EPA 2007; USDA/US-EPA 2012). The approach is illustrated in Figure 1. First, the health target is defined, depending on the required level of safety for the local context. Secondly, the model input assumptions are defined and include consumption volume (unheated water per person × day), dose-response relationship for the reference pathogens, exposure frequency (typically assumed to be daily for the entire year, however may be reduced for consideration of specific event concentrations) and Disability Adjusted Life Year (DALY) health-impact weightings. Then, for any level of contamination in the source water, the required level of pathogen reduction (by treatment and management controls) to meet the health-based (enteric pathogen) tolerable risk target can be determined by optimization. This conceptual approach is recommended by WHO for drinking water (WHO 2011b, 2011a) and for the safe use of wastewater, excreta and greywater (WHO 2006a, 2006b), and has been used in Australia for setting recreational water access (Roser et al. 2006) in order to quantify the required treatment, or the necessary controls, in order to achieve the tolerable risk target. Very limited work has been undertaken for the currently unregulated problem of water-based pathogens (Ashbolt 2015), such as Legionella pneumophila, which may grow post treatment and largely within premise plumbing (Schoen & Ashbolt 2011).
Figure 1

Illustration of approach for defining treatment requirements based on source water pathogen concentration (Petterson et al. 2015).

Figure 1

Illustration of approach for defining treatment requirements based on source water pathogen concentration (Petterson et al. 2015).

Case study examples

The principle

In using the QMRA framework to determine treatment requirements for surface water sources, the principle was presented with the earliest QMRA studies (Regli et al. 1991; Rose et al. 1991), revision of the enhanced surface water treatment rule (US-EPA 2006) and then later extended to account for variability in pathogen concentration (Masago et al. 2002). The US-EPA followed by many other jurisdictions have used QMRA in this way, including Health Canada and others. In principle, the same method could be used at the water supply system-specific scale and context.

In practice

There are limited examples of where the conduct of QMRA at a water supply system specific scale is a regulatory requirement. Most notably, the Dutch Drinking Water Act requires that risk assessment be undertaken for waterborne pathogens at every water supply to demonstrate microbiologically safe water, with a health based target of less than one infection per 10,000 per year (Anonymous 2001). In 2005, the inspectorate guideline 5318 was created to define the requirements of the QMRA including the reference pathogens to be used, the data requirements, means of quantifying treatment efficacy, selection of dose-response models and other important assumptions (Anonymous 2005). Defining these requirements created a clear basis on which the water utilities could quantify their risk and compare with the target, yet certain aspects were not addressed, such as post-treatment recontamination risks during drinking water distribution.

The Australian guidelines for water recycling (NWQMS 2006) were developed in keeping with the guidance of the WHO guidelines for the safe use of wastewater, excreta and greywater (WHO 2006a). The QMRA framework was used to provide a quantitative approach within a structured framework to evaluate a diverse range of potential recycled water uses. The guidelines allow for combinations of treatment options and preventative measures to be selected so as to ensure the water is fit-for-purpose and meets the 1 μDALY.person–1.year–1 health based target.

Software tools

Schijven et al. (2011), at the National Institute for Public Health and the Environment (RIVM) in the Netherlands, developed a software tool to support the evaluation of each water supply in comparison to the Dutch regulation. The model is constructed in Mathematica® software (Wolfram Research, USA), with the assumptions and data analysis approaches as defined in the inspectorate guidelines (Anonymous 2005). Water utilities are required to collect input data for the model and the output is the annual probability of infection for treatment compliance comparison with the annual target of 1 × 10–4 infection risk 95% of the time.

Strengths, limitations and data needs

The experiences of QMRA-based regulation in both the Netherlands and Australia were evaluated by Bichai & Smeets (2013). Their analysis of experiences and perceptions across regulatory bodies, government, water utilities and scientists in both countries identified the following advantages and challenges associated with QMRA-based regulation. In their evaluation, Bichai & Smeets (2013) argue that efficiently addressing small (remote) water systems is likely to require a centralized approach where human resources, data and knowledge are shared.

Advantages

  • QMRA scenario modeling provides a better assessment of water safety than the absence of fecal indicators.

  • Setting a health-based risk target addresses the balance between investments and public safety.

  • Helps staff responsible to better understand risks from their water sources, treatment operation and in particular, likely increased risks during hazardous events.

Challenges

  • Efficient monitoring and designing sufficient monitoring to meet the requirements of the Dutch regulation.

  • The need for institutional support for utilities since QMRA as a regulatory tool cannot stand alone, and effective implementation requires adapted institutional support; such as specific training, development of auditing approaches.

  • Interpretation of uncertainty by regulators. Uncertainty in QMRA science needs to be balanced with the policy approach to uncertainty (i.e. to draw a ‘thin line’ when in reality there is a wider, fuzzier line), in order to allow decision-making and use of the approach by utilities.

  • How to communicate risk to consumers in a consistent and balanced way; often boiled down to meeting best management practices and risk targets.

QMRA IN THE WATER SAFETY PLAN – SITE SPECIFIC APPLICATIONS

A hazard analysis critical control point (HACCP) form of water management was first proposed by Havelaar (1994), however it took another decade before the approach was formalized (Hrudey 2004; Hrudey et al. 2006) and included in guidelines published by the WHO (2004) and in Australia (NWQMS 2006). The EU MicroRisk project (Medema et al. 2006) (www.microrisk.com) aimed to demonstrate how QMRA may be used to aid a HACCP-type of water management that is site-specific. The WSP steps are illustrated in Figure 2, identifying how QMRA can provide valuable input for the development of the plan.
Figure 2

Potential input from QMRA to support water safety planning.

Figure 2

Potential input from QMRA to support water safety planning.

Know your system

An underlying principle of the WSP is that to manage the system effectively, it is important to ‘know your system’. Systems lacking understanding and systematic evaluation of microbial risks continue to cause outbreaks (Hrudey & Hrudey 2004, 2014). Considerations include the following:

  • How impacted is the source water? What are the sources of contamination and how do they vary (including various likely events (all termed hazardous events))?

  • Which pathogens are most relevant for the system? In general, when sewage or septic seepage is possible, enteric viruses, bacteria and parasitic protozoa from these sources all need to be represented by reference pathogens. Whereas in the absence of these human fecal sources (best demonstrated by the absence of human-targeted microbial source markers (Harwood et al. 2014) and supporting sanitary survey information) generally only enteric bacteria and parasitic protozoan reference pathogens are necessary to address zoonotic pathogens.

  • What is the relative importance of different pathogen sources by event types?

  • How effective are the existing treatment barriers for each reference pathogen?

  • What are the vulnerabilities of the drinking water treatment process?

  • What are the vulnerabilities of the drinking water distribution system?

  • What are the implications of these vulnerabilities to the consumer population?

These questions may be addressed qualitatively based on the expertise of the WSP team, however direct quantitative input based on scientific evidence is a valuable support to the risk identification, prioritization and management process.

Case study examples

The following examples demonstrate how the application of the QMRA framework has supported understanding of the system and risk management. By modelling the system from source to exposure, sensitive system components are identified. Most notably, source water fluctuations and treatment failures (Teunis et al. 1997; Jaidi et al. 2009; Cummins et al. 2010) were frequently identified as the most sensitive model components (noting that distribution system effects were not included in these studies, but have more recently begun to be addressed (Teunis et al. 2010; Besner et al. 2011; Yang et al. 2011)).

Know your source water. In investigating the source water contamination level, including the predicted pathogen concentration under baseline and event conditions, QMRA provided value with respect to:

Know your treatment. QMRA provided value with respect to:

  • identification of the importance of managing treatment failures to reduce consumer risk (Teunis et al. 1997);

  • identification of the relative importance of different failure events to prioritize management options (Westrell et al. 2003);

  • assessment of the safety or otherwise associated with recycling filter backwash water to the inlet of the water treatment plant (Loret et al. 2013);

  • demonstration of the importance of system compliance on overall consumer risk (Hunter et al. 2009; Enger et al. 2013; Petterson 2016);

  • the effectiveness of different risk management options for treatment including intake closure during hazardous events to reduce pathogen concentrations (Åström et al. 2007);

  • evaluation of treatment performance with respect to different pathogen loading (Smeets et al. 2007); and

  • demonstration of the value of monitoring and what to monitor to inform pathogen treatment performance (Signor & Ashbolt 2006; Smeets et al. 2010).

Know your distribution system. Overall, distribution systems could account for some 30% of outbreaks in North America (Craun et al. 2010), and premise plumbing may account for the majority of these via Legionella and non-tuberculous mycobacterial growth and subsequent infections that dominate US hospitalization costs via drinking water exposures (Collier et al. 2012), and drinking water outbreak health burden in general (Beer et al. 2015).

Drinking water distribution systems have only begun to be accounted for within drinking water QMRA and methodologies are developing. Case studies were identified that show how QMRA has improved understanding of the risk drivers associated with the distribution network (van Lieverloo et al. 2007; Teunis et al. 2010; Besner et al. 2011) and how that can be translated into risk management strategies (Yang et al. 2011). Critical premise plumbing Legionella concentrations have also been estimated by a reverse QMRA approach (Schoen & Ashbolt 2011), yet there are also other opportunistic pathogens to be addressed that grow post treatment (Ashbolt 2015).

Software tools

The Swedish water and wastewater association developed a QMRA tool for drinking water supplies (Svenskt Vatten 2015) specifically to support the application of QMRA for system understanding. The tool is constructed in the Analytica® (Lumina Decision Systems, USA) software platform, and is designed to be flexible and to allow a diverse range of treatment trains to be modelled, and is available freely online (Svenskt Vatten 2015). The tool includes eight reference pathogens (Salmonella, Campylobacter, Escherichia coli O157, Rotavirus, Norovirus, Adenovirus, Cryptosporidium and Giardia), and each treatment step allows for different failure scenarios to be selected and investigated against nominal conditions. The software is specifically designed as an educational tool for water managers within WSP workshops to demonstrate how microbial risks travel through water supplies and hence how they may be managed. The Health Canada QMRA tool (discussed under ‘System assessment’ below) designed for use by municipal engineers, water treatment plant (WTP) operators, and local decision makers also serves to support system understanding. While the primary purpose of the Canadian tool is to quantify treatment plant risk for comparison with a health target, it has also been identified that ‘the greatest value of using this tool may reside in the systematic evaluation of the process that WTP managers must follow to implement it’ (Tfaily et al. 2015).

Strengths, limitations and data needs

A major strength of the QMRA framework is to pool and interpret existing data, with the overall goal to support system understanding. Within this context, the success of QMRA is not dependent on the amount of data, but rather on effective construct and implementation. The key expertise is therefore in developing representative quantitative inputs of model parameters from sometimes very limited data. A representative quantitative input to the decision making process, even if vague due to data limitations, is superior to expert opinion alone. Perhaps most effective is when using the QMRA framework to test the validity of qualitative assumptions from the WSP team. Even with a basic sanitary survey and literature data the framework can help structure the knowledge base about system attributes and consumer risk to improve water quality management.

The development of the Swedish software tool was intended to make the QMRA approach accessible to water managers, without the need to understand detailed risk calculations. While some success was achieved, the complexity of the underlying assumptions of the model was a barrier to widespread implementation. The initial hurdles associated with understanding the QMRA approach, understanding the limitations of the microbial data and hence uncertainties, and understanding the implications of the model outcomes were underestimated. Further work is needed to communicate these fundamental concepts at an appropriate level of detail for technical professionals who are not QMRA specialists.

In contrast, the Health Canada QMRA tool only uses stochastic inputs for reference pathogens (not treatment performance), generally with default values, and largely misses the opportunity for managers/operators to explore and get to know their system. Further, while the results seem clear, because of the lack of uncertainty integrated within the tool's approach, results are very likely to be misleading and over-simplistic; giving a false sense of precision and safety. Neither tool addresses distribution systems nor premise plumbing-related pathogen risks and hence, both may fail to provide perspective for system managers to prioritize expenditures for overall pathogen management of the complete source to tap system.

System assessment

QMRA provides a clear and transparent approach for comparing system risks with a health target, making it possible to evaluate if a system or pathway is safe (Figure 1). When the system is not safe, the steps that need to be undertaken in order to achieve the required level of safety can be evaluated and compared. In addition, circumstances (or event-driven conditions) during which the safety may be compromised can be identified.

Case study examples

The following case study examples demonstrate the evaluation of risk for direct comparison with a tolerable risk target to evaluate safety:

  • Medema et al. (2003) evaluated three separate systems for Cryptosporidium. In one case the risk was well below the risk target, and therefore the system was deemed safe. In the second case the risk was well above the risk target and additional control measures were identified. In the third case, the risk was of a similar magnitude to the risk target and further investigation to reduce the uncertainty was recommended.

  • Stored household water in South Africa was identified as unsafe with respect to Salmonella (Steyn et al. 2004).

  • Risk of Cryptosporidium and Giardia from very small water supplies in the UK and France were evaluated to be extremely high (Hunter et al. 2011).

  • Risk from Adenovirus via drinking water and recreational waters exceeded health targets (van Heerden et al. 2005).

  • For surface water impacted by sewage effluent risks were demonstrated to be tolerable, even under hazardous event conditions (van den Akker et al. 2011).

  • Investigation of uncertainty and variability and the implications of comparing risk with a tolerable target have been presented (Pouillot et al. 2004; Jaidi et al. 2009).

  • Critical level for Helicobacter pylori in drinking water to be protective of life-time stomach cancer via drinking water (Ryan et al. 2014).

  • Critical values of Legionella (moving back ‘upstream’ to aid in identifying management targets) in aerosols, drinking water and actual sites of growth within biofilms associated with premise plumbing (Schoen & Ashbolt 2011).

Software tools

Health Canada (Tfaily et al. 2015) has developed a stochastic tool in Excel® for modeling health risks for comparison with the US-EPA used benchmark of 1 × 10–4 annual infection probability and the WHO benchmark adopted by Health Canada of the 1 μDALY.person–1.year–1. Reference pathogens include Cryptosporidium, Giardia, Rotavirus, Campylobacter and diarrheic E. coli. Users are required to input the concentration of pathogens in raw water as the mean and standard deviation for characterization by a log-normal distribution, and log10 reductions by treatment are defined based on reported values in the literature. The Guidelines for Canadian Drinking Water Quality encourage the implementation of a risk-based multiple barrier approach that includes QMRA (Health Canada 2012; Tfaily et al. 2015). The tool is intended to be applied by end users to evaluate alternative strategies to satisfy regulatory requirements, evaluate the robustness of a given treatment train, or determine crucial situations in which the risk of exposure may be increased, hence there is some overlap with the purpose of supporting system understanding (previous section).

Strengths, limitations and data needs

Measures for independently identifying whether a risk pathway is ‘safe’ are extremely limited. Traditional epidemiologic approaches that look for elevated levels of disease associated with a particular water source or pathway lack the sensitivity for verifying the low levels of risk desired for drinking water systems (i.e. best randomized control double blinded drinking water studies can only detect illness when it exceeds 10% within the study population (Hellard et al. 2001) not the desired 0.01%). In addition, they lack the statistical power and flexibility for undertaking system specific evaluation of water safety across a broad range of systems and event conditions. As for chemical risk assessment, QMRA overcomes both of these limitations and is therefore extremely valuable for comparing risks for a particular system with an independently determined benchmark.

There are, however, constraints in the practical implementation of the QMRA approach. The discussion points associated with defining treatment requirements in ‘Regulation and health-based treatment targets' above are also relevant to system assessment. Firstly, the target benchmark in terms of risk measure (e.g. daily probability of infection, annual probability of infection, probability of illness, DALY), and the associated statistic (e.g. mean, 95th percentile) need to be defined. There has been criticism that extremely low benchmarks (such as the WHO 1 μDALY.person–1.year–1) may be unnecessarily low and hence driving an imbalance in the focus on drinking water in comparison to other pathways of infection (such as the 3.5% GI illness required by recreational water criteria (EPA 2012)) – yet this harmonized approach is recognized by WHO in their overall framework for water exposure pathways (Fewtrell & Bartram 2001).

Secondly, it is important to note that the model input assumptions (exposure volumes, dose-response models, DALY weightings, etc.) can make an important impact on the quantified risk, and need to be carefully defined, particularly if a ‘level playing field’ is desired for comparative studies. In some cases choices between comparable inputs can make more than an order of magnitude difference in the estimated risk, and even larger changes if hazardous events are not sampled or estimated (e.g. Signor & Ashbolt 2009). The most suitable inputs for the local region and for the purpose of the assessment need to be selected, and if comparison between systems is the objective, then these inputs need to be constant among QMRAs.

Thirdly, the site-specific data collection and analysis requirements need to be carefully defined. For example, when the source water contamination level is based on a limited environmental dataset, concentrations predicted from samples collected during baseline conditions (including estimates of their standard deviations, as used in the Health Canada tool) can be expected to be much lower than concentrations/SD predicted from samples collected during events. Under the US-EPA surface water treatment rule (US-EPA 2003b) it is necessary to collect 48 samples and analyse for Cryptosporidium. The conditions under which these samples are to be collected are not prescribed. Given the extreme variability in Cryptosporidium concentration in many systems, the conditions under which samples were collected will fundamentally influence the required level of treatment to achieve safety – e.g. during dry versus wet and snow melt conditions.

Critical control points and setting critical limits

Operational limits and critical control points for water treatment plants need to be defined in order to manage the system. Parameters including turbidity (on raw water to define coagulant needs and on finished water to determine filtration efficacy), and chlorine residual are monitored to ensure that the plant is operating effectively. Appropriate values are typically chosen based on the design requirements and expected reduction performance of the treatment plant. It is possible to use QMRA to define the critical limits so that the required level of treatment is achieved, without undue redundancy.

Case study examples

The concepts that have led to the development of using QMRA for establishing critical limits for water treatment plant operation are included in the following examples:

  • In the US-EPA surface water treatment rule (US-EPA 2003a), filtration and chlorine dose is related to Log10 reductions for parasitic protozoa and enteric viruses (Regli et al. 1991; Rose & Gerba 1991).

  • QMRA has been used to compare risks between different agents to balance risks associated with disinfection dosing (Havelaar et al. 2000; US-EPA 2001; Petterson et al. 2010).

  • Evaluation of hydraulic requirements for achieving expected disinfection efficacy (Petterson & Stenström 2015).

  • Zhang et al. (2012) developed a performance assessment that integrated the concepts of reliability and robustness within QMRA. This was a first step in linking measurable plant performance indicators with achievement or failure to meet health-based targets.

The concepts of using QMRA to establish critical limits has been most clearly explored through examples presented by Smeets (Medema & Smeets 2009; Smeets et al. 2010), however all of these examples are still conceptual, and no examples or case studies were identified in the peer reviewed literature addressing real plant operations that had adopted this approach for defining critical limits and operational targets.

Software tools

As part of the EU-HiWATE project a software tool was developed to compare the risk benefit tradeoff between free chlorine disinfection and TTHM formation (Petterson et al. 2010). The overall objective was to move toward defining optimal chlorine dosing in order to achieve adequate pathogen reduction without excess disinfection by-product formation. The tool was constructed using the Analytica® (Lumina Decision Systems, USA) software platform and was run for six reference pathogens (Campylobacter, E. coli O157:H7, Rotavirus, Norovirus, Giardia, and Cryptosporidium) and compared DALYs associated with health outcomes of gastroenteritis and bladder cancer.

Strengths, limitations and data needs

The concepts associated with using QMRA to establish critical limits are well established and make good operational sense. Without this approach there is limited confidence that the limits are appropriately set to protect public health without major redundancy. The most important limitations relate to the scientific uncertainty associated between the measurable surrogate (e.g. chlorine residual, turbidity), the achieved treatment efficacy, and hence impact on health risk. For example, turbidity or 5–20 μm particle breakthrough of filters is known to be somewhat related to pathogen breakthrough (O'Halloran et al. 1999), however pathogens may breakthrough well before measurable turbidity or particle counts are detected by online meters. This disconnect is a function of the poor correlation between pathogens and these physical surrogates in raw waters and that it takes a considerably lower level of pathogens than ‘equivalent’ measurable particles to cause an increase in risk.

Defining or evaluating monitoring requirements

Closely linked to the setting of critical limits is the characterization of monitoring requirements to verify the performance of a treatment plant. Important factors for consideration are: what should be measured, and how frequently, to ensure that health targets continue to be met. Traditional end of pipe testing that relies only on the absence of fecal indicator, is inadequate for verifying the operational integrity of the treatment train, as raised in the Introduction.

Case study examples

Only two case study examples of investigating the role of QMRA for evaluating monitoring requirements were identified for this review:

  • Signor and Ashbolt evaluated the value of monitoring programs for Cryptosporidium and highlighted their inadequacy for protecting public health (Signor & Ashbolt 2006); and

  • Nilsson and coworkers explored the limitations of relying on online SCADA data to ensure health targets are met (Nilsson et al. 2007).

Strengths, limitations and data needs

QMRA can provide valuable input for the development of monitoring programs. The value of monitoring for pathogens in finished water (in terms of ensuring consumer safety) can be quantitatively investigated, and compared with cost.

Monitoring will predominantly rely on process indicators rather than microbial sampling. Process indicators are often not well correlated to consumer risk, however they are well suited to identifying events within the system. The importance of events of different sizes and duration can be explored using QMRA as presented by Westrell et al. (2003), which was based on review of several years of operational events, concluded that source water pathogen variability/events were more important to manage (discussed later for the Åström et al. (2007) work associated with the same Swedish treatment works).

QMRA can demonstrate the certainty of detecting a problem associated with monitoring strategies of different frequencies (e.g. weekly, daily, hourly, online), and can therefore provide input regarding the improvement in water security achieved with respect to the additional cost associated with more frequent monitoring. Given the rapid fluctuations associated with treatment performance, online monitoring is desirable, however operational issues related to calibration and verification require ongoing attention to ensure that the system is functioning properly.

System planning and development

QMRA can be used to help identify the next best step for improving health outcomes. Scenarios, such as different potential interventions or barriers, can be modelled and their relative impact on disease can be compared. These results can be compared alongside other economic, environmental and social factors to support decision making.

Case study examples

Several examples have been identified that explore this approach:

  • The Swedish Urban water project (Ashbolt et al. 2005) explored the application of QMRA within a multi-criteria decision making framework.

  • Microbial risks were included within a sustainability framework for assessing urban water system options in Australia (Lundie et al. 2008; Kobayashi et al. 2015) and in the USA (Schoen et al. 2014).

  • In Uganda, QMRA was applied with limited data to prioritize infrastructure spending to achieve health outcomes (Howard et al. 2006).

  • In Ghana, QMRA was applied to estimate the risk to human health from various sources of drinking water and evaluate the cost effectiveness of different interventions (Machdar et al. 2013).

  • When investigating a high incidence of Cryptosporidium in a southwestern Ontario community, Pintar et al. (2012) applied QMRA to effectively rule out the drinking water pathway as the cause, providing an effective communication tool and prioritizing future work.

  • To determine which arsenic mitigation option presents the lowest disease burden in a particular setting (Howard et al. 2007).

  • QMRA was applied to select among household water treatment devices in the developing context (Petterson 2016).

  • To support fire protection planning following the 2003 Lost Creek Wildfire in Alberta, Canada (Emelko et al. 2011).

  • In comparing life-cycle impacts with pathogen risks from Swedish wastewater treatment options (Harder et al. 2014, 2015), which could impact on source drinking waters.

Software tools

Petterson (2016) developed a software tool to support the application of QMRA as a decision support tool in selecting among household water treatment devices. Given the uncertainty and variability associated with model inputs, the tool was developed to support the exploration of model input assumptions on the outcomes of the analysis. The tool was constructed in Analytica® and used three reference pathogens (Campylobacter, Rotavirus and Cryptosporidium), one to represent each of the microbial groups.

Strengths, limitations and data needs

QMRA is unique in providing a structured approach for evaluating health impacts of hypothetical or planned future activities. It is important to include predicted health outcomes alongside other criteria to support decision-making. The outcomes of any risk assessment are dependent upon the integrity of the information underlying the calculations, and therefore the sensitivity of any decision to these input values must be tested. Additional data needs will depend on the sensitivity of these input values.

CONCLUSIONS AND RECOMMENDATIONS

The studies reviewed have demonstrated how QMRA can be applied to support water safety management and in particular the development of a WSP for a specific water supply. QMRA has unique value for providing quantitative information to support decision making; from the general objective of improved system understanding to the more specific operational requirements of the water treatment plant. While these concepts are well developed in the literature presented, there are limited examples of practical implementation within the water industry. There are several identified reasons for this, which drive the recommendations of this review.

Firstly, environmental and engineered systems demonstrate considerable variability and there is uncertainty associated with quantifying the model components for any QMRA. The data needs of applying QMRA in the WSP are summarized in Table 1, and are classified as fundamental (referring to scientific knowledge) or site specific. The current state of knowledge associated with each of the identified fundamental questions are summarized in Table 2. Some of these uncertainties are large, and appropriate consideration needs to be given in the application of QMRA. As a consequence, results need to be interpreted in the light of the certainty of the input assumptions. Confidence in how to handle uncertainty both from the perspective of the risk assessor and the regulator is essential to support implementation. The following general recommendations therefore follow:

  1. When applying QMRA to support system understanding (i.e. know your system) the most important factor is to ensure that the quantitative values used to represent the scientific evidence are representative. Regardless of the amount of information available, QMRA is valuable to support the understanding of the risk drivers within the system as described/assumed, however poor interpretation of the microbial data could lead to misleading results. It is also valuable (to the extent possible) to represent the uncertainty associated with the quantitative values so that the WSP team are able to understand the level of confidence associated with the quantitative value presented.

  2. When applying QMRA to determine if a system or pathway is safe (i.e. system assessment) the most important factor is to ensure that the underlying model inputs, data collection and analysis assumptions have been clearly defined. These underlying assumptions, together with the health target, define what is meant by ‘safety’. To rely only on the health target, and to leave the other assumptions undefined, creates an ambiguous target and does not address uncertainties.

  3. No practical cases where QMRA had been applied to determine critical limits and operational targets were identified. Rather, over-reliance on some surrogates, such as post-filtration turbidity, which is very site- and condition-specific, yet the water industry has a false sense of security in its use; turbidity is useful for filter performance, but is very loosely related to safe pathogen levels (Table 2). The uncertainty associated with the relationship between measurable surrogates and health impacts is assumed to be too great for the QMRA approach to replace existing limits. Fundamental research to overcome existing knowledge gaps is needed before this can be practically implemented.

  4. When implementing QMRA as a decision support tool for system planning, the most important factor is to ensure that the sensitivity of any decision to the model input assumptions is clearly explored. Outcomes can be dependent upon input values that are highly uncertain and this needs to be transparent to the decision maker. Conversely, in many cases where scenarios are being compared, many of the uncertain inputs are common among options and therefore comparisons can be clearly and confidently made.

Table 1

Objectives of QMRA within the WSP and associated data needs

Objective of QMRA Specific questions that need to be defined Data needs
 
Consequence of inadequate information 
Fundamental Site specific 
Know your system Use scientific evidence to support the development of the WSP Identification and prioritization of hazards and hazardous events 
  • Rather than requiring specific data, this approach allows for whatever data is available to be incorporated into the overall development of the WSP

 
 
System assessment Compare risk with health target to determine if system is ‘safe’ How will ‘safety’ and the model assumptions be defined? 
  • Determine measure for heath target

  • Dose-response relationship

  • Health impact assumptions and DALY weightings (if appropriate)

 
  • Reference pathogen concentrations

  • Exposure volumes

 
Model assumptions and definition of safety can influence the required treatment reductions by several orders of magnitude 
What is the magnitude of contamination of the source water? 
  • What are the sources/hosts of pathogens?

  • How many are excreted by those sources/hosts?

  • How many are human infectious?

  • How well do those pathogens persist in the environment?

 
  • Sanitary survey

  • Identification of pathogen mobilization events including their magnitude and frequency.

  • Water quality data

 
Quantified level of contamination may not be representative of the system, e.g. not considering all conditions (and events); or not considering only human infectious portion of pathogen loading 
How effective are the treatment barriers? 
  • Mechanisms of removal by different treatment barriers

  • Expected Log10 reduction by treatment barriers

  • Expected causes of poor performance

 
  • What treatment barriers are in place?

  • How well do they perform?

  • What fluctuations (magnitude and frequency) are likely for the barrier?

 
Treatment reduction may appear adequate based on expected performance, but site-specific differences or fluctuations may undermine overall performance 
Critical control points and critical limits Define limits for measurable surrogates How does the measurable surrogate relate to consumer risk? What is a safe level for the surrogate? 
  • How does microbial indicator presence and persistence through treatment relate to pathogens?

 
  • What level of treatment is required by all barriers?

  • What are the relevant surrogates for the treatment steps present at the plant to ensure each step achieves its target?

 
A poor understanding of the relationship between measurable surrogates and treatment performance can lead to critical limits that are not well linked with actual health risk. The result may be limits that are overly conservative, or not protective enough 
 Filtration 
  • How does turbidity in finished water relate to filtration performance of pathogens?

 
  • What is the source water turbidity? How do the principles apply to the specific plant?

 
 
 Disinfection 
  • What is the sensitivity of pathogens to disinfectants and the required Ct to achieve the necessary Log10 reduction

 
  • What is the residence time distribution of the disinfection contactor?

 
 
 Distribution 
  • Pathogen controls via SOPs for mains break/repairs, X-connection & back-siphoning, & reservoir cleaning

 
  • System integrity by on-line disinfectant measures, pressure transients, source-specific faecal markers

 
 
Define monitoring requirements What needs to be measured and how often to ensure consumer safety? How does the level of the surrogate relate to consumer risk? ‘As above’ ‘As above’ Monitoring may not be targeted toward the appropriate parameters for ensuring health targets are met, hence providing false security 
How much time is allowable before corrective action is necessary? Based on QMRA modelling Site-specific QMRA modelling 
Objective of QMRA Specific questions that need to be defined Data needs
 
Consequence of inadequate information 
Fundamental Site specific 
Know your system Use scientific evidence to support the development of the WSP Identification and prioritization of hazards and hazardous events 
  • Rather than requiring specific data, this approach allows for whatever data is available to be incorporated into the overall development of the WSP

 
 
System assessment Compare risk with health target to determine if system is ‘safe’ How will ‘safety’ and the model assumptions be defined? 
  • Determine measure for heath target

  • Dose-response relationship

  • Health impact assumptions and DALY weightings (if appropriate)

 
  • Reference pathogen concentrations

  • Exposure volumes

 
Model assumptions and definition of safety can influence the required treatment reductions by several orders of magnitude 
What is the magnitude of contamination of the source water? 
  • What are the sources/hosts of pathogens?

  • How many are excreted by those sources/hosts?

  • How many are human infectious?

  • How well do those pathogens persist in the environment?

 
  • Sanitary survey

  • Identification of pathogen mobilization events including their magnitude and frequency.

  • Water quality data

 
Quantified level of contamination may not be representative of the system, e.g. not considering all conditions (and events); or not considering only human infectious portion of pathogen loading 
How effective are the treatment barriers? 
  • Mechanisms of removal by different treatment barriers

  • Expected Log10 reduction by treatment barriers

  • Expected causes of poor performance

 
  • What treatment barriers are in place?

  • How well do they perform?

  • What fluctuations (magnitude and frequency) are likely for the barrier?

 
Treatment reduction may appear adequate based on expected performance, but site-specific differences or fluctuations may undermine overall performance 
Critical control points and critical limits Define limits for measurable surrogates How does the measurable surrogate relate to consumer risk? What is a safe level for the surrogate? 
  • How does microbial indicator presence and persistence through treatment relate to pathogens?

 
  • What level of treatment is required by all barriers?

  • What are the relevant surrogates for the treatment steps present at the plant to ensure each step achieves its target?

 
A poor understanding of the relationship between measurable surrogates and treatment performance can lead to critical limits that are not well linked with actual health risk. The result may be limits that are overly conservative, or not protective enough 
 Filtration 
  • How does turbidity in finished water relate to filtration performance of pathogens?

 
  • What is the source water turbidity? How do the principles apply to the specific plant?

 
 
 Disinfection 
  • What is the sensitivity of pathogens to disinfectants and the required Ct to achieve the necessary Log10 reduction

 
  • What is the residence time distribution of the disinfection contactor?

 
 
 Distribution 
  • Pathogen controls via SOPs for mains break/repairs, X-connection & back-siphoning, & reservoir cleaning

 
  • System integrity by on-line disinfectant measures, pressure transients, source-specific faecal markers

 
 
Define monitoring requirements What needs to be measured and how often to ensure consumer safety? How does the level of the surrogate relate to consumer risk? ‘As above’ ‘As above’ Monitoring may not be targeted toward the appropriate parameters for ensuring health targets are met, hence providing false security 
How much time is allowable before corrective action is necessary? Based on QMRA modelling Site-specific QMRA modelling 
Table 2

Current state of knowledge and knowledge gaps associated with fundamental research questions of relevance to WSP

Fundamental research question Current state of knowledgea Knowledge gaps 
What are the sources/hosts of pathogens in the drinking water catchment? ★★★★ Sources of human enteric pathogens within drinking water catchments are well characterized 
  • Emerging zoonotic pathogens

 
How many enteric pathogens are excreted by those sources/hosts? ★★★ The excretion density of various pathogens in different hosts is known within a few orders of magnitude 
  • The magnitude of asymptomatic excretion of some pathogens by humans

  • The variability in excretion over the course of an infection

 
How many of those enteric pathogens are human infectious? ★ Molecular methods enable sub-typing of samples to identify specific host links for zoonotic pathogens 
  • Still many unknowns with regard to the proportion of protozoa that are human infectious

 
How many of those pathogens are viable (detectable) and able to cause infection? ★ For culturable organisms the lower boundary of detectible organisms is well known, but not those viable but non-culturable (VBNC) 
  • Viability of organisms detected using modified molecular methods

  • The proportion of organisms that are VBNC

 
How well do pathogens persist in the environment ★★ Many studies under different specific conditions for a range of pathogens 
  • Translation of specific studies to variable environmental conditions & biotic effects

  • Persistence of pathogens for which there are no measures of viability

 
What are the mechanisms of removal by different drinking water treatment barriers ★★★★ Principles of pathogen reduction across a range of treatment barriers is well known 
  • Quantitative information regarding removal mechanisms is still limited

  • Hydrodynamic modeling is often missing to aid in estimating concentration.time (C.t) values to aid in estimating disinfection efficacy

 
Expected Log10 reduction by different treatment barriers ★ Many studies with surrogate organisms comparing inflow and outflow concentrations 
  • Quantitative translation of surrogate information to pathogens is limited. Translation of site specific results to other facilities is a challenge as most studies lack a mechanistic consideration of reduction values

  • Appropriate quantification of treatment performance (surrogate) based on microbial data

 
Expected causes of poor performance ★★ Principles of compromised performance for pathogens is well known 
  • Quantifying how poor the performance is under different conditions is not well known

  • Frequency and duration of periods of poor performance

 
How does microbial indicator presence and persistence through treatment relate to pathogens? ★★★ Principles are well known, and some quantitative data 
  • Presence of microbial indicators in finished water is often a challenge to explain

 
How does turbidity in finished water relate to filtration performance of pathogens? ★★ Some information on relationship between turbidity/particle counts and breakthrough of protozoa; some information of total organic carbon (TOC) and virus breakthough by membrane processes 
  • Improved information needed for all site-specific pathogens as related to inflow turbidity/particle count levels

  • Not well characterized for viruses, and may be a completely inappropriate indicator of virus removal performance, depending on process

 
What is the sensitivity of pathogens to disinfectants and the required Ct to achieve the necessary Log10 reduction ★★ Many studies in the literature on pathogen and surrogate reduction by various disinfectants in laboratory studies 
  • Quantifying the impact of pathogen shielding on reduction efficacy, somewhat limited too because of the pathogen assay method used (i.e. may miss VBNC and persistent forms (Wood et al. 2013))

  • Applicability of using laboratory strains to represent environmental strains is not well known

  • Inactivation of non-culturable organisms is not well known

 
Dose-response relationships for relevant pathogens ★ Human feeding studies and outbreak data available for most reference pathogens of interest 
  • Some studies have very limited number of datapoints at relatively high doses

  • Studies often only represent healthy adult volunteers – not more vulnerable groups

  • Studies only represent the specific strain used in the feeding study, unclear how representative

 
Conditions for growth of environmental pathogens ★ Warm waters, periods of stagnant flow and minimal disinfectant residual known to be important 
  • Specific effects of upstream microbiome and pipe materials on growth of human infectious Legionella and non-tuberculous mycobacteria in premise plumbing

 
Health impact assumptions and DALY weightings ★★ DALY weightings have been developed for most pathogens of interest 
  • Site specific weighting information is needed (likelihood of different disease outcomes for the local setting)

 
Risk based definitions of water safety (tolerable risk targets) ★★★ Tolerable risk targets have been suggested by WHO (1 × μDALY. Person–1.year–1) and US-EPA (1 × 10–4 annual probability of infection) 
  • Context specific targets as the suggested targets may be too low in some circumstances to give balance to all pathways

 
Fundamental research question Current state of knowledgea Knowledge gaps 
What are the sources/hosts of pathogens in the drinking water catchment? ★★★★ Sources of human enteric pathogens within drinking water catchments are well characterized 
  • Emerging zoonotic pathogens

 
How many enteric pathogens are excreted by those sources/hosts? ★★★ The excretion density of various pathogens in different hosts is known within a few orders of magnitude 
  • The magnitude of asymptomatic excretion of some pathogens by humans

  • The variability in excretion over the course of an infection

 
How many of those enteric pathogens are human infectious? ★ Molecular methods enable sub-typing of samples to identify specific host links for zoonotic pathogens 
  • Still many unknowns with regard to the proportion of protozoa that are human infectious

 
How many of those pathogens are viable (detectable) and able to cause infection? ★ For culturable organisms the lower boundary of detectible organisms is well known, but not those viable but non-culturable (VBNC) 
  • Viability of organisms detected using modified molecular methods

  • The proportion of organisms that are VBNC

 
How well do pathogens persist in the environment ★★ Many studies under different specific conditions for a range of pathogens 
  • Translation of specific studies to variable environmental conditions & biotic effects

  • Persistence of pathogens for which there are no measures of viability

 
What are the mechanisms of removal by different drinking water treatment barriers ★★★★ Principles of pathogen reduction across a range of treatment barriers is well known 
  • Quantitative information regarding removal mechanisms is still limited

  • Hydrodynamic modeling is often missing to aid in estimating concentration.time (C.t) values to aid in estimating disinfection efficacy

 
Expected Log10 reduction by different treatment barriers ★ Many studies with surrogate organisms comparing inflow and outflow concentrations 
  • Quantitative translation of surrogate information to pathogens is limited. Translation of site specific results to other facilities is a challenge as most studies lack a mechanistic consideration of reduction values

  • Appropriate quantification of treatment performance (surrogate) based on microbial data

 
Expected causes of poor performance ★★ Principles of compromised performance for pathogens is well known 
  • Quantifying how poor the performance is under different conditions is not well known

  • Frequency and duration of periods of poor performance

 
How does microbial indicator presence and persistence through treatment relate to pathogens? ★★★ Principles are well known, and some quantitative data 
  • Presence of microbial indicators in finished water is often a challenge to explain

 
How does turbidity in finished water relate to filtration performance of pathogens? ★★ Some information on relationship between turbidity/particle counts and breakthrough of protozoa; some information of total organic carbon (TOC) and virus breakthough by membrane processes 
  • Improved information needed for all site-specific pathogens as related to inflow turbidity/particle count levels

  • Not well characterized for viruses, and may be a completely inappropriate indicator of virus removal performance, depending on process

 
What is the sensitivity of pathogens to disinfectants and the required Ct to achieve the necessary Log10 reduction ★★ Many studies in the literature on pathogen and surrogate reduction by various disinfectants in laboratory studies 
  • Quantifying the impact of pathogen shielding on reduction efficacy, somewhat limited too because of the pathogen assay method used (i.e. may miss VBNC and persistent forms (Wood et al. 2013))

  • Applicability of using laboratory strains to represent environmental strains is not well known

  • Inactivation of non-culturable organisms is not well known

 
Dose-response relationships for relevant pathogens ★ Human feeding studies and outbreak data available for most reference pathogens of interest 
  • Some studies have very limited number of datapoints at relatively high doses

  • Studies often only represent healthy adult volunteers – not more vulnerable groups

  • Studies only represent the specific strain used in the feeding study, unclear how representative

 
Conditions for growth of environmental pathogens ★ Warm waters, periods of stagnant flow and minimal disinfectant residual known to be important 
  • Specific effects of upstream microbiome and pipe materials on growth of human infectious Legionella and non-tuberculous mycobacteria in premise plumbing

 
Health impact assumptions and DALY weightings ★★ DALY weightings have been developed for most pathogens of interest 
  • Site specific weighting information is needed (likelihood of different disease outcomes for the local setting)

 
Risk based definitions of water safety (tolerable risk targets) ★★★ Tolerable risk targets have been suggested by WHO (1 × μDALY. Person–1.year–1) and US-EPA (1 × 10–4 annual probability of infection) 
  • Context specific targets as the suggested targets may be too low in some circumstances to give balance to all pathways

 

aStars provide a subjective indication of the relative level of certainty associated with each fundamental question as it relates to the representivity of risk estimates from QMRA. Four stars indicates that this aspect is well characterized and has a limited impact on the representivity in the risk outcomes; 1 star indicates that this aspect is highly uncertainty and likely to have a high impact on the representivity of the risk outcomes.

VBNC, viable but non-culturable cells.

Secondly, QMRA calculations have historically been in the hands of specialists and researchers. The apparent complexity of the calculations has created a barrier for more widespread accessibility to the water industry. Various tools have been developed to increase accessibility of the QMRA approach to non-specialists. A range of different software platforms have been used to undertake QMRA calculations including specific Monte Carlo simulation tools (e.g. @Risk®, Crystal Ball® and Analytica®) and more generic mathematical software (R, Mathematica®, Matlab® and Excel®). The following recommendations relate to the development and application of QMRA software tools:

  1. There is currently no single QMRA tool that can meet all of the needs of the water industry. Tools need to be developed and created for specific purposes. Examples of these have been identified in this review.

  2. User friendly interfaces are valuable, however basic training in QMRA and understanding the underlying model assumptions is necessary to support tool implementation. Given the uncertainties that have been presented, it is important that anyone applying QMRA (even with a simple online tool) has an understanding of the framework and the objectives of the calculations including any limitations to applying the results.

  3. All underlying assumptions need to be accessible and transparent to the user so that the above requirements of uncertainty can be verified. There is limited value associated with a ‘black box’ QMRA tool, as the greatest value is not in the result alone, but in understanding the interactions between the model parameters and relative risk estimates – so you get to know your system.

Thirdly, the cases reviewed in this study varied greatly in their level of mathematical detail. Some studies used point values and others contained detailed statistical modelling and uncertainty analysis. The more complex studies are not necessarily better, in fact, a QMRA should be as simple as possible in order to achieve the required outcomes of the analysis. Following the tiered approach presented previously (Medema & Smeets 2009), the complexity of an assessment only needs to be increased if it is required to inform management of the risk. This tiered approach can be extended for all of the different objectives presented in this paper. The most simple, conservative approach should be applied first, and then additional detail undertaken only as necessary to achieve the required outcome.

Fourthly, an important purpose of QMRA that underpins all potential applications is as a support for risk communication. Management of risk requires interaction between a diverse range of stakeholders, and the implications of scientific knowledge can be challenging to communicate. Simple outputs from QMRA such as: the reduction in risk due to interventions; or selection of one intervention over another; demonstration of a process vulnerability; or identification that a current practice is unsafe, are of great value for communication to the relevant stakeholders.

In practice, whilst the QMRA methodology is fundamentally robust, its reliability is limited by the quality of supporting data and evidence. Critics of QMRA often focus on these uncertainties. However, it should be recognized that often QMRA does provide the best-supported estimate of risk. Furthermore, QMRA permits explicit identification and quantification of uncertainties. This provides a good platform for starting with a best-supported evidence-based estimate of risk whilst highlighting research and data collection needs.

ACKNOWLEDGEMENTS

The authors wish to acknowledge funding from Alberta Environment and Sustainable Resource Development (now Environment and Parks, Alberta Government) under contract 140199 for SRP and Alberta Innovates – Health Solutions for funding NJA.

REFERENCES

REFERENCES
Anonymous
2001
Waterleidingbesluit 2001
.
Staatsblad van het Koninkrijk der Nederlanden.
31
,
1
53
.
Anonymous
2005
VROM-Inspectierichtlijn Analyse microbiologische veiligheid drinkwater. VROM-Inspectie. Artikelcode 5318. Available from
: .
Ashbolt
N. J.
Petterson
S. A.
Stenström
T. A.
Schönning
C.
Westrell
T.
Ottoson
J.
2005
Microbial Risk Assessment (MRA) Tool. Urban Water Report 2005: 7
.
Chalmers University of Technology
,
Gothenberg
.
Åström
J.
Petterson
S.
Bergstedt
O.
Pettersson
T. J.
Stenstrom
T. A.
2007
Evaluation of the microbial risk reduction due to selective closure of the raw water intake before drinking water treatment
.
J. Water Health
5
(
Suppl 1
),
81
97
.
Åström
J.
Pettersson
T. J. R.
Reischer
G. H.
Hermansson
M.
2013
Short-term microbial release during rain events from on-site sewers and cattle in a surface water source
.
J. Water Health
11
(
3
),
430
442
.
Beer
K. D.
Gargano
J. W.
Roberts
V. A.
Hill
V. R.
Garrison
L. E.
Kutty
P. K.
Hilborn
E. D.
Wade
T. J.
Fullerton
K. E.
Yoder
J. S.
2015
Surveillance for waterborne disease outbreaks associated with drinking water – United States, 2011–2012
.
Morb. Mortal. Wkly. Rep.
64
,
842
848
.
Collier
S. A. S.
Hicks
L. A.
Garrison
L. E.
Zhou
F. J.
Beach
M. J.
2012
Direct healthcare costs of selected diseases primarily or partially transmitted by water
.
Epidemiol. Infect.
140
(
11
),
2003
2013
.
Craun
G. F. B.
Yoder
J. S.
Roberts
V. A.
Carpenter
J.
Wade
T.
Calderon
R. L.
Roberts
J. M.
Beach
M. J.
Roy
S. L.
2010
Causes of outbreaks associated with drinking water in the United States from 1971 to 2006
.
Clin. Microbiol. Rev.
23
(
3
),
207
228
.
Cummins
E.
Kennedy
R.
Cormican
M.
2010
Quantitative risk assessment of Cryptosporidium in tap water in Ireland
.
Sci. Total Environ.
408
(
4
),
740
753
.
Fewtrell
L.
Bartram
J.
2001
Water Quality: Guidelines, Standards and Health. Risk Assessment and Management for Water Related Infectious Diseases
.
IWA Publishing
,
London
.
Harder
R.
Peters
G. M.
Molander
S.
Ashbolt
N. J.
Svanström
M.
2015
Including pathogen risk in life cycle assessment: the effect of modelling choices in the context of sewage sludge management
.
Int. J. Life Cycle Assess.
21
(
1
),
60
69
.
Havelaar
A. H.
De Hollander
A. E.
Teunis
P. F.
Evers
E. G.
Van Kranen
H. J.
Versteegh
J. F.
Van Koten
J. E.
Slob
W.
2000
Balancing the risks and benefits of drinking water disinfection: disability adjusted life-years on the scale
.
Environ. Health Perspect.
108
(
4
),
315
321
.
Health Canada
2012
Guidelines for Canadian Drinking Water Quality: Guideline Technical Document – Enteric Protozoa: Giardia and Cryptosporidium. Available from
:
www.healthcanada.gc.ca (accessed 8 March 2016)
.
Hellard
M.
Sinclair
M. I.
Forbes
A. B.
Fairley
C. K.
2001
A randomized blinded controlled trial investigating the gastrointestinal health effects of drinking water quality
.
Environ. Health Perspect.
109
(
8
),
773
778
.
Howard
G.
Pedley
S.
Tibatemwa
S.
2006
Quantitative microbial risk assessment to estimate health risks attributable to water supply: can the technique be applied in developing countries with limited data?
J. Water Health
4
(
1
),
49
65
.
Hrudey
S. E.
Hrudey
E. J.
2004
Safe Drinking Water. Lessons from Recent Outbreaks in Affluent Nations
.
IWA Publishing
,
London
.
Hrudey
S. E.
Hrudey
E. J.
2014
Ensuring Safe Drinking Water – Learning from First Hand Experience with Contamination
.
American Water Works Association
,
Denver CO
.
Hrudey
S. E.
Hrudey
E. J.
Pollard
S. J.
2006
Risk management for assuring safe drinking water
.
Environ. Int.
32
(
8
),
948
957
.
Hunter
P. R.
Anderle de Sylor
M.
Risebro
H. L.
Nichols
G. L.
Kay
D.
Hartemann
P.
2011
Quantitative microbial risk assessment of cryptosporidiosis and giardiasis from very small water supplies
.
Risk Anal.
31
(
2
),
228
236
.
Jaidi
K.
Barbeau
B.
Carriere
A.
Desjardins
R.
Prevost
M.
2009
Including operational data in QMRA model: development and impact of model inputs
.
J. Water Health
7
(
1
),
77
95
.
Kobayashi
Y.
Peters
G. M.
Ashbolt
N. J.
Shiels
S.
Khan
S. J.
2015
Assessing burden of disease as disability adjusted life years in life cycle assessment
.
Sci. Total Environ.
530–531C
,
120
128
.
Loret
J. F.
Cossalter
L.
Robert
S.
Baudin
I.
Conan
M.
Charles
P.
2013
Assessment and management of health risks related to the recycling of filter backwash water in drinking water production
.
Water Pract. Technol.
8
(
2
),
166
179
.
Lundie
S.
Ashbolt
N.
Peters
G.
Livingston
D.
Lai
E.
Karman
E.
Blaikie
J.
Anderson
J.
2008
Sustainability Framework WSAA Occasional Paper No. 17 PART A: Methodology for Evaluating the Overall Sustainability of Urban Water Systems. PART B: A Review Comparing the WSAA Sustainability Framework to the Gold Coast Waterfuture Process
.
Report of the Water Services Association of Australia
,
Melbourne
.
Masago
Y.
Katayama
H.
Hashimoto
A.
Hirata
T.
Ohgaki
S.
2002
Assessment of risk of infection due to Cryptosporidium parvum in drinking water
.
Water Sci. Technol.
46
(
11–12
),
319
324
.
McBride
G. B.
Stott
R.
Papps
D.
Palliser
C.
Jenner
G.
Macdonald
G.
2012
Estimating health risks to water users: marrying hydrodynamic models and risk models
.
Water Pract. Technol.
7
(
4
),
doi: 10.2166/wpt.2012.078
.
Medema
G. J.
Hoogenboezem
W.
van der Veer
A. J.
Ketelaars
H. A. M.
Hijnen
W. A. M.
Nobel
P. J.
2003
Quantitative risk assessment of Cryptosporidium in surface water treatment systems
.
Water Sci. Technol.
47
(
3
),
241
247
.
Medema
G.
Loret
J.-C.
Stenström
T. A.
Ashbolt
N.
2006
Quantitative Microbial Risk Assessment in the Water Safety Plan. Final Report on the EU MicroRisk Project
.
European Commission
,
Brussels
.
Mena
K. D.
Gerba
C. P.
Haas
C. N.
Rose
J. B.
2003
Risk assessment of waterborne coxsackievirus
.
J. Am. Water Works Assoc.
95
(
7
),
122
131
.
Nieuwenhuijsen
M. J.
Smith
R.
Golfinopoulos
S.
Best
N.
Bennett
J.
Aggazzotti
G.
Righi
E.
Fantuzzi
G.
Bucchini
L.
Cordier
S.
Villanueva
C. M.
Moreno
V.
La Vecchia
C.
Bosetti
C.
Vartiainen
T.
Rautiu
R.
Toledano
M.
Iszatt
N.
Grazuleviciene
R.
Kogevinas
M.
2009
Health impacts of long-term exposure to disinfection by-products in drinking water in Europe: HIWATE
.
J. Water Health
7
(
2
),
185
207
.
Nilsson
P.
Roser
D.
Thorwaldsdotter
R.
Petterson
S.
Davies
C.
Signor
R.
Bergstedt
O.
Ashbolt
N.
2007
SCADA Data and the quantification of hazardous events for QMRA
.
J. Water Health
5
(
Suppl 1
),
99
105
.
NWQMS
2006
Guidelines for Water Recycling: Managing Health and Environmental Risks (Phase 1) – November 2006, National Water Quality Management Strategy Report of the Natural Resource Management Ministerial Council
.
Environment Protection and Heritage Council, Australian Health Ministers’ Conference
,
Canberra
.
O'Halloran
K. R.
Ongerth
J. E.
Ashbolt
N. J.
Waite
T. D.
1999
Assessment of On-Line Particle Counters for Routine Control of Microbial Pathogens at Water Treatment Plants
.
Urban Water Research Association of Australia
,
Melbourne
.
Petterson
S.
Grellier
J.
Stenström
T.
Meriläinen
P.
Bucchini
L.
Nieuwenhuijsen
M.
2010
Development of a Software Tool for Undertaking Comparative Risk Assessment of DBPs and Pathogens in Drinking Water Final Report on Risk/Benefit Analysis for the EU HiWATE Project
.
Report of the European Commission
,
Brussels
.
Regli
S.
Rose
J. B.
Haas
C. N.
Gerba
C. P.
1991
Modeling the risk from Giardia and viruses in drinking water
.
J. Am. Water Works Assoc.
83
(
11
),
76
84
.
Rose
J. B.
Gerba
C. P.
1991
Use of risk assessment for development of microbial standards
.
Water Sci. Technol.
24
(
2
),
29
34
.
Rose
J. B.
Haas
C. N.
Regli
S.
1991
Risk assessment and control of waterborne giardiasis
.
Am. J. Public Health
81
,
709
713
.
Ryan
M.
Hamilton
K.
Hamilton
M.
Haas
C. N.
2014
Evaluating the potential for a Helicobacter pylori drinking water guideline
.
Risk Anal.
34
(
9
),
1651
1662
.
Schijven
J. F.
Teunis
P. F.
Rutjes
S. A.
Bouwknegt
M.
Husman
A. M.
2011
QMRAspot: a tool for quantitative microbial risk assessment from surface water to potable water
.
Water Res.
45
(
17
),
5564
5576
.
Schmidt
P. J.
Pintar
K. D.
Fazil
A. M.
Flemming
C. A.
Lanthier
M.
Laprade
N.
Sunohara
M. D.
Simhon
A.
Thomas
J. L.
Topp
E.
Wilkes
G.
Lapen
D. R.
2013b
Using Campylobacter spp. and Escherichia coli data and Bayesian microbial risk assessment to examine public health risks in agricultural watersheds under tile drainage management
.
Water Res.
47
(
10
),
3255
3272
.
Schoen
M. E.
Soller
J. A.
Ashbolt
N. J.
2011
Evaluating the importance of faecal sources in human-impacted waters
.
Water Res.
45
(
8
),
2670
2680
.
Schoen
M. E.
Xue
X.
Hawkins
T. R.
Ashbolt
N. J.
2014
Comparative human health risk analysis of coastal community water and waste service options
.
Environ. Sci. Technol.
48
,
9728
9736
.
Smeets
P. W.
Rietveld
L. C.
van Dijk
J. C.
Medema
G. J.
2010
Practical applications of quantitative microbial risk assessment (QMRA) for water safety plans
.
Water Sci. Technol.
61
(
6
),
1561
1568
.
Sokolova
E.
Petterson
S. R.
Dienus
O.
Nyström
F.
Lindgren
P.-E.
Pettersson
T. J.
2015
Microbial risk assessment of drinking water based on hydrodynamic modelling of pathogen concentrations in source water
.
Sci. Total Environ.
526
,
177
186
.
Steyn
M.
Jagals
P.
Genthe
B.
2004
Assessment of microbial infection risks posed by ingestion of water during domestic water use and full-contact recreation in a mid-southern African region
.
Water Sci. Technol.
50
(
1
),
301
308
.
Svenskt Vatten
2015
Mikrobiologisk Risk Analys (MRA) – riskanalys för dricksvatten. Available from
: .
Teunis
P. F. M.
Ever
E. G.
Slob
W.
1999
Analysis of variable fractions resulting from microbial counts
.
Quant. Microbiol.
1
,
63
88
.
Teunis
P. F.
Xu
M.
Fleming
K. K.
Yang
J.
Moe
C. L.
Lechevallier
M. W.
2010
Enteric virus infection risk from intrusion of sewage into a drinking water distribution network
.
Environ. Sci. Technol.
44
(
22
),
8561
8566
.
Tfaily
R.
Papineau
I.
Andrews
R.
Barbeau
B.
2015
Application of quantitative microbial risk assessment at 17 Canadian water treatment facilities
.
J. Am. Water Works Assoc.
107
(
10
),
E479
E508
.
USDA/US-EPA
2012
Microbial Risk Assesssment Guideline: Pathogenic Microorganisms with Focus on Food and Water
.
Report of the Interagency Microbiological Risk Assessment Guideline Workgroup
, pp.
207
.
US-EPA
2001
National primary drinking water regulations: filter backwash recycling rule; final rule
.
Federal Register
66
(
111
),
31086
31105
.
US-EPA
2003a
LT1ESWTR Disinfection Profiling and Benchmarking Technical Guidance Manual. Report of the Office of Water
.
Report of the US-EPA
,
Washington, DC
.
US-EPA
2003b
LT2ESWTR Long Term Second Enhanced Surface Water Treatment Rule and Draft Toolbox Guidance Manual
.
Report of the US-EPA
,
Washington, DC
.
US-EPA
2006
National primary drinking water regulations: long term 2 enhanced surface water treatment rule
.
Federal Register
71
(
3
),
654
702
.
US-EPA
2007
Thesaurus Of Terms Used In Microbiological Risk Assessment
.
Report of the Office of Water, United States Environmental Protection Agency
,
Washington, DC
.
US-EPA
2012
Recreational Water Quality Criteria
.
Report of the US Environmental Protection Agency
,
Washington, DC
.
van den Akker
B.
Whiffin
V.
Cox
P.
Beatson
P.
Ashbolt
N. J.
Roser
D. J.
2011
Estimating the risk from sewage treatment plant effluent in the Sydney catchment area
.
Water Sci. Technol.
63
(
8
),
1707
1715
.
van Heerden
J.
Ehlers
M. M.
Vivier
J. C.
Grabow
W. O.
2005
Risk assessment of adenoviruses detected in treated drinking water and recreational water
.
J. Appl. Microbiol.
99
(
4
),
926
933
.
Westrell
T.
Bergstedt
O.
Stenström
T. A.
Ashbolt
N. J.
2003
A theoretical approach to assess microbial risks due to failures in drinking water treatment
.
Int. J. Environ. Health Res.
13
(
2
),
181
197
.
WHO
2004
Guidelines for Drinking Water Quality
,
3rd edn.Report of the World Health Organization
,
Geneva
.
WHO
2006a
Guidelines for the Safe use of Wastewater, Excreta and Greywater, Volume 2: Wastewater use in Agriculture
.
Report of the World Health Organization
,
Geneva
.
WHO
2006b
Guidelines for the Safe use of Wastewater, Excreta and Greywater, Volume 4: Excreta and Greywater use in Agriculture
.
Report of the World Health Organization
,
Geneva
.
WHO
2009
Water Safety Plan Manual: Step-by-Step Risk Management for Drinking-Water Suppliers
.
Report of the World Health Organization
,
Geneva
.
WHO
2011a
Evaluating Household Water Treatment Options: Health-Based Targets and Microbiological Performance Specifications
.
Report of the World Health Organization
,
Geneva
.
WHO
2011b
Guidelines for Drinking-Water Quality Fourth Edition
.
Report of the World Health Organization
,
Geneva
.
WHO
2016
Quantitative Microbial Risk Assessment: application for water safety management
.
World Health Organization
,
Geneva
.
Wood
T. K.
Knabel
S. J.
Kwan
B. W.
2013
Bacterial persister cell formation and dormancy
.
Appl. Environ. Microbiol.
79
(
23
),
7116
7121
.
Zhang
K.
Achari
G.
Sadiq
R.
Langford
C.
Dore
M.
2012
An integrated performance assessment framework for water treatment plants
.
Water Res.
46
,
1673
1683
.