Human noroviruses are a leading cause of food- and water-borne disease, which has led to an interest in quantifying norovirus health risks using quantitative microbial risk assessment (QMRA). Given the limited availability of quantitative norovirus data to input to QMRA models, some studies have applied a conversion factor to estimate norovirus exposure based on measured fecal indicator bacteria (FIB) concentrations. We conducted a review of peer-reviewed publications to identify the concentrations of noroviruses and FIB in raw, secondary-treated, and disinfected wastewater. A meta-analysis was performed to determine the ratios of norovirus-FIB pairs in each wastewater matrix and the variables that significantly impact these ratios. Norovirus-to-FIB ratios were found to be significantly impacted by the norovirus genotype, month of sample collection, geographic location, and the extent of wastewater treatment. Additionally, we evaluated the impact of using a FIB-to-virus conversion factor in QMRA and found that the choice of conversion ratio has a great impact on estimated health risks. For example, the use of a conversion ratio previously used in the World Health Organization Guidelines for the Safe Use of Wastewater, Excreta and Greywater predicted health risks that were significantly lower than those estimated with measured norovirus concentrations used as inputs. This work emphasizes the gold standard of using measured pathogen concentrations directly as inputs to exposure assessment in QMRA. While not encouraged, if one must use a FIB-to-virus conversion ratio to estimate norovirus dose, the ratio should be chosen carefully based on the target microorganisms (i.e., strain, genotype, or class), prevalence of disease, and extent of wastewater treatment.

  • It is inappropriate to use a static ratio to estimate norovirus concentrations based on measured fecal indicator bacteria (FIB) abundances in all cases.

  • Calculated ratios between measured norovirus and FIB differed depending on the virus genotype and FIB class considered. Ratios were significantly influenced by the extent of wastewater treatment.

  • The gold standard is to use measured pathogen concentrations directly as inputs to exposure assessment in quantitative microbial risk assessment.

Human noroviruses are a leading etiology of acute gastroenteritis in patients of all age groups (Patel et al. 2009; Scallan et al. 2011; Hall et al. 2013) and have been hypothesized to account for approximately 50% of outbreaks of gastroenteritis around the world (Patel et al. 2009). Noroviruses are transmitted by the fecal-oral route, and exposure can include ingestion of sewage-influenced food and water, such as recreational waters, crops irrigated with treated or untreated wastewater, or foods contaminated post-harvest during processing or preparation (Patel et al. 2009). In the United States, for example, an estimated 58% of foodborne illnesses are attributed to norovirus infections (Scallan et al. 2011).

Given the importance of noroviruses as etiologies of disease, there has been an effort to quantify norovirus health risks due to different exposure scenarios using quantitative microbial risk assessment (QMRA) (Barker et al. 2013; Mok et al. 2014; Sales-Ortells et al. 2015; Seto et al. 2018; Gonzales-Gustavson et al. 2019). QMRA is a mathematical modeling tool that can be used to predict health risks caused by environmental exposure to a specified pathogen and to evaluate the ability of control strategies and policy recommendations to mitigate those risks (e.g., WHO 2006).

A common challenge in conducting QMRA is the limited availability of high-quality, relevant, and timely data for some model inputs (Gardner 2004), including the abundance of the target pathogen in a source water (Mok et al. 2014; Owusu-Ansah et al. 2017). Human virus concentrations are not commonly measured in water sources due to the technical complexity of virus quantification assays and subsequent cost constraints. This is especially true in resource-constrained areas, such as low-income countries. Therefore, the microbial quality of recreational waters and wastewater is typically monitored through the measurement of fecal indicator bacteria (FIB), such as Escherichia coli, and the bacterial classes of total coliform and fecal coliform (FC; USEPA 2000; NYSDEC 2014). This presents a challenge for conducting QMRA for viral pathogens based on previously collected water quality data. To overcome this challenge, some studies have applied a conversion factor to estimate the concentration of the pathogen of interest based on measured FIB concentrations.

An example of this is presented in the World Health Organization (WHO) Guidelines for the Safe Use of Wastewater, Excreta and Greywater (WHO 2006), which advocated for governments to employ a QMRA approach when developing national regulations for the use of treated wastewater in agricultural irrigation. The WHO guidelines provided an example of estimating rotavirus health risks due to the consumption of wastewater-irrigated lettuce and onions, in which an assumed ratio of between 0.1 and 1 rotavirus per 105E. coli was used as an input (i.e., 10−6–10−5 rotaviruses per E. coli). Subsequent studies have adopted this QMRA approach for wastewater irrigation and used one of the following conversion factors to estimate norovirus or rotavirus abundance in irrigation water based on measured FIB concentrations: (i) a static ratio of 10−5 viruses per FIB (Shuval et al. 1997; Mara et al. 2007; Seidu et al. 2008; Machdar et al. 2013; Kundu et al. 2018); (ii) a uniform distribution between 10−6 and 10−5 viruses per FIB (Mara & Sleigh 2010; Pavione et al. 2013); or (iii) a PERT distribution of (10−6, 10−5.3, 10−5) (Fuhrimann et al. 2016, 2017).

The use of a static ratio to convert measured FIB abundances to human virus concentrations is controversial, and correlations between pathogen and FIB concentrations may not be accurate across water sources. Some studies have found a limited correlation between concentrations of FIB and enteric viruses in wastewater (Haramoto et al. 2008; He et al. 2012; Petterson et al. 2016), given that the occurrence of viral pathogens in wastewater is a function of disease incidence and is therefore variable, whereas FIB are more consistently excreted by the population (Ottoson et al. 2006). Moreover, the FIB-to-virus conversion ratios previously used in QMRA may not reflect significant geographical or seasonal variations observed for noroviruses (Eftim et al. 2017). For example, several studies found higher norovirus concentrations in raw wastewater in the winter than in the summer (Nordgren et al. 2009; Montazeri et al. 2015; Eftim et al. 2017). FIB concentrations are less likely to have temporal variation (Haramoto et al. 2006, 2008; Flannery et al. 2012; Montazeri et al. 2015), which would cause the ratio between norovirus and FIB concentrations (RNoV:FIB) to vary across seasons.

RNoV:FIB can also be affected by the specific norovirus genotype or FIB type considered. For example, the predominant norovirus genotypes that infect humans (i.e., norovirus (NoV) GI and NoV GII) are present in wastewater at different concentrations. Namely, NoV GII infections are more prevalent (Siebenga et al. 2009), and NoV GII have been observed at higher densities in wastewater than NoV GI (Eftim et al. 2017). Furthermore, E. coli and FC are subcategorizations of the class of total coliform and are therefore typically quantified at lower concentrations than total coliform in wastewater and environmental waters. Therefore, a static RNoV:FIB would not be accurate for all norovirus genotype and FIB combinations.

An additional factor that influences the concentrations of enteric viruses and FIB measured in sewage-influenced waters is the degree of wastewater treatment. Viruses and bacteria differ in their size and composition and subsequently have different disinfection and removal efficiencies when exposed to treatment processes. Katayama et al. (2008), for example, found FC to be removed to a greater extent than noroviruses during wastewater treatment (noroviruses were measured by reverse transcription-quantitative polymerase chain reaction (RT-QPCR)), and Tree et al. (2005) found feline calicivirus (a culturable surrogate for human norovirus) to be inactivated by chlorine at slower rates than E. coli. Noroviruses have also been observed to be persistent in the aquatic environment (Boehm et al. 2019). Therefore, it is hypothesized that RNoV:FIB would vary as a function of the extent of wastewater treatment.

The findings discussed above suggest that the relative abundances of noroviruses and FIB (and therefore RNoV:FIB) would depend on (i) the location and season that water samples were collected, (ii) the FIB used for water quality monitoring, and (iii) the degree of wastewater treatment. Subsequently, true values of RNoV:FIB may fall outside of the range used by previous studies to estimate norovirus concentrations based on measured FIB. This discrepancy could lead to under- or overestimation of virus concentrations input to QMRA models, and thus a poor prediction of disease burden. To evaluate the impact of assumed RNoV:FIB on health risks estimated by QMRA, the objectives of this study were to (1) perform a systematic review and meta-analysis of observed RNoV:FIB in wastewater from the literature and identify factors that significantly influence these values and (2) conduct QMRA using different values of RNoV:FIB and compare to health risks estimated by QMRA conducted using measured norovirus concentrations directly. Clearly, the gold standard for conducting QMRA is to use actual measured pathogen concentrations as inputs. Therefore, we are not promoting the use of a FIB-to-NoV conversion ratio, but rather assessing the impact of its use on estimated health risks.

Systematic review and meta-analysis

A systematic review of the peer-reviewed literature was conducted to obtain data on the concentrations of noroviruses and commonly used FIB (i.e., E. coli, total coliform, or FC) in wastewater collected at various stages of a conventional treatment system. As a starting point, the systematic reviews of norovirus concentrations in wastewater conducted by Eftim et al. (2017) and Pouillot et al. (2015) were used to identify papers to include in the present systematic review, given that the previous studies used search terms and inclusion criteria that were similar to the present study (described below). Five papers from the previous reviews met our inclusion criteria: Flannery et al. (2012), Katayama et al. (2008), Kauppinen et al. (2014), Montazeri et al. (2015), and Sima et al. (2011). The Eftim et al. (2017) and Pouillot et al. (2015) reviews included articles that were published up until 2015 and 2013, respectively. Therefore, we conducted an additional literature search to identify articles published between January 2015 and June 2019, when the search was conducted. The following terms were used to search the Web of Science core collection, Scopus, and PubMed databases: ‘(norovirus OR Norwalk) AND (sewage OR influent OR effluent OR wastewater) AND (E. coli OR coliform OR Escherichia).’

The inclusion criteria applied to the identified publications were the following. (1) Articles were primary research articles (i.e., not review articles), published in English in peer-reviewed journals. (2) Article scope: articles included measured concentrations of NoV GI or NoV GII and a FIB (i.e., E. coli, total coliform, or FC) in the same sample, quantified using clearly described and justifiable methods; included sample types were (i) raw municipal wastewater, (ii) conventional secondary-treated municipal wastewater (i.e., after biological treatment, such as activated sludge, but before disinfection), and (iii) disinfected final effluent (i.e., after chlorination). (3) Assay type: included studies used RT-QPCR or most probable number RT-PCR for quantifying noroviruses and an appropriate culture-based assay for quantifying FIB; RT-QPCR assays included recovery controls and methods to assess or mitigate PCR inhibition. (4) Data availability: articles reported norovirus and FIB concentrations in the form of measured individual points, means, geometric means, or medians. To be included, norovirus concentrations were required to be for a single month (i.e., means calculated over multiple months were not included), whereas FIB concentrations could be presented as an average over time, given relatively consistent FIB concentrations over the course of the year.

After duplicate entries were removed, the literature search resulted in 76 articles, of which one – Worley-Morse et al. (2019) – met the inclusion criteria. For the included papers, measured concentrations of noroviruses and FIB were extracted. If data were provided in a table, they were extracted directly. If data were reported using a figure, then the individual points were extracted using a plot digitizer (WebPlotDigitizer, v4.2; https://automeris.io/WebPlotDigitizer/). Samples that had concentrations of noroviruses or FIB that were below the detection limit were not included in subsequent analyses. Treated wastewater samples collected from treatment systems employing membrane bioreactors were not included, given that the focus of this work was on conventional wastewater treatment systems.

Each individual sample with reported values for one norovirus genotype and one FIB was considered an individual data point. Additional data that were extracted from papers included the month of sample collection, the sample location within the wastewater treatment facility (e.g., raw wastewater, secondary effluent, and final disinfected effluent), the norovirus and FIB concentrations and quantification methods employed, and the geographic location (i.e., city and country) where the study was conducted. To ensure consistency of data extraction, 15% of the included articles were selected randomly for a second round of data extraction by a second reviewer and compared for consistency.

NoV-to-FIB ratios (RNoV:FIB) were calculated for each sample by dividing the reported norovirus concentration by the reported FIB concentration; each of these ratios constituted one data point. Each data point was then log-transformed to calculate log10RNoV:FIB. Mean, median, and standard deviation were calculated for log10RNoV:FIB values pooled for each of the following categories in each type of wastewater sample (i.e., raw, secondary, and disinfected wastewater): NoV GI: E. coli, NoV GII: E. coli, NoV GI:FC, and NoV GII:FC. No data points were identified for total coliform; therefore, total coliform data were not included in subsequent data analysis. Statistical analysis and figures were produced using GraphPad Prism v8.

Multiple linear regression was conducted to model log10RNoV:FIB as a function of (i) the norovirus genogroup (i.e., GI or GII), (ii) the type of FIB (i.e., E. coli or FC), (iii) the extent of wastewater treatment, (iv) the month of sample collection, and (v) the geographical location. This analysis was conducted to determine whether RNoV:FIB was significantly impacted by the above-mentioned variables. The analysis was performed in RStudio (v1.4.1717) by applying the ‘lm’ command using Equation (1), where is the intercept; xi are the dummy variables used to indicate the categorical data of the virus type (i.e., NoVGI (xNoVG1) and NoVGII (xNoVG2)), the FIB type (i.e., E. coli (xEC) and FC (xFC)), the extent of wastewater treatment (i.e., raw (xR), secondary (xS), and disinfected (xD)), and the geographical location (i.e., North America (xNA), Europe (xE), and Asia (xA)); is a numerical month variable (e.g., January = 1, December = 12); and are the associated coefficients.
formula
(1)

Quantitative microbial risk assessment

To evaluate the extent to which the use of different assumed RNoV:FIB affects disease risks estimated by QMRA, we considered the predicted risk of norovirus infection due to the consumption of wastewater-irrigated lettuce. Two hypothetical scenarios were considered: (1) irrigation with a surface water source contaminated with untreated wastewater and (2) irrigation with undiluted, disinfected wastewater effluent. For each scenario, three strategies were used to estimate the norovirus concentration present in irrigation water. The first strategy utilized the actual measured concentration of norovirus in the water source as an input for exposure assessment. The second strategy utilized measured E. coli concentrations in the water source and applied the conversion ratio previously used by the WHO Guidelines for the Safe Use of Wastewater, Excreta and Greywater (RW = 10−6–10−5 viruses per E. coli) to calculate an estimated norovirus concentration; a uniform distribution of values within this range was utilized. The third strategy also utilized measured E. coli concentrations in the water source but applied one of the RNoV:FIB determined in the present study.

Exposure model

Norovirus doses were estimated based on NoV GII concentrations, given that NoV GII is often measured at higher concentrations than NoV GI in surface waters (Van Abel et al. 2017) and wastewater (Haramoto et al. 2006; Montazeri et al. 2015; Eftim et al. 2017), and there is a higher prevalence of NoV GII infection than NoV GI (Xue et al. 2015). For each exposure scenario, the daily dose (d; viruses/person/day) of NoV GII ingested by a consumer via the consumption of wastewater-irrigated lettuce was calculated following Equation (2), where CNoV is the norovirus concentration in the treated or untreated wastewater (virus/mL); V is the volume of water that clings to the lettuce surface after irrigation (mL/g); Dw is the log10 NoV reduction during regular washing practices; Dd is the log10 reduction of NoV due to dilution of wastewater into surface water; and I is the daily ingestion of lettuce (g/person/day). The distributions of input parameters are summarized in Table 1; the decay of noroviruses due to field conditions and storage was not accounted for in this study.
formula
(2)
Table 1

Summary of input parameters for the QMRA model

VariableParameter nameUnitsValue or distributionReferences
Exposure parameters 
V Volume of water retained on lettuce after irrigation mL/g Normal (0.108,0.019), truncated at zero Barker et al. (2013); Hamilton et al. (2006); Mok et al. (2014); Owusu-Ansah et al. (2017)  
I Daily consumption of lettuce g/day/person Uniform (10,20) Fung et al. (2011); Owusu-Ansah et al. (2017); Seidu et al. (2008)  
Dw Log virus reduction due to washing practices Unitless PERT (0.1,1,2) Baert et al. (2009); Barker et al. (2013); Mok et al. (2014); Owusu-Ansah et al. (2017); Seidu et al. (2008)  
Dd Log virus reduction due to dilution Unitless Scenario 1: 1.48; Scenario 2: 0 McBride et al. (2013)  
RW Previously used FIB-to-NoV conversion ratio Unitless Uniform (10−6,10−5WHO (2006)  
RNoV:FIB FIB-to-NoV GII conversion ratio from the present study Unitless For log10RNoV:FIB – raw wastewater: normal (−1.5,1.4); disinfected wastewater: normal (0.51,2.3) This paper 
Dose-response model parameters 
d Daily virus dose Viruses/person/day Equation (2)  
Pinf,event Probability of infection per exposure event using the fractional Poisson model Unitless Equation (4) Messner et al. (2014)  
P Fraction of secretor-positive individuals Unitless 0.72 Messner et al. (2014)  
μ Mean virus aggregate size Unitless Messner et al. (2014)  
Pill|inf Probability of illness among those infected Unitless Equation (5) Teunis et al. (2008)  
η Scale parameter Unitless 0.00255 Teunis et al. (2008)  
r Shape parameter Unitless 0.086 Teunis et al. (2008)  
Pill,event Probability of illness per exposure event Unitless Equation (6) Teunis et al. (2008)  
Pill,ann Annual probability of illness Unitless Equation (7) Mok et al. (2014)  
n Number of exposures in 1 year Day Uniform (208,365)  
DALY 
 DALY Annual DALY form of NoV GII illness DALY/person/year Equation (8)  
DB Disease burden per case of illness DALY per case of illness Uniform (3.7 × 10−4, 6.2 ×10−3Mok et al. (2014)  
VariableParameter nameUnitsValue or distributionReferences
Exposure parameters 
V Volume of water retained on lettuce after irrigation mL/g Normal (0.108,0.019), truncated at zero Barker et al. (2013); Hamilton et al. (2006); Mok et al. (2014); Owusu-Ansah et al. (2017)  
I Daily consumption of lettuce g/day/person Uniform (10,20) Fung et al. (2011); Owusu-Ansah et al. (2017); Seidu et al. (2008)  
Dw Log virus reduction due to washing practices Unitless PERT (0.1,1,2) Baert et al. (2009); Barker et al. (2013); Mok et al. (2014); Owusu-Ansah et al. (2017); Seidu et al. (2008)  
Dd Log virus reduction due to dilution Unitless Scenario 1: 1.48; Scenario 2: 0 McBride et al. (2013)  
RW Previously used FIB-to-NoV conversion ratio Unitless Uniform (10−6,10−5WHO (2006)  
RNoV:FIB FIB-to-NoV GII conversion ratio from the present study Unitless For log10RNoV:FIB – raw wastewater: normal (−1.5,1.4); disinfected wastewater: normal (0.51,2.3) This paper 
Dose-response model parameters 
d Daily virus dose Viruses/person/day Equation (2)  
Pinf,event Probability of infection per exposure event using the fractional Poisson model Unitless Equation (4) Messner et al. (2014)  
P Fraction of secretor-positive individuals Unitless 0.72 Messner et al. (2014)  
μ Mean virus aggregate size Unitless Messner et al. (2014)  
Pill|inf Probability of illness among those infected Unitless Equation (5) Teunis et al. (2008)  
η Scale parameter Unitless 0.00255 Teunis et al. (2008)  
r Shape parameter Unitless 0.086 Teunis et al. (2008)  
Pill,event Probability of illness per exposure event Unitless Equation (6) Teunis et al. (2008)  
Pill,ann Annual probability of illness Unitless Equation (7) Mok et al. (2014)  
n Number of exposures in 1 year Day Uniform (208,365)  
DALY 
 DALY Annual DALY form of NoV GII illness DALY/person/year Equation (8)  
DB Disease burden per case of illness DALY per case of illness Uniform (3.7 × 10−4, 6.2 ×10−3Mok et al. (2014)  

V was assumed to follow a normal distribution with a mean of 0.108 mL/g and a standard deviation of 0.019 mL/g, truncated at zero similar to previous studies (Hamilton et al. 2006; Barker et al. 2013; Mok et al. 2014; Owusu-Ansah et al. 2017). Dw was assumed to follow a PERT distribution (0.1,1,2) (Seidu et al. 2008; Barker et al. 2013; Mok et al. 2014; Owusu-Ansah et al. 2017); it was assumed that all consumers wash lettuce before consumption. Dd was assumed to be 1.478 for Scenario 1, which followed a simplified assumption that raw wastewater was diluted by a factor of 1:30 into a receiving surface water before use in irrigation (McBride et al. 2013). For Scenario 2, we assumed that treated wastewater was used for irrigation directly, without dilution into surface water (i.e., Dd = 0). While these assumptions did not account for complex virus fate and transport processes that would require the assessment of site-specific conditions, we believe that this simplification is appropriate for the present study given that the QMRA was conducted for illustrative and comparison purposes.

Daily lettuce ingestion (I) varies between countries. For instance, lettuce consumption in Ghana has been estimated to be 10–12 g/day, 4 days/week (Seidu et al. 2008), whereas consumption in the United States has been estimated to be 0.539 g of lettuce per kg body mass per day (USEPA 2003). In previous QMRA studies, a uniform distribution between 10 and 20 g/person/day was used in Ghana (Owusu-Ansah et al. 2017), a lognormal distribution of (−2.23, 1.62) g/kg-person/day was used in Australia (Mok et al. 2014), and a value of 4.6 g/day was used in Japan (Ito et al. 2017). In the present study, a uniform distribution of 5–20 g/person/day was used to account for a wide range of consumption values.

Norovirus and E. coli concentrations in raw and treated wastewater used as QMRA inputs were extracted from Hata et al. (2013) (Table 2). This study reported NoV GII and E. coli concentrations in the same samples of raw and disinfected wastewater, collected from an urban wastewater treatment plant in Japan over a 6-month monitoring period (i.e., between October 2007 and March 2008), and was not included in our systematic review given its publication date before the start date of our search (i.e., 2015). Hata et al. (2013) reported concentrations of NoV GII and E. coli that were averaged over samples analyzed each month. The wastewater treatment plant evaluated by Hata et al. (2013) employed a conventional activated sludge process, followed by chlorination and sand filtration. The methods used for quantification were RT-QPCR for NoV GII (using the COG2F/COG2R/RING2-TP primer/probe set from Kageyama et al. (2003)) and a single-agar-layer method with Chromocult coliform agar for E. coli.

Table 2

Measured and estimated concentrations of E. coli (CFIB) and NoV GII (CNoV) in raw and disinfected wastewater

ParameterMean measured concentration (Hata et al. 2013)Concentration calculated with RWConcentration calculated with RNoVGII:FIB
Units log10 (CFU or gc/100 mL) log10 (gc/100 mL) log10 (gc/100 mL) 
Raw wastewater 
E. coli 7.1  
 NoV GII 4.5 Uniform (1.1, 2.1) Normal (5.6,1.4) 
Disinfected wastewater effluent 
E. coli 2.2  
 NoV GII 2.1 Uniform (−3.8, −2.8) Normal (2.7,2.3) 
ParameterMean measured concentration (Hata et al. 2013)Concentration calculated with RWConcentration calculated with RNoVGII:FIB
Units log10 (CFU or gc/100 mL) log10 (gc/100 mL) log10 (gc/100 mL) 
Raw wastewater 
E. coli 7.1  
 NoV GII 4.5 Uniform (1.1, 2.1) Normal (5.6,1.4) 
Disinfected wastewater effluent 
E. coli 2.2  
 NoV GII 2.1 Uniform (−3.8, −2.8) Normal (2.7,2.3) 

These values were used as inputs to calculate daily norovirus dose (Equation (2)). Measured CFIB and CNoV are from Hata et al. (2013). E. coli and NoV GII units are colony-forming units (CFU)/100 mL and gene copies (gc)/100 mL, respectively.

For the QMRA strategies that used assumed values of RNoV:FIB to estimate NoV GII concentrations based on measured FIB, CNoV was calculated following Equation (3), where R is the conversion ratio (either RNoV:FIB calculated in the present study or RW), and CFIB is the measured FIB concentration (CFU/mL).
formula
(3)

Dose-response model and risk characterization

A fractional Poisson dose-response model was employed to estimate the probability of infection per event (Pinf,event) for NoV GII (Messner et al. 2014; Equation (4)), where P is the fraction of secretor-positive individuals in the exposed population, d is the daily dose of NoV GII (Equation (2)), and μ is the mean virus aggregate size (where a value of 1 represents a disaggregated state). The fractional Poisson model assumes that secretor-positive individuals are fully susceptible to the virus and secretor-negative individuals are completely non-susceptible. Input parameters are summarized in Table 1.
formula
(4)

Through an assessment of virus aggregation in the context of QMRA, Abel et al. (2017) proposed that norovirus aggregation is less likely if the pH of water is greater than the virus's isoelectric point (IEP). The pH of wastewater often lies between 6.6 and 7.8 (Farkas et al. 2018) and is typically greater than the IEP of NoV GII (between 5.5 and 6.9; Goodridge et al. 2004). Therefore, and given ongoing uncertainties regarding norovirus aggregation and its impacts (McBride 2014), we assumed that the viruses were disaggregated (i.e., μ = 1). We note that the dose-response model employed herein was developed using data for NoV GI (Teunis et al. 2008; Messner et al. 2014), and there are presently no available dose-response data for NoV GII. This limitation is shared by other studies employing QMRA for NoV GII risk assessment.

The conditional probability of illness (; Equation (5)) was estimated according to the model presented by Teunis et al. (2008) and Mok et al. (2014), where η and r are the scale and shape parameters, respectively. The probability of illness per exposure event (Pill,event) was calculated by multiplying the probability of illness by the probability of infection (Equation (6); Teunis et al. 2008).
formula
(5)
formula
(6)

Annual probability of illness and disease burden

The annual probability of illness (Pill,ann) due to the consumption of wastewater-irrigated lettuce was calculated using Equation (7), where n is the number of days of exposure per person per year. It was assumed that lettuce consumption follows a uniform distribution between 208 and 365 days (Owusu-Ansah et al. 2017). Disease burden was estimated as disability-adjusted life years (DALY) lost using Equation (8), where DB is the NoV disease burden in units of DALY per case of norovirus illness.
formula
(7)
formula
(8)

Model implementation

To account for variability and uncertainty, the QMRA model was implemented using Monte Carlo simulation with 10,000 iterations. For each iteration, the probability distribution of each parameter was sampled to serve as the model input data. The QMRA model was implemented using RStudio and data preparation was performed using Excel (Microsoft). A sensitivity analysis was performed in RStudio using the ‘cor.test’ command with Spearman's correlation coefficient.

Systematic review

A total of 365 data points were extracted from the six publications that met the systematic review inclusion criteria (i.e., five publications from the Eftim et al. (2017) and Pouillot et al. (2015) reviews, and one from our literature search); each data point consisted of concentration data for a NoV-FIB pair. One hundred and seventy-nine data points included measured concentrations of NoV GI, and 186 data points included concentrations of NoV GII. A breakdown of the number of data points identified for each NoV-FIB pair is provided in Table 3.

Table 3

Summary statistics of pooled log10R for each NoV-FIB pair and each category of wastewater

NoV GI:E. coliNoV GI:FCNoV GII:E. coliNoV GII:FC
Raw wastewater 
 Mean log10RNoV:FIB −2.5 −3.0 −1.5 −1.9 
 Standard deviation 1.4 0.85 1.4 1.5 
n 79 22 77 24 
Secondary wastewater effluent 
 Mean log10RNoV:FIB −2.5 −1.9 −2.4 −2.1 
 Standard deviation 0.65 0.88 0.76 0.81 
n 38 12 39 12 
Disinfected wastewater effluent 
 Mean log10RNoV:FIB 1.1 −0.52 2.8 0.51 
 Standard deviation 1.1 1.6 1.3 2.3 
n 12 16 10 24 
NoV GI:E. coliNoV GI:FCNoV GII:E. coliNoV GII:FC
Raw wastewater 
 Mean log10RNoV:FIB −2.5 −3.0 −1.5 −1.9 
 Standard deviation 1.4 0.85 1.4 1.5 
n 79 22 77 24 
Secondary wastewater effluent 
 Mean log10RNoV:FIB −2.5 −1.9 −2.4 −2.1 
 Standard deviation 0.65 0.88 0.76 0.81 
n 38 12 39 12 
Disinfected wastewater effluent 
 Mean log10RNoV:FIB 1.1 −0.52 2.8 0.51 
 Standard deviation 1.1 1.6 1.3 2.3 
n 12 16 10 24 

n is the number of data points used to calculate each value.

All extracted FIB concentrations were quantified with culture-based assays, whereas all norovirus concentrations were quantified by RT-QPCR. A few different RT-QPCR forward primer/reverse primer/probe combinations were used for norovirus enumeration by the six included papers. For NoV GI, two studies used the QNIF4/NV1LCR/NV1LCpr primer/probe set (Sima et al. 2011; Worley-Morse et al. 2019), two studies used COG1F/COG1R/RING1-TP (Katayama et al. 2008; Montazeri et al. 2015), Flannery et al. (2012) used QNIF4/NV1LCR/NVGG1p, and Kauppinen et al. (2014) used NVGIF/NVGIR/NVGIP-MGB; furthermore, all of these assays differ from the NVKS1/NVKS2/NVKS3 primer/probe set used by Teunis et al. (2008) to quantify viruses for the NoV GI dose-response model. For NoV GII, three studies used the QNIF2d/COG2R/QNIFS primer/probe combination for quantification (Sima et al. 2011; Flannery et al. 2012; Worley-Morse et al. 2019), two used COG2F/COG2R/RING2-TP (Katayama et al. 2008; Montazeri et al. 2015), and Kauppinen et al. (2014) used QNIF2d/COG2R/RING2-TP. It is possible that the use of different primer/probe sets, RT-QPCR standards, reaction chemistries, or thermocycling conditions could result in different quantitation across studies, which would require data harmonization to best compare values across studies (McBride et al. 2013). However, to the best of our knowledge, there are no studies that provide direct comparisons between norovirus RT-QPCR assays, and we are therefore unable to apply harmonization factors for cross-study comparison. There is, therefore, some uncertainty regarding how norovirus concentrations quantified by different RT-QPCR assays relate to that used to develop the dose-response model; this limitation is shared by other studies conducting QMRA with measured or assumed norovirus concentrations as inputs.

Sample types included wastewater collected along the treatment train: 202 data points were measured in raw wastewater, 101 in secondary-treated wastewater (i.e., after activated sludge), and 62 from disinfected final effluent (i.e., after chlorination). NoV concentrations measured in raw wastewater displayed seasonal variability, whereas measured FIB concentrations were relatively similar across months (Figure 1). The included studies were conducted in five countries on three continents in the northern hemisphere: Europe (Ireland, Finland, and France), North America (USA), and Asia (Japan). A spreadsheet containing metadata is included as a Supplement.

Figure 1

Monthly variation in measured concentrations of the following in raw wastewater: (a) NoV GI, (b) NoV GII, (c) E. coli, and (d) FC. The data presented were all collected in the northern hemisphere, where December–January are winter months and June–August are summer months. GC, gene copies; MPN, most probably number; CFU, colony forming units.

Figure 1

Monthly variation in measured concentrations of the following in raw wastewater: (a) NoV GI, (b) NoV GII, (c) E. coli, and (d) FC. The data presented were all collected in the northern hemisphere, where December–January are winter months and June–August are summer months. GC, gene copies; MPN, most probably number; CFU, colony forming units.

Close modal

NoV-to-FIB ratios

A summary of log10RNoV:FIB means and standard deviations is provided in Table 3 and Figure 2. Mean ratios of NoV GI and GII to E. coli (RNoVGI:EC and RNoVGII:EC, respectively) in raw wastewater, averaged across all months, were 10−2.5±1.4 and 10−1.5±1.4, respectively. Both ratios were significantly greater than RW used in the WHO Guidelines for the Safe Use of Wastewater, Excreta and Greywater (i.e., 10−6–10−5 viruses per FIB), meaning that using RW to convert measured FIB concentrations to estimated norovirus concentrations would underpredict the number of viruses present. The average ratios of NoV GI and GII to FC (RNoVGI:FC and RNoVGII:FC, respectively) in raw wastewater were slightly lower (10−3.0±0.85 and 10−1.9±1.5, respectively) due to FC concentrations in wastewater typically occurring at greater concentrations than E. coli. Given the seasonal dependence observed for measured NoV concentrations in raw wastewater, values of RNoVGI:EC and RNoVGII:EC in this matrix displayed a seasonal trend as well (Figure 3).

Figure 2

Box and whisker plot of log10RNoV:FIB for each NoV-FIB pair in (a) raw wastewater, (b) secondary-treated wastewater, and (c) disinfected final effluent. Each ratio is the norovirus concentration divided by the FIB concentration (i.e., E. coli or FC). For each box plot, left and right borders represent the 25th and 75th percentiles, respectively; the line in the middle of the box represents the median. The left and right whiskers of each box extend to the 10th and 90th percentiles, respectively. The individual points plotted beyond the whisker lines are the outliers. The vertical shaded bar in each panel represents the range of log10RW used by the WHO Guidelines for the Safe Use of Wastewater, Excreta and Greywater (WHO 2006); i.e., 10−6–10−5 viruses per FIB). Please refer to the online version of this paper to see this figure in color: http://dx.doi.org/10.2166/wh.2021.068.

Figure 2

Box and whisker plot of log10RNoV:FIB for each NoV-FIB pair in (a) raw wastewater, (b) secondary-treated wastewater, and (c) disinfected final effluent. Each ratio is the norovirus concentration divided by the FIB concentration (i.e., E. coli or FC). For each box plot, left and right borders represent the 25th and 75th percentiles, respectively; the line in the middle of the box represents the median. The left and right whiskers of each box extend to the 10th and 90th percentiles, respectively. The individual points plotted beyond the whisker lines are the outliers. The vertical shaded bar in each panel represents the range of log10RW used by the WHO Guidelines for the Safe Use of Wastewater, Excreta and Greywater (WHO 2006); i.e., 10−6–10−5 viruses per FIB). Please refer to the online version of this paper to see this figure in color: http://dx.doi.org/10.2166/wh.2021.068.

Close modal
Figure 3

Seasonal trend of mean log10RNoV:FIB in raw wastewater for (a) NoV GI:E. coli and (b) NoV GII:E. coli. Error bars represent standard deviation.

Figure 3

Seasonal trend of mean log10RNoV:FIB in raw wastewater for (a) NoV GI:E. coli and (b) NoV GII:E. coli. Error bars represent standard deviation.

Close modal

Mean RNoVGI:EC in secondary effluent was similar to that in raw wastewater, whereas RNoVGII:EC and RNoVGII:FC in secondary effluent were slightly lower than those in raw wastewater. Mean RNoVGI:FC was greater in secondary effluent than in raw wastewater. The greatest RNoV:FIB values for all NoV-FIB pairs were observed in disinfected final effluent: mean RNoVGI:EC and RNoVGII:EC in final effluent were 101.1±1.1 and 102.8±1.3, respectively; mean RNoVGI:FC and RNoVGII:FC were 10−0.52±1.6 and 100.51±2.3, respectively. This finding is due to the observation that viruses are generally more persistent than bacteria when exposed to disinfectants (Tree et al. 2005; Hijnen et al. 2006), which results in a relatively greater number of measured viruses per bacteria post-disinfection. A limitation to this finding is that the methods used to quantify FIB and noroviruses have different abilities to quantify microorganism viability: the culture-based assays used to quantify bacteria are able to discern viability, whereas RT-QPCR methods used to quantify norovirus concentrations detect both viable and inactivated viruses. Therefore, although RNoV:FIB increased in disinfected effluent, it is unknown what proportion of measured noroviruses were in fact viable, and the value of RNoV:FIB in disinfected effluent may over-estimate the number of noroviruses that contribute to health risks.

While we have confidence in the general finding that the ratios between NoV and E. coli (i.e., RNoVGI:EC and RNoVGII:EC) in disinfected effluent were greater than those in raw wastewater or secondary effluent, there remains some uncertainty surrounding their values, given that there were limited data points available for the calculation of average RNoVGI:EC and RNoVGII:EC for disinfected final effluent (n = 12 and 10, respectively), and all were from a single paper (Montazeri et al. 2015). Future studies that measure concentrations of noroviruses and E. coli in disinfected wastewater effluent at different times of year and additional geographic regions (i.e., the southern hemisphere) are needed for an improved estimate of RNoV:EC in this matrix.

Multiple linear regression was conducted to determine factors that significantly contributed to the variability in log10RNoV:FIB (Equation (1), R2 = 0.13). Norovirus genogroup, exposure to disinfection, geographic location, and month were found to have a significant effect on log10RNoV:FIB (p < 0.05). This finding indicates that it is not appropriate to use one static ratio to estimate norovirus concentrations based on measured FIB abundances across all scenarios.

Influence of RNoV:FIB on QMRA-estimated risks

To evaluate the impact of assumed values of RNoV:FIB on health risks estimated by QMRA, risks from the consumption of wastewater-irrigated lettuce were assessed using three strategies to estimate the concentration of NoV in source water: (i) input of actual, measured NoV concentrations, (ii) use of measured E. coli concentrations paired with RW (i.e., uniform distribution between 10−6 and 10−5 NoV per E. coli), and (iii) use of measured E. coli concentrations paired with RNoV:FIB determined in the present study. For the latter, we used average RNoV:FIB values of 10−1.5±1.4 for raw wastewater and 100.51±2.3 for disinfected effluent. RNoV:FIB used for disinfected effluent is the ratio between NoV GII and FC (i.e., RNoVGII:FC); this value was used instead of the ratio between NoV GII and E. coli, given that limited data were available to calculate RNoVGII:EC (as discussed above).

The QMRA was conducted for illustrative and comparative purposes; it was not our aim to calculate absolute risks for a specific scenario, but rather to assess relative risks as impacted by the use of different values of RNoV:FIB. Therefore, the estimated risks presented below should not be applied to other scenarios. Additionally, for the purposes of this exercise, we assumed that noroviruses in solution were entirely disaggregated, which we acknowledge may not actually be the case in all wastewater matrices. Based on analysis by McBride (2014), if noroviruses experienced some degree of aggregation, we would expect the median infectious dose to increase and subsequent estimated health risks to decrease. Nonetheless, we expect that the relative risks calculated using the different RNoV:FIB to be similar in cases with and without aggregated noroviruses.

The annual probability of NoV GII illness (Pill,ann) from consuming lettuce irrigated with water contaminated with raw wastewater was estimated to be 0.038 when predicted using norovirus concentrations actually measured in wastewater (Table 4). The use of RW to predict the norovirus concentration based on measured FIB concentrations resulted in an estimate of Pill,ann that was almost six orders of magnitude lower, whereas Pill,ann estimated using RNoV:FIB determined in this study was greater than that predicted by the first strategy by approximately an order of magnitude (0.49). Predictions of the median annual disease burden (DALY) followed the same trend. A similar finding was observed for an exposure scenario that involved the consumption of lettuce irrigated with undiluted, disinfected wastewater effluent (Table 4). Although the RNoV:FIB values determined in this paper overpredicted norovirus exposure and subsequent health risks as compared to using measured virus concentrations directly for exposure assessment, they performed better than the use of RW, which severely underestimated health risks.

Table 4

Results of QMRA of health risks from NoV GII infection through the consumption of wastewater-irrigated lettuce

NoV GII dose (d)Pinf,eventPill,eventPill,annDisease burden (DALY)
Irrigation water: raw wastewater diluted 1:30 in surface water 
 Measured NoV GII 1.2 0.51 1.0 × 10−4 3.8 × 10−2 1.0 × 10−4 
 NoV GII estimated with RW 1.4 × 10−3 1.0 × 10−3 3.5 × 10−10 9.9 × 10−8 2.7 × 10−10 
 NoV GII estimated with RNoV:FIB= 10−1.5±1.4 15.7 0.72 2.5 × 10−4 0.49 9.8 × 10−4 
Irrigation water: undiluted, disinfected wastewater effluent 
 Measured NoV GII 0.20 0.13 5.6 × 10−6 1.6 × 10−4 4.2 × 10−6 
 NoV GII estimated with RW 6.0 × 10−7 4.3 × 10−7 5.6 × 10−17 2.3 × 10−14 1.2 × 10−17 
 NoV GII estimated with RNoV:FIB= 100.51±2.3 0.63 0.33 4.7 × 10−5 1.3 × 10−2 3.5 × 10−5 
NoV GII dose (d)Pinf,eventPill,eventPill,annDisease burden (DALY)
Irrigation water: raw wastewater diluted 1:30 in surface water 
 Measured NoV GII 1.2 0.51 1.0 × 10−4 3.8 × 10−2 1.0 × 10−4 
 NoV GII estimated with RW 1.4 × 10−3 1.0 × 10−3 3.5 × 10−10 9.9 × 10−8 2.7 × 10−10 
 NoV GII estimated with RNoV:FIB= 10−1.5±1.4 15.7 0.72 2.5 × 10−4 0.49 9.8 × 10−4 
Irrigation water: undiluted, disinfected wastewater effluent 
 Measured NoV GII 0.20 0.13 5.6 × 10−6 1.6 × 10−4 4.2 × 10−6 
 NoV GII estimated with RW 6.0 × 10−7 4.3 × 10−7 5.6 × 10−17 2.3 × 10−14 1.2 × 10−17 
 NoV GII estimated with RNoV:FIB= 100.51±2.3 0.63 0.33 4.7 × 10−5 1.3 × 10−2 3.5 × 10−5 

Two exposure scenarios were evaluated, as were three strategies for estimating the norovirus dose.

Owusu-Ansah et al. (2017) conducted a similar analysis to determine the influence of an assumed RNoV:FIB (i.e., 10−5 NoV per E. coli) on QMRA-estimated disease burden due to ingestion of noroviruses on wastewater-irrigated lettuce and cabbage. The authors found the median disease burden (quantified as DALYs) predicted by QMRA to be one to three orders of magnitude lower when norovirus concentrations were predicted with the 10−5 ratio than if the QMRA was administered with an actual, measured norovirus concentration as an input.

Some additional studies have used alternative values for RNoV:FIB. For example, given similar concerns with the use of the 10−5 NoV:FIB conversion ratio, Barker et al. (2014) used a uniform distribution of 10−3.66–10−0.77 to represent RNoV:FIB in QMRA of ingestion of wastewater-irrigated produce in Kumasi, Ghana. The conversion ratio used by Barker et al. (2014) was determined using bacteria and virus data measured by Haramoto et al. (2006), Katayama et al. (2008), La Rosa et al. (2010), and Silverman et al. (2013), and is similar to the range of raw wastewater RNoV:FIB determined in the present study. Similarly, Eregno et al. (2016) and Mohammed & Seidu (2019) assumed a conversion factor of 10−1.06 NoV per E. coli in QMRA of surface waters used for recreational purposes and drinking water, respectively, based on norovirus concentrations measured in nearby wastewater effluent.

A sensitivity analysis was conducted to evaluate which inputs to the QMRA model were most correlated with estimates of annual disease burden (DALY). To conduct the sensitivity analysis, annual DALYs were calculated for ingestion of lettuce irrigated with raw wastewater diluted in river water, and input–output correlations were performed using the ‘cor.test’ command in RStudio with Spearman's correlation coefficient. RNoV:FIB had the greatest magnitude of correlation with estimated DALYs (ρ = 0.81), followed by Dw (ρ = − 0.21). Correlation coefficients between estimated DALYs and I, n, and V were lower (ρ = 0.09, 0.04, and 0.07, respectively), indicating that these inputs had a smaller influence on modeled disease risks. Overall, this analysis highlights the influence of RNoV:FIB on QMRA-modeled risk estimates.

The observed discrepancies in health risk predictions that stem from estimating virus exposure based on RNoV:FIB emphasize that it is ideal to use measured pathogen concentrations as inputs to QMRA. In cases where only FIB data are available, conversion ratios to determine virus exposure should be used with caution and selected carefully based on the data available and local conditions (e.g., time of year; microorganism strain, genotype, or class; extent of wastewater treatment; and prevalence of infection). These findings extend to other enteric viruses of public health concern (e.g., rotaviruses and enteroviruses). Additionally, even when actual measured concentrations of a viral pathogen are used, there can be challenges in determining doses of infectious viruses. For example, all currently available data for noroviruses – including data used for the dose-response model, data used to calculate RNoV:FIB, and data used for direct input to QMRA – have been quantified with RT-QPCR, which cannot discern virus viability. An additional limitation is that virus recovery can vary between quantification methods and is not always included in estimates of exposure assessment.

  • Calculated ratios between measured norovirus and FIB concentrations (RNoV:FIB) differed depending on the norovirus genotype and FIB class considered, the extent of wastewater treatment, and the time of year. It is, therefore, inappropriate to use one static ratio to estimate norovirus concentrations based on measured FIB abundances in all scenarios.

  • Factors that influence norovirus disease prevalence, and therefore abundance in raw wastewater (such as norovirus genotype, month of sample collection, and geographic location), had a significant impact on RNoV:FIB, given that FIB concentrations in wastewater were found to be less variable.

  • RNoV:FIB was significantly influenced by the extent of wastewater treatment, given differences in the susceptibilities of noroviruses and FIB to treatment processes. However, it remains unclear what proportion of noroviruses measured in disinfected wastewater effluent are viable, given the use of RT-QPCR for virus quantification.

  • The choice of a value for RNoV:FIB has a large impact on QMRA-predicted health risks from exposure to norovirus due to ingestion of wastewater-irrigated lettuce. A value of RNoV:FIB previously used as an example by the WHO Guidelines for the Safe Use of Wastewater, Excreta and Greywater (WHO 2006) greatly underestimated health risks, as compared to risks predicted with measured norovirus concentrations input to the QMRA model.

  • The gold standard is to use measured pathogen concentrations directly as inputs to exposure assessment in QMRA. While not encouraged, if one has no option but to use a conversion ratio to estimate norovirus doses based on FIB measurements, the ratio should be chosen wisely based on available data, prevalence of infection, extent of wastewater treatment, and target microorganism strain, genotype, or class.

All relevant data are included in the paper or its Supplementary Information.

Abel
N. V.
,
Schoen
M. E.
,
Kissel
J. C.
&
Meschke
J. S.
2017
Comparison of risk predicted by multiple norovirus dose–response models and implications for quantitative microbial risk assessment
.
Risk Anal.
37
,
245
264
.
https://doi.org/10.1111/risa.12616
.
Baert
L.
,
Vandekinderen
I.
,
Devlieghere
F.
,
Van Coillie
E.
,
Debevere
J.
&
Uyttendaele
M.
2009
Efficacy of sodium hypochlorite and peroxyacetic acid to reduce murine norovirus 1, B40-8, Listeria monocytogenes, and Escherichia coli o157:H7 on shredded iceberg lettuce and in residual wash water
.
J. Food Prot.
72
,
1047
1054
.
https://doi.org/10.4315/0362-028x-72.5.1047
.
Barker
S. F.
,
O'Toole
J.
,
Sinclair
M. I.
,
Leder
K.
,
Malawaraarachchi
M.
&
Hamilton
A. J.
2013
A probabilistic model of norovirus disease burden associated with greywater irrigation of home-produced lettuce in Melbourne, Australia
.
Water Res.
47
,
1421
1432
.
https://doi.org/10.1016/j.watres.2012.12.012
.
Boehm
A. B.
,
Silverman
A. I.
,
Schriewer
A.
&
Goodwin
K.
2019
Systematic review and meta-analysis of decay rates of waterborne mammalian viruses and coliphages in surface waters
.
Water Res.
164
,
114898
.
https://doi.org/10.1016/j.watres.2019.114898
.
Eftim
S. E.
,
Hong
T.
,
Soller
J.
,
Boehm
A.
,
Warren
I.
,
Ichida
A.
&
Nappier
S. P.
2017
Occurrence of norovirus in raw sewage – a systematic literature review and meta-analysis
.
Water Res.
111
,
366
374
.
https://doi.org/10.1016/j.watres.2017.01.017
.
Eregno
F. E.
,
Tryland
I.
,
Tjomslandb
T.
,
Myrmel
M.
,
Robertson
L.
&
Heistad
A.
2016
Quantitative microbial risk assessment combined with hydrodynamic modelling to estimate the public health risk associated with bathing after rainfall events
.
Sci. Total Environ
.
548–549
,
270
279
. https://doi.org/10.1016/j.scitotenv.2016.01.034
.
Farkas
K.
,
Marshall
M.
,
Cooper
D.
,
McDonald
J. E.
,
Malham
S. K.
,
Peters
D. E.
,
Maloney
J. D.
&
Jones
D. L.
2018
Seasonal and diurnal surveillance of treated and untreated wastewater for human enteric viruses
.
Environ. Sci. Pollut. Res.
25
,
33391
33401
.
https://doi.org/10.1007/s11356-018-3261-y
.
Flannery
J.
,
Keaveney
S.
,
Rajko-Nenow
P.
,
O'Flaherty
V.
&
Doré
W.
2012
Concentration of norovirus during wastewater treatment and its impact on oyster contamination
.
Appl. Environ. Microbiol.
78
,
3400
3406
.
https://doi.org/10.1128/AEM.07569-11
.
Fuhrimann
S.
,
Winkler
M. S.
,
Stalder
M.
,
Niwagaba
C. B.
,
Babu
M.
,
Kabatereine
N. B.
,
Halage
A. A.
,
Utzinger
J.
,
Cissé
G.
&
Nauta
M.
2016
Disease burden due to gastrointestinal pathogens in a wastewater system in Kampala, Uganda
.
Microb. Risk Anal.
4
,
16
28
.
https://doi.org/10.1016/j.mran.2016.11.003
.
Fuhrimann
S.
,
Nauta
M.
,
Pham-Duc
P.
,
Tram
N. T.
,
Nguyen-Viet
H.
,
Utzinger
J.
,
Cissé
G.
&
Winkler
M. S.
2017
Disease burden due to gastrointestinal infections among people living along the major wastewater system in Hanoi, Vietnam
.
Adv. Water Resour.
108
,
439
449
.
https://doi.org/10.1016/j.advwatres.2016.12.010
.
Fung
J.
,
Keraita
B. N.
,
Konradsen
F.
,
Moe
C.
&
Akple
M. S. K.
2011
Microbiological Quality of Urban-Vended Salad and Its Association with Gastrointestinal Diseases in Kumasi, Ghana
.
Gonzales-Gustavson
E.
,
Rusiñol
M.
,
Medema
G.
,
Calvo
M.
&
Girones
R.
2019
Quantitative risk assessment of norovirus and adenovirus for the use of reclaimed water to irrigate lettuce in Catalonia
.
Water Res.
153
,
91
99
.
https://doi.org/10.1016/j.watres.2018.12.070
.
Goodridge
L.
,
Goodridge
C.
,
Wu
J.
,
Griffiths
M.
&
Pawliszyn
J.
2004
Isoelectric point determination of norovirus virus-like particles by capillary isoelectric focusing with whole column imaging detection
.
Anal. Chem.
76
,
48
52
.
https://doi.org/10.1021/ac034848s
.
Hall
A. J.
,
Lopman
B. A.
,
Payne
D. C.
,
Patel
M. M.
,
Gastañaduy
P. A.
,
Vinjé
J.
&
Parashar
U. D.
2013
Norovirus disease in the United States
.
Emerg. Infect. Dis.
19
,
1198
1205
.
https://doi.org/10.3201/eid1908.130465
.
Hamilton
A. J.
,
Stagnitti
F.
,
Premier
R.
,
Boland
A.-M.
&
Hale
G.
2006
Quantitative microbial risk assessment models for consumption of raw vegetables irrigated with reclaimed water
.
Appl. Environ. Microbiol.
72
,
3284
3290
.
https://doi.org/10.1128/AEM.72.5.3284-3290.2006
.
Haramoto
E.
,
Katayama
H.
,
Oguma
K.
,
Yamashita
H.
,
Tajima
A.
,
Nakajima
H.
&
Ohgaki
S.
2006
Seasonal profiles of human noroviruses and indicator bacteria in a wastewater treatment plant in Tokyo, Japan
.
Water Sci. Technol.
54
,
301
308
.
https://doi.org/10.2166/wst.2006.888
.
Haramoto
E.
,
Katayama
H.
,
Phanuwan
C.
&
Ohgaki
S.
2008
Quantitative detection of sapoviruses in wastewater and river water in Japan
.
Lett. Appl. Microbiol.
46
,
408
413
.
https://doi.org/10.1111/j.1472-765X.2008.02330.x
.
He
X.
,
Wei
Y.
,
Cheng
L.
,
Zhang
D.
&
Wang
Z.
2012
Molecular detection of three gastroenteritis viruses in urban surface waters in Beijing and correlation with levels of fecal indicator bacteria
.
Environ. Monit. Assess.
184
,
5563
5570
.
https://doi.org/10.1007/s10661-011-2362-6
.
Hijnen
W. A. M.
,
Beerendonk
E. F.
&
Medema
G. J.
2006
Inactivation credit of UV radiation for viruses, bacteria and protozoan (oo)cysts in water: a review
.
Water Res.
40
,
3
22
.
https://doi.org/10.1016/j.watres.2005.10.030
.
Ito
T.
,
Kitajima
M.
,
Kato
T.
,
Ishii
S.
,
Segawa
T.
,
Okabe
S.
&
Sano
D.
2017
Target virus log10 reduction values determined for two reclaimed wastewater irrigation scenarios in Japan based on tolerable annual disease burden
.
Water Res.
125
,
438
448
.
https://doi.org/10.1016/j.watres.2017.08.057
.
Kageyama
T.
,
Kojima
S.
,
Shinohara
M.
,
Uchida
K.
,
Fukushi
S.
,
Hoshino
F. B.
,
Takeda
N.
&
Katayama
K.
2003
Broadly reactive and highly sensitive assay for Norwalk-like viruses based on real-time quantitative reverse transcription-PCR
.
J. Clin. Microbiol.
41
,
1548
1557
.
https://doi.org/10.1128/JCM.41.4.1548–1557.2003
.
Katayama
H.
,
Haramoto
E.
,
Oguma
K.
,
Yamashita
H.
,
Tajima
A.
,
Nakajima
H.
&
Ohgaki
S.
2008
One-year monthly quantitative survey of noroviruses, enteroviruses, and adenoviruses in wastewater collected from six plants in Japan
.
Water Res.
42
,
1441
1448
.
https://doi.org/10.1016/j.watres.2007.10.029
.
Kauppinen
A.
,
Martikainen
K.
,
Matikka
V.
,
Veijalainen
A.-M.
,
Pitkänen
T.
,
Heinonen-Tanski
H.
&
Miettinen
I. T.
2014
Sand filters for removal of microbes and nutrients from wastewater during a one-year pilot study in a cold temperate climate
.
J. Environ. Manage.
133
,
206
213
.
https://doi.org/10.1016/j.jenvman.2013.12.008
.
Kundu
A.
,
Wuertz
S.
&
Smith
W. A.
2018
Quantitative microbial risk assessment to estimate the risk of diarrheal diseases from fresh produce consumption in India
.
Food Microbiol.
75
,
95
102
.
https://doi.org/10.1016/j.fm.2018.01.017
.
Machdar
E.
,
van der Steen
N. P.
,
Raschid-Sally
L.
&
Lens
P. N. L.
2013
Application of quantitative microbial risk assessment to analyze the public health risk from poor drinking water quality in a low income area in Accra, Ghana
.
Sci. Total Environ.
449
,
134
142
.
https://doi.org/10.1016/j.scitotenv.2013.01.048
.
Mara
D.
&
Sleigh
A.
2010
Estimation of norovirus infection risks to consumers of wastewater-irrigated food crops eaten raw
.
J. Water Health
8
,
39
43
.
https://doi.org/10.2166/wh.2009.140
.
Mara
D. D.
,
Sleigh
P. A.
,
Blumenthal
U. J.
&
Carr
R. M.
2007
Health risks in wastewater irrigation: comparing estimates from quantitative microbial risk analyses and epidemiological studies
.
J. Water Health
5
,
39
50
.
https://doi.org/10.2166/wh.2006.055
.
McBride
G. B.
2014
Norovirus dose-response in sewage-related QMRA: the importance of virus aggregation
. In:
Proceedings of the 7th International Congress on Environmental Modelling and Software, June 15–19
(
Ames
D. P.
,
Quinn
N. W. T.
&
Rizzoli
A. E.
, eds).
San Diego, CA
,
USA
.
ISBN: 978-88-9035-744-2
.
McBride
G. B.
,
Stott
R.
,
Miller
W.
,
Bambic
D.
&
Wuertz
S.
2013
Discharge-based QMRA for estimation of public health risks from exposure to stormwater-borne pathogens in recreational waters in the United States
.
Water Res.
47
,
5282
5297
.
https://doi.org/10.1016/j.watres.2013.06.001
.
Messner
M. J.
,
Berger
P.
&
Nappier
S. P.
2014
Fractional Poisson – a simple dose-response model for human norovirus
.
Risk Anal.
34
,
1820
1829
.
https://doi.org/10.1111/risa.12207
.
Mohammed
H.
&
Seidu
R.
2019
Climate-driven QMRA model for selected water supply systems in Norway accounting for raw water sources and treatment processes
.
Sci. Total Environ.
660
,
306
320
.
https://doi.org/10.1016/j.scitotenv.2018.12.460
.
Montazeri
N.
,
Goettert
D.
,
Achberger
E. C.
,
Johnson
C. N.
,
Prinyawiwatkul
W.
&
Janes
M. E.
2015
Pathogenic enteric viruses and microbial indicators during secondary treatment of municipal wastewater
.
Appl. Environ. Microbiol.
81
,
6436
6445
.
https://doi.org/10.1128/AEM.01218-15
.
Nordgren
J.
,
Matussek
A.
,
Mattsson
A.
,
Svensson
L.
&
Lindgren
P.-E.
2009
Prevalence of norovirus and factors influencing virus concentrations during one year in a full-scale wastewater treatment plant
.
Water Res.
43
,
1117
1125
.
https://doi.org/10.1016/j.watres.2008.11.053
.
NYSDEC
2014
New York state design standards for intermediate sized wastewater treatment systems
. In
New York State Department of Environmental Conservation
.
Ottoson
J.
,
Hansen
A.
,
Björlenius
B.
,
Norder
H.
&
Stenström
T. A.
2006
Removal of viruses, parasitic protozoa and microbial indicators in conventional and membrane processes in a wastewater pilot plant
.
Water Res.
40
,
1449
1457
.
https://doi.org/10.1016/j.watres.2006.01.039
.
Owusu-Ansah
E. d-G. J.
,
Sampson
A.
,
Amponsah
S. K.
,
Abaidoo
R. C.
,
Dalsgaard
A.
&
Hald
T.
2017
Probabilistic quantitative microbial risk assessment model of norovirus from wastewater irrigated vegetables in Ghana using genome copies and fecal indicator ratio conversion for estimating exposure dose
.
Sci. Total Environ.
601–602
,
1712
1719
.
https://doi.org/10.1016/j.scitotenv.2017.05.168
.
Patel
M. M.
,
Hall
A. J.
,
Vinjé
J.
&
Parashar
U. D.
2009
Noroviruses: a comprehensive review
.
J. Clin. Virol.
44
,
1
8
.
https://doi.org/10.1016/j.jcv.2008.10.009
.
Pavione
D. M. S.
,
Bastos
R. K. X.
&
Bevilacqua
P. D.
2013
Quantitative microbial risk assessment applied to irrigation of salad crops with waste stabilization pond effluents
.
Water Sci. Technol.
67
,
1208
1215
.
https://doi.org/10.2166/wst.2013.674
.
Petterson
S. R.
,
Stenström
T. A.
&
Ottoson
J.
2016
A theoretical approach to using faecal indicator data to model norovirus concentration in surface water for QMRA: Glomma River, Norway
.
Water Res.
91
,
31
37
.
https://doi.org/10.1016/j.watres.2015.12.037
.
Pouillot
R.
,
Doren
J. M. V.
,
Woods
J.
,
Plante
D.
,
Smith
M.
,
Goblick
G.
,
Roberts
C.
,
Locas
A.
,
Hajen
W.
,
Stobo
J.
,
White
J.
,
Holtzman
J.
,
Buenaventura
E.
,
Burkhardt
W.
,
Catford
A.
,
Edwards
R.
,
DePaola
A.
&
Calci
K. R.
2015
Meta-analysis of the reduction of norovirus and male-specific coliphage concentrations in wastewater treatment plants
.
Appl. Environ. Microbiol.
81
,
4669
4681
.
https://doi.org/10.1128/AEM.00509-15
.
Sales-Ortells
H.
,
Fernandez-Cassi
X.
,
Timoneda
N.
,
Dürig
W.
,
Girones
R.
&
Medema
G.
2015
Health risks derived from consumption of lettuces irrigated with tertiary effluent containing norovirus
.
Food Res. Int.
68
,
70
77
.
https://doi.org/10.1016/j.foodres.2014.08.018
.
Scallan
E.
,
Hoekstra
R. M.
,
Angulo
F. J.
,
Tauxe
R. V.
,
Widdowson
M.-A.
,
Roy
S. L.
,
Jones
J. L.
&
Griffin
P. M.
2011
Foodborne illness acquired in the United States – major pathogens
.
Emerg. Infect. Dis.
17
,
7
15
.
https://doi.org/10.3201/eid1701.P11101
.
Seidu
R.
,
Heistad
A.
,
Amoah
P.
,
Drechsel
P.
,
Jenssen
P. D.
&
Stenström
T.-A.
2008
Quantification of the health risk associated with wastewater reuse in Accra, Ghana: a contribution toward local guidelines
.
J. Water Health
6
,
461
471
.
https://doi.org/10.2166/wh.2008.118
.
Seto
E.
,
Olivieri
A. W.
&
Danielson
R. E.
2018
Quantitative microbial risk assessment used to evaluate seasonal wastewater treatment limits: case study in Vacaville, CA
.
Water Supply
18
,
910
925
.
https://doi.org/10.2166/ws.2017.162
.
Shuval
H.
,
Lampert
Y.
&
Fattal
B.
1997
Development of a risk assessment approach for evaluating wastewater reuse standards for agriculture
.
Water Sci. Technol.
1996
(
35
),
15
20
.
https://doi.org/10.1016/S0273-1223(97)00228-X
.
Siebenga
J. J.
,
Vennema
H.
,
Zheng
D.-P.
,
Vinjé
J.
,
Lee
B. E.
,
Pang
X.-L.
,
Ho
E. C. M.
,
Lim
W.
,
Choudekar
A.
,
Broor
S.
,
Halperin
T.
,
Rasool
N. B. G.
,
Hewitt
J.
,
Greening
G. E.
,
Jin
M.
,
Duan
Z.-J.
,
Lucero
Y.
,
O'Ryan
M.
,
Hoehne
M.
,
Schreier
E.
,
Ratcliff
R. M.
,
White
P. A.
,
Iritani
N.
,
Reuter
G.
&
Koopmans
M.
2009
Norovirus illness is a global problem: emergence and spread of norovirus GII.4 variants, 2001–2007
.
J. Infect. Dis.
200
,
802
812
.
https://doi.org/10.1086/605127
.
Silverman
A. I.
,
Akrong
M. O.
,
Amoah
P.
,
Drechsel
P.
&
Nelson
K. L.
2013
Quantification of human norovirus GII, human adenovirus, and fecal indicator organisms in wastewater used for irrigation in Accra, Ghana
.
J. Water Health
11
,
473
488
.
https://doi.org/10.2166/wh.2013.025
.
Sima
L. C.
,
Schaeffer
J.
,
Le Saux
J.-C.
,
Parnaudeau
S.
,
Elimelech
M.
&
Le Guyader
F. S.
2011
Calicivirus removal in a membrane bioreactor wastewater treatment plant
.
Appl. Environ. Microbiol.
77
,
5170
5177
.
https://doi.org/10.1128/AEM.00583-11
.
Teunis
P. F. M.
,
Moe
C. L.
,
Liu
P.
,
Miller
S. E.
,
Lindesmith
L.
,
Baric
R. S.
,
Pendu
J. L.
&
Calderon
R. L.
2008
Norwalk virus: how infectious is it?
J. Med. Virol.
80
,
1468
1476
.
https://doi.org/10.1002/jmv.21237
.
Tree
J. a.
,
Adams
M. r.
&
Lees
D. n.
2005
Disinfection of feline calicivirus (a surrogate for norovirus) in wastewaters
.
J. Appl. Microbiol.
98
,
155
162
.
https://doi.org/10.1111/j.1365-2672.2004.02442.x
.
USEPA
2000
Beaches Environmental Assessment and Coastal Health Act
.
United States Environmental Protection Agency
,
33 U.S.C. §1313 et seq
.
USEPA
2003
CSFII Analysis of Food Intake Distributions
.
United States Environmental Protection Agency
.
Report #EPA/600/R-03/029
.
WHO
2006
Guidelines for the Safe Use of Wastewater, Excreta and Greywater in Agriculture. Volume II: Wastewater Use in Agriculture
.
World Health Organization
,
Geneva
.
Worley-Morse
T.
,
Mann
M.
,
Khunjar
W.
,
Olabode
L.
&
Gonzalez
R.
2019
Evaluating the fate of bacterial indicators, viral indicators, and viruses in water resource recovery facilities
.
Water Environ. Res.
91
,
830
842
.
https://doi.org/10.1002/wer.1096
.
Xue
Y.
,
Pan
H.
,
Hu
J.
,
Wu
H.
,
Li
J.
,
Xiao
W.
,
Zhang
X.
,
Yuan
Z.
&
Wu
F.
2015
Epidemiology of norovirus infections among diarrhea outpatients in a diarrhea surveillance system in Shanghai, China: a cross-sectional study
.
BMC Infect. Dis.
15
.
https://doi.org/10.1186/s12879-015-0922-z
This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY-NC-ND 4.0), which permits copying and redistribution for non-commercial purposes with no derivatives, provided the original work is properly cited (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Supplementary data