The mEI, Chromocult® enterococci, and m-Enterococcus culture-based methods used to assess water quality by the detection of Enterococcus spp. were first compared in terms of sensitivity using (1) 41 different type strains of Enterococcus spp. and (2) environmental colonies identified by 16S rRNA sequencing. Then, two specific-rtPCR assays targeting Enterococcus spp. and Enterococcus faecalis/faecium were tested for their ability to confirm the identity of putative enterococcal colonies. The mEI, Chromocult® enterococci, and m-Enterococcus methods detected β-glucosidase activity for 28 (68.3%), 32 (78.0%), and 12 (29.3%) of the 41 reference enterococcal strains tested, respectively. Analysis with environmental colonies showed that mEI and Chromocult® enterococci media had false positive rates of 4.3% and 5.0%, respectively. Finally, the two rtPCR assays showed a specificity of 100%. Only two (2/19) colonies of E. faecium isolated from mEI agar were not detected by the Enterococcus faecium rtPCR assay, for a sensitivity of 89.5%. Our results showed that Chromocult® enterococci medium recovered more E. faecalis/faecium cells than the two other methods. Thus, the use of Chromocult® enterococci combined with the Enterococcus faecalis/faecium rtPCR assay showed the best combination to decrease the high false-positive rate obtained when the entire Enterococcus genus is targeted.

In the Province of Québec (Canada), the Programme d'accréditation des laboratoires d'analyse (‘Accreditation program of analytic laboratories’; PALA) is administered by the Centre d'expertise en analyse environnementale du Québec (CEAEQ) which accredits private, municipal and institutional laboratories. The Regulation respecting the quality of drinking water (RRQDW) states that when the water supplied by a distribution system comes in whole or in part from nondisinfected and vulnerable groundwater, the person in charge of the distribution system is bound to control the presence of Escherichia coli, enterococci, and coliphage viruses (Government of Québec 2013). Consequently, a water sample is considered contaminated by fecal pollution if at least one colony of enterococci bacteria is detected. The presence of enterococci in water is also considered by the United States Environmental Protection Agency (USEPA) as an indication of fecal pollution and of the possible presence of enteric pathogens, although some enterococcal species are naturally found in the environment and not necessarily related to fecal pollution (Kjellander 1960; Cabelli et al. 1982; Franz et al. 1999; USEPA 2005).

In fact, detecting Enterococcus spp. is of limited significance for determining the source of contamination in water since the broad spectrum of species cannot be used to distinguish non-fecal (environmental) from fecal contamination (Bonds et al. 2006; Converse et al. 2009). Indeed, there are many possible sources of Enterococcus sp. in water including animal waste (Devriese & De Plesmaecker 1987; Devriese et al. 1991; Sinton et al. 1993; Harwood et al. 2001), soil (Fujioka et al. 1999), invertebrates (Martin & Mundt 1972; Svec et al. 2002), and plants (Müller et al. 2001).

Thus, water quality assessment should more focus on a group of Enterococcus sp. that is associated with sources of fecal pollution rather than relying on the entire Enterococcus genus. Enterococcus faecalis and Enterococcus faecium are the predominant species of the Enterococcus genus found in human feces (Ruoff et al. 1990). All mammals carry these microorganisms in the colon (Noble 1978). Consequently, E. faecalis and E. faecium are potentially good fecal species as they have been consistently identified as predominant enterococcal species in warm-blooded animal feces and sewage, but not from environmental sources (Chenoweth & Schaberg 1990; Ruoff et al. 1990; Manero et al. 2002; Gelsomino et al. 2003). Furthermore, since Escherichia coli is 100 to 1,000 times more concentrated than Enterococcus spp. in feces, the probability of detecting non-faecalis/faecium Enterococcus species in water without any detection of E. coli nor E. faecalis/E. faecium is highly improbable (Slanetz & Bartley 1957; Layton et al. 2010). Thus, a detection method that would allow the specific detection of E. faecalis/E. faecium cells rather than all Enterococcus spp. should be more appropriate to assess water quality by the detection of fecal contamination.

Currently, enterococci in water are detected by different chromogenic culture-based methods comprising USEPA method 1600 on mEI agar (USEPA 2005), Chromocult® enterococci agar, and method 9230C of the Standard Methods for the Examination of Water and Wastewater manual (membrane filtration on m-Enterococcus agar; American Public Health Association/American Water Works Association/Water Environment Federation (APHA/AWWA/WEF 2012)). However, these methods do not discriminate between enterococci of environmental origin and enterococci of fecal origin.

In this study, we first compared mEI agar, Chromocult® enterococci agar, and m-Enterococcus agar, in terms of sensitivity, using a panel of different types of strains of enterococci and harvested environmental colonies identified by 16S rRNA sequencing. Secondly, two specific real-time polymerase chain reaction (rtPCR) assays targeting Enterococcus spp. and Enterococcus faecalis/faecium were tested for their ability to confirm, in less than 2 hours, the identity of putative enterococcal colonies grown on chromogenic culture-based agar in order to decrease the high false-positive rate obtained when the entire Enterococcus genus is targeted.

Analytical comparison

Bacterial strains

The ability of mEI, Chromocult® enterococci, and m-Enterococcus agar as well as Enterococcus spp.- and Enterococcus faecalis/faecium-specific rtPCR assays to detect enterococci strains was verified using 41 (for culture-based methods evaluation) and 55 (for rtPCR validation) different strains of Enterococcus spp. (Tables 1 and 2). Species identification was reconfirmed using an automated MicroScan Autoscan-4 system (Siemens Healthcare Diagnostic Inc., Newark, DE, USA) or a Vitek 2 system (bioMérieux SA, Marcy l’Étoile, France). Bacterial strains were grown from frozen stock kept at −80 °C in Brucella medium (Beckton Dickinson, Mississauga, Ontario, Canada), containing 10% glycerol. The strains were cultured on Brain Heart Infusion (BHI) agar. Three passages were performed prior to analysis of each strain with each culture-based method.

Table 1

Ability of the mEI, Chromocult® enterococci, and m-Enterococcus agar methods to detect enterococci strains

Tested species (n = 41)OriginNo. ReferencemEIChromocult® enterococcim-Enterococcus
E. aquimarinus CCRI-15963 − − − 
E. avium ATCC 14025 − − − 
E. caccae ATCC BAA1240 − 
E. canintestini CCUG 37857 − 
E. canis CCUG 46666 +/ − − 
E. casseliflavus ATCC 25788 
E. casseliflavus N/A ATCC 51328 
E. casseliflavus ATCC 12819 
E. cecorum ATCC 43198 − − 
E. columbae ATCC 51263 − − 
E. devriesi CCUG 37865 − − 
E. dispar ATCC 51266 − − 
E. durans ATCC 19432 − 
E. faecalis ATCC 19433 
E. faecalis ATCC 29212 
E. faecalis ATCC 51299 − 
E. faecium ATCC 19434 − 
E. faecium ATCC 700221 − − 
E. faecium CCRI-16518 
E. gallinarum CCRI-1433 
E. gallinarum LSPQ 3364 
E. gallinarum LSPQ 5375 
E. gilvus ATCC BAA350 +/ − − 
E. haemoperoxidus CCUG 45916 − 
E. hirae ATCC 8043 
E. hirae CCUG 37829 
E. italicus CCUG 50447 − − − 
E. malodoratus ATCC 43197 − − 
E. mundtii ATCC 43186 − 
E. pallens ATCC BAA351 +/ − − 
E. phoeniculicoli CCUG 48923 − 
E. pseudoavium ATCC 49372 − − − 
E. quebecensis CCRI-16985 − 
E. raffinosus ATCC 49427 − − − 
E. ratti ATCC 700914 − − − 
E. sileciacus CCUG 53830 − 
E. saccharolyticus ATCC 43076 − − 
E. sulfureus ATCC 49903 − − 
E. termitis CCUG 53831 − 
E. ureasiticus CCRI-16620 − 
E. villorum CCRI-8858 − − 
Total:   28/41 (68.3%) 32/41 (78.0%) 12/41 (29.3%) 
Tested species (n = 41)OriginNo. ReferencemEIChromocult® enterococcim-Enterococcus
E. aquimarinus CCRI-15963 − − − 
E. avium ATCC 14025 − − − 
E. caccae ATCC BAA1240 − 
E. canintestini CCUG 37857 − 
E. canis CCUG 46666 +/ − − 
E. casseliflavus ATCC 25788 
E. casseliflavus N/A ATCC 51328 
E. casseliflavus ATCC 12819 
E. cecorum ATCC 43198 − − 
E. columbae ATCC 51263 − − 
E. devriesi CCUG 37865 − − 
E. dispar ATCC 51266 − − 
E. durans ATCC 19432 − 
E. faecalis ATCC 19433 
E. faecalis ATCC 29212 
E. faecalis ATCC 51299 − 
E. faecium ATCC 19434 − 
E. faecium ATCC 700221 − − 
E. faecium CCRI-16518 
E. gallinarum CCRI-1433 
E. gallinarum LSPQ 3364 
E. gallinarum LSPQ 5375 
E. gilvus ATCC BAA350 +/ − − 
E. haemoperoxidus CCUG 45916 − 
E. hirae ATCC 8043 
E. hirae CCUG 37829 
E. italicus CCUG 50447 − − − 
E. malodoratus ATCC 43197 − − 
E. mundtii ATCC 43186 − 
E. pallens ATCC BAA351 +/ − − 
E. phoeniculicoli CCUG 48923 − 
E. pseudoavium ATCC 49372 − − − 
E. quebecensis CCRI-16985 − 
E. raffinosus ATCC 49427 − − − 
E. ratti ATCC 700914 − − − 
E. sileciacus CCUG 53830 − 
E. saccharolyticus ATCC 43076 − − 
E. sulfureus ATCC 49903 − − 
E. termitis CCUG 53831 − 
E. ureasiticus CCRI-16620 − 
E. villorum CCRI-8858 − − 
Total:   28/41 (68.3%) 32/41 (78.0%) 12/41 (29.3%) 

A, Animal; C, Clinical; E, Environmental; F, Food; W, Water; N/A, Non applicable; CCRI, Centre de recherche en infectiologie strain collection; ATCC, American Type Culture Collection; CCUG, Culture Collection (University of Gothenburg); LSPQ, Laboratoire de santé publique du Québec.

Table 2

Ability of the Enterococcus spp.-specific rtPCR assay and the multiplex Enterococcus faecalis/faecium-specific rtPCR assay to detect enterococci strains

Enterococcus-specific rtPCR assays
Tested species (n = 55)OriginNo. ReferenceEnterococcus spp.aE. faecalisbE. faeciumb
E. aquimarinus CCRI-15963 − − 
E. avium ATCC 14025 − − 
E. caccae ATCC BAA1240 − − 
E. canintestini CCUG 37857 − − 
E. canis CCUG 46666 − − 
E. casseliflavus ATCC 25788 − − 
E. casseliflavus N/A ATCC 51328 − − 
E. casseliflavus ATCC 12819 − − 
E. cecorum ATCC 43198 − − 
E. columbae ATCC 51263 − − 
E. devriesi CCUG 37865 − − 
E. dispar ATCC 51266 − − 
E. durans ATCC 19432 − − 
E. faecalis ATCC 19433 − 
E. faecalis ATCC 23241 − 
E. faecalis ATCC 29212 − 
E. faecalis ATCC 49533 − 
E. faecalis ATCC 51299 − 
E. faecalis CCRI-16012 − 
E. faecalis N/A LSPQ 5192 − 
E. faecalis N/A LSPQ 5378 − 
E. faecalis N/A LSPQ 5548 − 
E. faecalis N/A LSPQ 5570 − 
E. faecalis N/A LSPQ 5638 − 
E. faecalis N/A LSPQ 5660 − 
E. faecium ATCC 19434 − 
E. faecium ATCC 700221 − 
E. faecium CCRI-16518 − 
E. faecium CCRI-19447 − 
E. faecium CCRI-19448 − 
E. flavescens ATCC 49996 − − 
E. flavescens ATCC 49997 − − 
E. gallinarum CCRI-1433 − − 
E. gallinarum LSPQ 3364 − − 
E. gallinarum LSPQ 5375 − − 
E. gilvus ATCC BAA350 − − 
E. haemoperoxidus CCUG 45916 − − 
E. hirae ATCC 8043 − − 
E. hirae CCUG 37829 − − 
E. italicus CCUG 50447 − − 
E. malodoratus ATCC 43197 − − 
E. moraviensis CCUG 45913 − − 
E. mundtii ATCC 43186 − − 
E. pallens ATCC BAA351 − − 
E. phoeniculicoli CCUG 48923 − − 
E. pseudoavium ATCC 49372 − − 
E. quebecensis CCRI-16985 − − 
E. raffinosus ATCC 49427 − − 
E. ratti ATCC 700914 − − 
E. sileciacus CCUG 53830 − − 
E. saccharolyticus ATCC 43076 − − 
E. sulfureus ATCC 49903 − − 
E. termitis CCUG 53831 − − 
E. ureasiticus CCRI-16620 − − 
E. villorum CCRI-8858 − − 
Enterococcus-specific rtPCR assays
Tested species (n = 55)OriginNo. ReferenceEnterococcus spp.aE. faecalisbE. faeciumb
E. aquimarinus CCRI-15963 − − 
E. avium ATCC 14025 − − 
E. caccae ATCC BAA1240 − − 
E. canintestini CCUG 37857 − − 
E. canis CCUG 46666 − − 
E. casseliflavus ATCC 25788 − − 
E. casseliflavus N/A ATCC 51328 − − 
E. casseliflavus ATCC 12819 − − 
E. cecorum ATCC 43198 − − 
E. columbae ATCC 51263 − − 
E. devriesi CCUG 37865 − − 
E. dispar ATCC 51266 − − 
E. durans ATCC 19432 − − 
E. faecalis ATCC 19433 − 
E. faecalis ATCC 23241 − 
E. faecalis ATCC 29212 − 
E. faecalis ATCC 49533 − 
E. faecalis ATCC 51299 − 
E. faecalis CCRI-16012 − 
E. faecalis N/A LSPQ 5192 − 
E. faecalis N/A LSPQ 5378 − 
E. faecalis N/A LSPQ 5548 − 
E. faecalis N/A LSPQ 5570 − 
E. faecalis N/A LSPQ 5638 − 
E. faecalis N/A LSPQ 5660 − 
E. faecium ATCC 19434 − 
E. faecium ATCC 700221 − 
E. faecium CCRI-16518 − 
E. faecium CCRI-19447 − 
E. faecium CCRI-19448 − 
E. flavescens ATCC 49996 − − 
E. flavescens ATCC 49997 − − 
E. gallinarum CCRI-1433 − − 
E. gallinarum LSPQ 3364 − − 
E. gallinarum LSPQ 5375 − − 
E. gilvus ATCC BAA350 − − 
E. haemoperoxidus CCUG 45916 − − 
E. hirae ATCC 8043 − − 
E. hirae CCUG 37829 − − 
E. italicus CCUG 50447 − − 
E. malodoratus ATCC 43197 − − 
E. moraviensis CCUG 45913 − − 
E. mundtii ATCC 43186 − − 
E. pallens ATCC BAA351 − − 
E. phoeniculicoli CCUG 48923 − − 
E. pseudoavium ATCC 49372 − − 
E. quebecensis CCRI-16985 − − 
E. raffinosus ATCC 49427 − − 
E. ratti ATCC 700914 − − 
E. sileciacus CCUG 53830 − − 
E. saccharolyticus ATCC 43076 − − 
E. sulfureus ATCC 49903 − − 
E. termitis CCUG 53831 − − 
E. ureasiticus CCRI-16620 − − 
E. villorum CCRI-8858 − − 

A, Animal; C, Clinical; E, Environmental; F, Food; W, Water; N/A, Non applicable; CCRI, Centre de recherche en infectiologie strain collection; ATCC, American Type Culture Collection; CCUG, Culture Collection (University of Gothenburg); LSPQ, Laboratoire de santé publique du Québec.

Preparation of the bacterial cell suspension for analytical analysis

Colonies obtained from frozen stocks were suspended in BHI broth and adjusted to a 0.5 McFarland standard (Fisher Scientific Company, Ottawa, Ontario, Canada) before being serially diluted 10-fold in phosphate-buffered saline (PBS; 137 mM NaCl, 6.4 mM Na2HPO4, 2.7 mM KCl, 0.88 mM KH2PO4, pH 7.4). For each strain, an aliquot of the 10−5 dilution was spiked in sterile reverse osmosis-purified water (resistivity of 18 MΩ-cm min at 25 °C) to produce suspensions containing approximately 102 colony forming units (CFU) per 100 mL of water. Bacterial counts were verified by filtering three 100 mL volumes of each spiked water sample through Millipore membrane filters (47 mm diameter, 0.45 μm pore size; Millipore Corporation, Billerica, MA, USA) with a standard platform manifold (Millipore Corporation) followed by an incubation on BHI agar for 24 ± 2 h at 35.0 ± 0.5 °C. Tests to confirm the sterility of filter membranes and buffer used for rinsing the filtration apparatus were also performed.

Membrane filtration method

Membrane filtration was performed according to Maheux et al. (2009). Volumes (300 mL) spiked with reference enterococcal bacteria were split into three 100 mL volumes and filtered on Millipore filters with a standard platform manifold. The first filter was incubated on mEI agar (BD, Franklin Lakes, NJ, USA), the second filter on Chromocult® enterococci agar (Merk KGaA, Darmstadt, Germany) and both were incubated for 24 ± 2 h at 35.0 ± 0.5 °C. The third filter was incubated on m-Enterococcus agar plates (BD Company, Franklin Lakes, NJ, USA) for 48 ± 3 h at 35.0 ± 0.5 °C before determining colony counts and color (Table 1). Each preparation of mEI, Chromocult® enterococci, and m-Enterococcus was tested for performance using pure cultures of target and non-target microorganisms, as recommended by the USEPA microbiology methods manual. Tests were also performed to confirm the sterility of the filter membranes and buffer used for rinsing the filtration apparatus (APHA/AWWA/WEF 2012).

Sample collection

The sewage water sample used in this study to test environmental colonies was harvested at the discharge of the grit chambers of the west wastewater treatment plant of Québec City, in December 2014. Three 2 μL volumes of sewage water were spiked in 100 mL sterile water and filtered on Millipore filters with a standard platform manifold. The filters were then incubated on mEI agar, Chromocult® enterococci agar, and m-Enterococcus agar as described in the above section ‘Membrane filtration method’.

Molecular-based confirmation method

Preparation of bacterial suspensions

Each environmental colony recovered on mEI agar, Chromocult® enterococci agar, and m-Enterococcus agar was touched with a sterile toothpick and resuspended in 100 μL of sterile reverse osmosis-purified water (resistivity of 18 MΩ-cm min at 25 °C). This suspension was used for rtPCR tests described below.

PCR primers

The identity of the environmental colonies isolated on mEI, Chromocult® enterococci, and m-Enterococcus plates was confirmed by nucleotide sequencing of 16S rRNA gene using amplification and sequencing primers, SSU27 and SSU534R, an adaptation of Sistek et al. (2012). The SSU534R PCR primer, designed for this study, was developed as follows. First, 16S rRNA gene sequences available from public databases were analyzed with GCG programs (version 8.0; Accelrys, Madison, WI, USA). Based on a multiple sequence alignment and the Oligo primer analysis software (version 5.0; National Biosciences, Plymouth, MN, USA), the SSU534R PCR primer was designed from highly conserved regions of the gene. Primers and probes for Enterococcus spp.- and E. faecalis/faecium-specific rtPCR assays were used as described by Maheux et al. (2011). Sequences of the PCR primers are presented in Table 3. Oligonucleotide primers were synthesized by Integrated DNA Technologies (Coralville, IA, USA).

Table 3

Real-time PCR primers and probe used in this study

Microorganisms (Targeted gene)Primer and probePrimer and probe sequence (5′ → 3′)Reference
Sequencing and genotypic identification (16S rRNA) SSU27
SSU534R 
AGAGTTTGATCMTGGCTCAG
ATTACCGCGGCTGCTGG 
Adapted from Sistek et al. (2012)  
Enterococcus spp. (23S rRNA) ECST784F AGAAATTCCAAACGAACTTG Frahm & Obst (2003)  
ENC854R CAGTGCTCTACCTCCATCATT  
GPL813TQ FAMa-TGGTTCTCTCCGAAATAGCTTTAGGGCTA-BHQ-1b  
Enterococcus faecalis (mtlf) Mefs569 GAACAGAAGAAGCCAAAAAA Maheux et al. (2011)  
Mefs670 GCAATCCCAAATAATACGGT  
Mefs-TL1-A1 FAMa-CALGGAATLCTGTLGTALGTGLCAAG-BHQ-1b  
Enterococcus faecium (ddlDefm273 TGCTTTAGCAACAGCCTATCAG  
Defm468 TAAACTTCTTCCGGCACTTCG  
Defm-T1-F2 CalFluorRed610c-CTCGAGCAATCGTTGAACAAGGAATTG-BHQ-2d  
Microorganisms (Targeted gene)Primer and probePrimer and probe sequence (5′ → 3′)Reference
Sequencing and genotypic identification (16S rRNA) SSU27
SSU534R 
AGAGTTTGATCMTGGCTCAG
ATTACCGCGGCTGCTGG 
Adapted from Sistek et al. (2012)  
Enterococcus spp. (23S rRNA) ECST784F AGAAATTCCAAACGAACTTG Frahm & Obst (2003)  
ENC854R CAGTGCTCTACCTCCATCATT  
GPL813TQ FAMa-TGGTTCTCTCCGAAATAGCTTTAGGGCTA-BHQ-1b  
Enterococcus faecalis (mtlf) Mefs569 GAACAGAAGAAGCCAAAAAA Maheux et al. (2011)  
Mefs670 GCAATCCCAAATAATACGGT  
Mefs-TL1-A1 FAMa-CALGGAATLCTGTLGTALGTGLCAAG-BHQ-1b  
Enterococcus faecium (ddlDefm273 TGCTTTAGCAACAGCCTATCAG  
Defm468 TAAACTTCTTCCGGCACTTCG  
Defm-T1-F2 CalFluorRed610c-CTCGAGCAATCGTTGAACAAGGAATTG-BHQ-2d  

aFAM, 6-carboxyfluorescein, fluorescence reporter dye.

bBHQ-1, Black Hole Quencher-1, fluorescence quencher dye.

cCalFluorRed610, fluorescence reporter dye.

dBHQ-2, Black Hole Quencher-1, fluorescence quencher dye.

LN: locked nucleic acid (LNA) analog of a nucleotide.

PCR amplification

For sequencing of the 16S rRNA gene for genotypic identification, 1 μL of each bacterial suspension was transferred directly to 49 μL of PCR mixture containing 50 mM KCl, 10 mM Tris-HCl (pH 9.0), 0.1% Triton X-100, 2.5 mM MgCl2, 0.4 mM of each primer, 200 mM each of deoxyribonucleoside triphosphate (GE Healthcare Bio-Sciences Inc., Baie d'Urfé, Québec, Canada), 3.3 mg per mL of bovine serum albumin (BSA; Sigma-Aldrich Canada Ltd, Oakville, Ontario, Canada), 0.06 μg/μL methoxalen (Sigma-Aldrich Canada Ltd), 0.5 enzyme unit (U) of Taq DNA polymerase (Promega, Madison, WI, USA) and TaqStart antibody (Clonetech Laboratories, Mountain View, CA, USA). Decontamination of the PCR mixtures prior to PCR was achieved using the UV crosslinker Spectrolinker™ model XL-1000 (Spectronics Corporation, Westbury, NY, USA; Maheux et al. 2008). For each experiment, 1 μL of sterile water was added to the PCR mixture as a negative control. The PCR mixtures were subjected to thermal cycling (3 min at 95 °C and then 40 cycles of 1 s at 95 °C, 30 s at 60 °C, and 30 s at 72 °C, with a 5-min final extension step at 72 °C) with a Mastercycler PRO (Eppendorf Canada Ltd, Mississauga, ON, Canada). An agarose gel analysis of the amplified PCR products was performed as previously described (Martineau et al. 1998).

Sequencing of the 16S rRNA gene was performed as described by Picard et al. (2004). Molecular analysis of sequences was conducted using NCBI Blast (http://blast.ncbi.nlm.nih.gov/Blast.cgi, Standard Nucleotide BLAST optimized for highly similar sequence) and the Ribosomal database project (http://rdp.cme.msu.edu/, isolates with good quality sequence in Sequence Match search).

For the validation of the Enterococcus spp. and the E. faecalis/faecium rtPCR assays and or confirmation of colony identity, 1 μL of each bacterial suspension was transferred directly to a 24 μL PCR mixture containing 50 mM KCl, 10 mM Tris-HCl (pH 9.0), 0.1% Triton X-100, 2.5 mM MgCl2, 0.4 μM of Enterococcus spp. or E. faecalis/faecium, primers, 0.2 μM of Enterococcus spp. or E. faecalis/faecium probe, 200 μM each deoxyribonucleoside triphosphate (GE Healthcare Bio-Sciences Inc., Baie d'Urfé, Québec, Canada), 3.3 μg per μL of bovine serum albumin (BSA; Sigma-Aldrich Canada Ltd, Oakville, Ontario, Canada), 0.025 enzyme unit (U) of Taq DNA polymerase (Promega, Madison, WI, USA), and TaqStart antibody (Clontech Laboratories, Mountain View, CA, USA). For each experiment, 1 μL of sterile water was added to the rtPCR mixture as negative control. The rtPCR mixtures were subjected to thermal cycling (1 min at 95 °C and then 45 cycles of 15 s at 95 °C and 60 s at 60 °C for Enterococcus spp. rtPCR assay; 1 min at 95 °C and then 45 cycles of 15 s at 95 °C, 10 s at 60 °C and 20 s at 72 °C for E. faecalis/faecium rtPCR assay) with a Rotor-Gene Thermocycler (QIAGEN Inc., Mississauga, Ontario, Canada).

Ability of mEI, Chromocult® enterococci, and m-Enterococcus agar methods as well as Enterococcus spp.- and Enterococcus faecalis/faecium-specific real-time PCR assays to detect enterococcal strains

The specificity of the mEI, Chromocult® enterococci, and m-Enterococcus agar methods was demonstrated by testing genomic DNA isolated from 41 enterococcal strains from different species and origin (Table 1). The mEI agar method detected 68.3% (28/41) of the 41 enterococcal strains tested in which only 75% (3/4) of E. faecalis and 66.7% (2/3) E. faecium tested were detected. A positive signal on mEI agar was also detected for Enterococcus caccae, Enterococcus canintestini, Enterococcus canis, Enterococcus casseliflavus, Enterococcus columbae, Enterococcus dispar, Enterococcus durans, Enterococcus gallinarum, Enterococcus gilvus, Enterococcus haemoperoxidus, Enterococcus hirae, Enterococcus mundtii, Enterococcus pallens, Enterococcus phoeniculicoli, Enterococcus quebecensis, Enterococcus sileciacus, Enterococcus sulfureus, Enterococcus termitis, and Enterococcus villorum. These results showed that the recovery of E. faecalis and E. faecium cells could not be optimal with this medium. Furthermore, strictly environmental Enterococcus spp. (E. mundtii and E. sulfureus seem to be strictly associated to plants and soil (Müller et al. 2001)) are also detected by the mEI agar method suggesting that colonies obtained from water samples should be identified to confirm a contamination of fecal origin.

The Chromocult® enterococci agar method detected 78.0% (32/41) of the 41 enterococcal strains tested in which all (100%) of E. faecalis and E. faecium tested were detected. The only enterococcal species not detected by Chromocult® enterococci method were E. aquimarinus, E. avium, E. columbae, E. dispar, E. italicus, E. pseudoavium, E. raffinosus, E. ratti, and E. sulfureus. These results showed that the Chromocult® enterococci method could be a good choice to assess water quality since all E. faecalis and E. faecium tested were detected. However, as for the mEI agar method, strictly environmental Enterococcus spp. are also detected suggesting that colonies obtained from water samples should be identified to confirm a contamination of fecal origin.

Finally, the m-Enterococcus agar method detected only 29.3% (12/41) of the 41 enterococcal strains tested in which all (100%) of E. faecalis but only 33% (1/3) of E. faecium tested were detected. With the exception of E. faecalis and E. faecium, the spectrum of detection of m-Enterococcus agar method was only E. casseliflavus, E. gallinarum, and E. hirae when colonies were subjected to phenotypical confirmation tests (CEAEQ 2014). However, in order to bypass those phenotypic confirmations tests, colonies harvested on m-Enterococcus agar from sewage water described below were only subjected to confirmation using rtPCR assays. These results showed that the recovery of E. faecium cells could not be optimal with this medium.

In traditional water quality assessment, the definition of Enterococcus sp. is phenotypical. Indeed, an Enterococcus is defined as spherical bacterium, in pair or chain, Gram-positive, catalase-negative and facultatively anaerobic. It does not form endospores and some species demonstrate mobility. In addition, an Enterococcus hydrolyzes esculin in the presence of bile and has the ability to grow at 10 °C and 45 °C, pH 9.6 or in the presence of NaCl 6.5% (Mundt 1986; Knudtson & Hartman 1992; APHA/AWWA/WEF 2012). However, since the use of genotypic classification, it has been shown some species of the Enterococcus genus do not express β-glucosidase at 35 °C after 24–48 hours on chromogenic media (Maheux et al. 2008; Sistek et al. 2012). However, that does not mean that they do not express the enzyme at all. But using those culture conditions, the expression of β-glucosidase is not detected.

The ability of Enterococcus spp.- and Enterococcus faecalis/faecium-specific real-time PCR assays to detect enterococcal strains was demonstrated by testing genomic DNA isolated from 55 enterococcal strains including 12 E. faecalis and 5 E. faecium strains (Table 2). The Enterococcus sp.-specific rtPCR primers and probe efficiently amplified DNA from all 55 enterococcal strains tested whereas the multiplexed E. faecalis/faecium rtPCR assay efficiently amplified DNA from 12 of 12 (100%) E. faecalis and 5 of 5 (100%) E. faecium strains tested, respectively. Thus, against all enterococcal strains, the Enterococcus sp. rtPCR assay is 100% sensitive in its ability to detect all enterococcal strains, whereas the multiplex E. faecalis/faecium rtPCR assay is 100% sensitive for the detection of E. faecalis and E. faecium. Maheux et al. (2011) tested 150 closely related non-enterococcal species among the Enterococcus spp.-specific rtPCR assay and showed that there was no specific amplification of the 150 non-enterococcal bacterial species with the exception of Tetragenococcus solitarius. Phylogenetically, T. solitarius is very closely related to enterococci (Ke et al. 1999; Ennahar & Cai 2005) and controversy in its taxonomical classification persists.

Ability of mEI agar, Chromocult® enterococci agar, and m-Enterococcus agar to detect Enterococcus spp. from a sewage water sample

Three 2 μL volumes of a 1-L sewage water sample harvested at the discharge of the grit chambers of the west wastewater treatment plant of Québec City were used for testing by mEI agar, Chromocult® enterococci agar, and m-Enterococcus agar methods to verify their comparative ability to detect Enterococcus spp. (Table 4). All the colonies were harvested and subjected to molecular identification using a 527 base pairs fragment of the 16S rRNA gene. On mEI agar, 23 blue (phenotype +) and 14 pink (phenotype −) colonies were harvested for a total of 37 colonies. On Chromocult® enterococci and m-Enterococcus agar, 40 pink (phenotype +) and 49 red/pink (phenotype +) colonies were harvested, respectively. All these colonies were harvested and subjected to molecular identification using the 16S rRNA gene. Molecular analysis showed that 1 blue (1/23) colony on mEI agar and 2 pink (2/40) colonies on Chromocult® enterococci agar were not enterococcal species for a false positive rate of 4.3% and 5.0%, respectively. No false positive results have been found on m-Enterococcus agar. On the contrary, 2 pink (2/12) colonies on mEI agar were enterococcal species for a false negative rate of 16.7%. By using 16S rRNA molecular identification as gold standard, m-Enterococcus agar method has presented the highest specificity (100%), sensitivity (100%), and positive predictive value (100%; Table 4) for the detection of Enterococcus spp. Finally, Chromocult® enterococci agar and m-Enterococcus agar detected 38 and 39 Enterococcus spp. colonies respectively, whereas mEI agar detected only 24 Enterococcus spp. colonies.

Table 4

Specificitya, sensitivityb, and predictive values of each method for the detection of Enterococcus spp. as compared to 16S rRNA gene identification

Method and16S rRNA identification
Positive predictiveNegative predictive
results+SpecificityaSensitivitybvaluecvalued
mEI agar 
  + 22 92.3% 91.7% 95.6% 85.7% 
  − 12 
Chromocult® enterococci agar 
  + 38 0% 100.0% 100.0% N/C 
  − 
m-Enterococcus agar 
  + 39 100% 100.0% 100.0% N/C 
  − 
Method and16S rRNA identification
Positive predictiveNegative predictive
results+SpecificityaSensitivitybvaluecvalued
mEI agar 
  + 22 92.3% 91.7% 95.6% 85.7% 
  − 12 
Chromocult® enterococci agar 
  + 38 0% 100.0% 100.0% N/C 
  − 
m-Enterococcus agar 
  + 39 100% 100.0% 100.0% N/C 
  − 

N/C, not calculable.

aNo. of true negative results/(no. of true negative + false positive results).

bNo. of true positive results/(no. of true positive + false negative results).

cNo. of true positive results/(no. of true positive + false positive results).

dNo. of true negative results/(no. of true negative + false negative results).

Population of Enterococcus per species detected by mEI agar, Chromocult® enterococci agar, and m-Enterococcus agar isolated from the same sewage water sample

After colonies were identified using 16S rRNA analysis, the population of total coliforms enumerated by mEI agar, Chromocult® enterococci agar, and m-Enterococcus agar, were classified per species (Table 5). Results showed that the population of enterococcal species detected was more extended for m-Enterococcus agar (7 different species) than mEI and Chromocult® coliform agar (only 3 and 4 different species, respectively; Table 5). Like the Maheux et al. (2009) study, the results of the present study showed the lack of correlation between test methods based on the same enzymatic principle to recognize a strain as Enterococcus spp. Indeed, our results showed that there is a weak correlation between the three methods tested within the same species. Since all colonies of the present study were isolated from the same water sample and treated in the same way (filtration, incubation, etc.), the difference observed in the population of strains detected by each method cannot just be attributed to environmental factors. The composition of each medium is also involved. As observed in the analytical sensitivity section above, the mEI agar method detected less E. faecalis than the two other methods. Furthermore, m-Enterococcus detected less E. faecium than the two other methods. Consequently, the Chromocult® enterococci method seems to be the best method among the three tested to recover E. faecalis and E. faecium cells.

Table 5

Population of Enterococcus per species detected by mEI agar, Chromocult® enterococci agar, and m-Enterococcus agar from 2 μL of sewage water

mEI agar
Chromocult® enterococci agar
m-Enterococcus agar
SpeciesNo.%SpeciesNo.%SpeciesNo.%
E. faecium 22 91.7 E. faecium 20 28.2 E. faecalis 15 38.5 
E. faecalis 4.2 E. faecalis 25.6 E. faecium 13 33.3 
E. casseliflavus 4.3 E. hirae 23.1 E. hirae 15.4 
   E. casseliflavus  E. casseliflavus 5.1 
      E. avium 2.6 
       E. mundtii 2.6 
        E. sulfureus 2.6 
Total 24 100.0   38 100.0   39 100.0 
mEI agar
Chromocult® enterococci agar
m-Enterococcus agar
SpeciesNo.%SpeciesNo.%SpeciesNo.%
E. faecium 22 91.7 E. faecium 20 28.2 E. faecalis 15 38.5 
E. faecalis 4.2 E. faecalis 25.6 E. faecium 13 33.3 
E. casseliflavus 4.3 E. hirae 23.1 E. hirae 15.4 
   E. casseliflavus  E. casseliflavus 5.1 
      E. avium 2.6 
       E. mundtii 2.6 
        E. sulfureus 2.6 
Total 24 100.0   38 100.0   39 100.0 

Ability of Enterococcus spp.- and Enterococcus faecalis/faecium-specific rtPCR assays to detect colonies isolated on chromogenic culture-based methods from a sewage water sample

It is impossible to exclude that an E. faecalis or an E. faecium detected by a culture-based method was naturally found in the environment. However, some non-faecalis and non-faecium Enterococcus species are not present in feces, but are naturally and highly present in the environment. By detecting these non-fecal Enterococcus species, we cause false-positive results that could be avoided by only detecting the Enterococcus species mostly found in feces rather than relying on the entire Enterococcus genus. Furthermore, since Escherichia coli is 100 to 1,000 times more concentrated than Enterococcus spp. in feces, the probability of detecting non-E. faecalis/E. faecium in water without any detection of E. coli nor E. faecalis/E. faecium is highly improbable (Slanetz & Bartley 1957; Layton et al. 2010). Thus, the sole detection of E. faecalis and E. faecium in water could decrease the false-positive rate obtained with the detection of the entire Enterococcus genus and by this, improve the water quality assessment.

In the present study, colonies harvested on mEI agar, Chromocult® enterococci agar, and m-Enterococcus agar were identified using two specific-rtPCR assays targeting Enterococcus spp. and Enterococcus faecalis/faecium to confirm, in less than 2 hours, their identity. A positive rtPCR signal obtained with the Enterococcus faecalis/faecium-specific rtPCR assay allows the discrimination between fecal and environmental contamination of water samples.

All colonies harvested, for the present study, on mEI, Chromocult® enterococci, and m-Enterococcus agar were subjected to Enterococcus spp.- and Enterococcus faecalis/faecium-specific rtPCR assays (Table 6). Both of the rtPCR assays showed a specificity of 100% on the three media tested. Only two (2/19) colonies of E. faecium were not detected by the Enterococcus faecalis/faecium-specific rtPCR assay on mEI agar for a sensitivity of 89.5%. These results showed that the Enterococcus faecalis/faecium-specific rtPCR assay could be used with success to discriminate between fecal and environmental contamination of water samples when testing on colonies isolated on mEI, Chromocult® enterococci, or m-Enterococcus agar.

Table 6

Specificitya, sensitivityb, and predictive values of each culture-based method combined to Enterococcus spp.-, Enterococcus faecalis-, and Enterococcus faecium-specific rtPCR assays for the identification of Enterococcus spp., E. faecalis, and E. faecium, respectively, as compared to 16 s rRNA genotypic identification

 PCR16S rRNA identification
  Positive predictive 
Culture-based methodscassays+SpecificityaSensitivitybvaluedNegative predictive valuee
mEI agar (n = 22) Enterococcus spp. 
21 100% 100% 100% 100% 
− 
E. faecalis 
100% 100% 100% 100% 
− 
E. faecium 
17 100% 89.5% 100% 0% 
− 
Chromocult® enterococci agar (n = 40) Enterococcus spp. 
38 100% 100% 100% 100% 
− 
E. faecalis 
100% 100% 100% 100% 
− 
E. faecium 
20 100% 100% 100% 100% 
− 
m-Enterococcus agar (n = 39) Enterococcus spp. 
39 100% 100% 100% 100% 
− 
E. faecalis 
15 100% 100% 100% 100% 
− 
E. faecium 
12 100% 100% 100% 100% 
− 
 PCR16S rRNA identification
  Positive predictive 
Culture-based methodscassays+SpecificityaSensitivitybvaluedNegative predictive valuee
mEI agar (n = 22) Enterococcus spp. 
21 100% 100% 100% 100% 
− 
E. faecalis 
100% 100% 100% 100% 
− 
E. faecium 
17 100% 89.5% 100% 0% 
− 
Chromocult® enterococci agar (n = 40) Enterococcus spp. 
38 100% 100% 100% 100% 
− 
E. faecalis 
100% 100% 100% 100% 
− 
E. faecium 
20 100% 100% 100% 100% 
− 
m-Enterococcus agar (n = 39) Enterococcus spp. 
39 100% 100% 100% 100% 
− 
E. faecalis 
15 100% 100% 100% 100% 
− 
E. faecium 
12 100% 100% 100% 100% 
− 

aNo. of true negative results/(no. of true negative + false positive results).

bNo. of true positive results/(no. of true positive + false negative results).

cCalculated on Enterococcus spp. colony phenotype.

dNo. of true positive results/(no. of true positive + false positive results).

eNo. of true negative results/(no. of true negative + false negative results).

The results obtained in the present study were obtained using sewage water samples. Results could differ with other types of water.

The sensitivity evaluation of the three culture-based methods tested using both reference strains and environmental strains identified by 16S rRNA gene sequencing, as well as the validation of the Enterococcus spp.- and Enterococcus faecalis/faecium-specific rtPCR assays showed that the detection and identification of enterococcal colonies on Chromocult® enterococci agar combined with Enterococcus faecalis/faecium rtPCR assay presents the best combination to decrease the high false-positive rate obtained when the entire Enterococcus genus is targeted.

We wish to thank Dr Steve Charette (IBIS; Institut de Biologie Intégrative et des Systèmes, Université Laval, Québec City (Québec), Canada) for providing laboratory space and equipment as well as Philippe Cantin of the MDDELCC for providing sewage water and plates. This research was supported by AFM Water Consulting.

APHA/AWWA/WEF
2012
Standard Methods for the Examination of Water and Wastewater
. 22nd edn.
American Public Health Association, Inc.
,
Washington, DC
.
Bonds
 
B.
Christensen
 
A. B.
Flaherty
 
D. K.
2006
Enterococci species in Gulf Coast marine water samples as measured by the Environmental Protection Agency Method 1600
.
Texas Journal of Science
58
,
141
146
.
Cabelli
 
V. J.
Dufour
 
A. P.
McCabe
 
L. J.
Levin
 
M. A.
1982
Swimming-associated gastroenteritis and water quality
.
American Journal of Epidemiology
115
,
606
616
.
CEAEQ
2014
Recherche et dénombrement des entérocoques: méthode par filtration sur membrane (Research and enumeration of enterococci by membrane filtration method), MA. 700 – Ent 1.0, Rév. 5 Ministère du Développement durable, de l'Environnement et de la Lutte contre les changements climatiques du Québec, 23 pp
.
Chenoweth
 
C. E.
Schaberg
 
D.
1990
The epidemiology of enterococci
.
European Journal of Clinical Microbiology & Infectious Diseases
9
,
80
89
.
Converse
 
R. R.
Blackwood
 
A. D.
Kirs
 
M.
Griffith
 
J. F.
Noble
 
T. R.
2009
Rapid QPCR-based assay for fecal Bacteroides spp. as a tool for assessing fecal contamination in recreational waters
.
Water Research
43
,
4828
4837
.
Devriese
 
L. A.
Hommez
 
J.
Wijfels
 
R.
Haesebrouck
 
F.
1991
Composition of the enterococcal and streptococcal intestinal flora of poultry
.
Journal of Applied Bacteriology
71
,
46
50
.
Franz
 
C. M.
Holzapfel
 
W. H.
Stiles
 
M. E.
1999
Enterococci at the crossroads of food safety?
International Journal of Food Microbiology
47
,
1
24
.
Fujioka
 
R. S.
Sian-Denton
 
C.
Borja
 
M.
Castro
 
J.
Morphew
 
K.
1999
Soil: the environmental source of Escherichia coli and enterococci in Guam's streams
.
Journal of Applied Microbiology Symposium Supplement
85
,
83S
89S
.
Gelsomino
 
R.
Vancanneyt
 
M.
Cogan
 
T. M.
Swings
 
J.
2003
Effect of raw-milk cheese consumption on the enterococcal flora of human feces
.
Applied and Environmental Microbiology
69
,
312
319
.
Government of Québec
2013
Environment Quality Act. Chapter Q-2, r. 40. Regulation respecting the quality of drinking water
. .
Harwood
 
V. J.
Brownell
 
M.
Perusek
 
W.
Whitlock
 
J. E.
2001
Vancomycin-resistant Enterococcus spp. isolated from wastewater and chicken feces in the United States
.
Applied and Environmental Microbiology
67
,
4930
4933
.
Ke
 
D.
Picard
 
F. J.
Martineau
 
F.
Ménard
 
C.
Roy
 
P. H.
Ouellette
 
M.
Bergeron
 
M. G.
1999
Development of a PCR assay for rapid detection of enterococci
.
Journal of Clinical Microbiology
37
,
3497
3503
.
Kjellander
 
J.
1960
Enteric streptococci as indicators of fecal contamination of water
.
Acta Pathologica et Microbiologica Scandinavica
48
,
3
124
.
Knudtson
 
L. M.
Hartman
 
P. A.
1992
Routine procedures for isolation of enterococci and fecal streptococci
.
Applied and Environmental Microbiology
58
(
9
),
3027
3031
.
Layton
 
B. A.
Walters
 
S. P.
Lam
 
L. H.
Boehm
 
A. B.
2010
Enterococcus species distribution among human and animal hosts using multiplex PCR
.
Journal of Applied Microbiology
109
(
2
),
539
574
.
Maheux
 
A. F.
Huppé
 
V.
Boissinot
 
M.
Picard
 
F. J.
Bissonnette
 
L.
Bernier
 
J.-L. T.
Bergeron
 
M. G.
2008
Analytical limits of four β-glucuronidase and β-galactosidase-based commercial methods used to detect Escherichia coli and total coliforms
.
Journal of Microbiological Methods
75
,
506
514
.
Maheux
 
A. F.
Picard
 
F. J.
Boissinot
 
M.
Bissonnette
 
L.
Paradis
 
S.
Bergeron
 
M. G.
2009
Analytical comparison of nine PCR primer sets designed to detect the presence of Escherichia coli/Shigella in water samples
.
Water Research
43
,
3019
3028
.
Maheux
 
A. F.
Bissonnette
 
L.
Boissinot
 
M.
Bernier
 
J.-L.
Huppé
 
V.
Bérubé
 
È.
Boudreau
 
D. K.
Picard
 
F. J.
Huletsky
 
A.
Bergeron
 
M. G.
2011
Method for rapid and sensitive detection of Enterococcus spp. and Enterococcus faecalis/faecium cells in potable water samples
.
Water Research
45
(
6
),
2342
2354
.
doi:10.1016/j.watres.2011.01.019
.
Manero
 
A.
Vilanova
 
X.
Cerda-Cuellar
 
M.
Blanch
 
A. R.
2002
Characterization of sewage waters by biochemical fingerprinting of enterococci
.
Water Research
36
,
2831
2835
.
Martin
 
J. D.
Mundt
 
J. O.
1972
Enterococci in insects
.
Applied Microbiology
24
,
575
580
.
Martineau
 
F.
Picard
 
F. J.
Roy
 
P. H.
Ouellette
 
M.
Bergeron
 
M. G.
1998
Species-specific and ubiquitous-DNA-based assays for rapid identification of Staphylococcus aureus
.
Journal of Clinical Microbiology
36
,
618
623
.
Müller
 
T.
Ulrich
 
A.
Ott
 
E. M.
Muller
 
M.
2001
Identification of plant-associated enterococci
.
Journal of Applied Microbiology
91
,
268
278
.
Mundt
 
J. O.
1986
Enterococci in Bergey's Manual of Systematic Bacteriology
,
Vol. 2
, 1st edn.
William and Wilkins
,
Baltimore, MD
, pp.
1063
1065
.
Noble
 
C. J.
1978
Carriage of group D streptococci in the human bowel
.
Journal of Clinical Pathology
31
,
1182e1186
.
Picard
 
F. J.
Ke
 
D.
Boudreau
 
D. K.
Boissinot
 
M.
Huletsky
 
A.
Richard
 
D.
Ouellette
 
M.
Roy
 
P. H.
Bergeron
 
M. G.
2004
Use of Tuf sequences for genus-specific PCR detection and phylogenetic analysis of 28 streptococcal species
.
Journal of Clinical Microbiology
42
,
3686
3695
.
Ruoff
 
K. L.
de la Maza
 
L.
Murtagh
 
M. J.
Spargo
 
J. S.
Ferraro
 
M. J.
1990
Species identities of enterococci isolated from clinical specimens
.
Journal of Clinical Microbiology
28
,
435
437
.
Sinton
 
L. W.
Donnison
 
A. M.
Hastie
 
C. M.
1993
Faecal streptococci as faecal pollution indicators: a review. Part II: sanitary significance, survival and use
.
New Zealand Journal of Marine and Freshwater Research
27
,
117
137
.
Sistek
 
V.
Maheux
 
A. F.
Boissinot
 
M.
Bernard
 
K. A.
Cantin
 
P.
Cleenwerck
 
P.
Bergeron
 
M. G.
2012
Enterococcus ureasiticus sp. nov. and Enterococcus quebecensis sp. nov., isolated from water
.
International Journal of Systematic and Evolutionary Microbiology
62
,
1314
1320
.
Slanetz
 
L. W.
Bartley
 
C. H.
1957
Numbers of enterococci in water, sewage, and feces determined by the membrane filtration technique with an improved medium
.
Journal of Bacteriology
74
(
5
),
591
595
.
Svec
 
P.
Devriese
 
L. A.
Sedlacek
 
I.
Baele
 
M.
Vancanneyt
 
M.
Hesebrouck
 
F.
Swings
 
J.
Doskar
 
J.
2002
Characterization of yellow-pigmented and motile enterococci isolated from intestines of the garden snail Helix aspersa
.
Journal of Applied Microbiology
92
,
951
957
.
US Environmental Protection Agency
2005
Method 1600: Enterococci in Water by Membrane Filtration Using Membrane-Enterococcus Indoxyl-β-D-glucoside Agar (mEI), EPA 821-R-04e023 Office of Water (4303T)
.
United States Environmental Protection Agency
,
Washington, DC
.
13
pp.