As a 2010 earthquake and cholera response project, in 2013–2014, an international non-governmental organization, working with local organizations, installed water systems in two Haitian communities. In 2016, 2 years after installation, we conducted a mixed-methods evaluation to assess sustained effectiveness, including 20 infrastructure assessments, 199 household surveys, water quality testing, 30 key informant interviews, and financial analysis of two water committee records. One of 14 (7.1%) installed kiosks were functional during the evaluation, with 42–60% of survey respondents near installed sources reporting using system water for drinking. No household water sample had detectable chlorine residual and non-detectable E. coli, except when household water treatment was reported. Informants expressed appreciation for the project, and discussed difficulties with operations, maintenance, and community engagement. Revenue was initially sufficient for operations and maintenance, although not sustainable because lack of trust led to poor quality service and then lack of payment. While the functional kiosks did provide water to some residents, overall project sustained effectiveness was not reached due to operational and maintenance issues, and lack of governance and community engagement. We recommend future implementers of small-scale water systems in complex contexts like Haiti plan for the technical and social components necessary for sustained effectiveness.

  • Evaluating sustained effectiveness of water infrastructure interventions in rural Haiti.

  • Identifying factors that could improve small-scale water infrastructure/water system management in complex contexts.

  • Financial analyses of water infrastructure implementation.

Investments in drinking water infrastructure are crucial to prevent transmission of diarrheal diseases, which, in 2019, were the third leading cause of death in children <5 (IHME 2019). Despite the known benefits of safely managed water, worldwide, in 2020, one in four people lacked safely managed drinking water (defined as water from an improved source (e.g. household connections, public standpipes, boreholes), located on premises, available when needed, and free from fecal and chemical contamination) (WHO/UNICEF 2020). The remaining 25% of the world's population uses lower tiers of service, including ‘basic’ (water from an improved source less than 30 min away); ‘limited’ (water from an improved source more than 30 min away); and, ‘unimproved’ (water from an unprotected well or spring).

In Haiti, specifically, data on safely managed water sources in 2020 are not available, although 77% of the population were estimated to use improved sources in 2020 (WHO/UNICEF 2020). This lack of access to safely managed water is partially attributed to the fact that, in 2010, two major emergencies occurred in Haiti, an earthquake on January 12, 2010, that killed over 200,000 and left another 2 million people homeless, and a cholera outbreak that began in October 2010 and has since infected over 820,000 and killed nearly 10,000 people (Gelting et al. 2013; Le Ministère de la Santé Publique et de la Population (MSPP) 2020).

Following these two 2010 emergencies, there was significant international interest and response in providing safe water and sanitation to the Haitian population. The humanitarian response to the combined Haitian emergencies led to more than 50 million US dollars investments in water, sanitation, and hygiene (WASH) interventions between 2011 and 2012, and over 100 non-governmental organizations (NGOs) implementing WASH programs were identified in Haiti in 2011 (Financial Tracking Services 2011; Gelting et al. 2013).

Therefore, between November 2013 and August 2014, with funding from Cartier Philanthropy (previously Cartier Charitable Foundation), an international non-governmental organization (INGO) completed a water, sanitation, and hygiene (WASH) program that included water source protection, water supply infrastructure development, centralized water treatment, and additional WASH activities in two rural communities in Haiti that lacked safely managed water at the time. Cartier Philanthropy framed this programming as an emergency response project related to the 2010 Haitian earthquake with the goal of improving access to clean water and sanitation. Of note, in 2013–2014, when this water infrastructure program was implemented, 63.2% of the Haitian population had access to at least basic water (WHO/UNICEF 2020), and 5,055 deaths were attributed to diarrhea (IHME 2019). Additionally, although the SDGs and safely managed water were not the current metrics, it was known that these would be the metrics, and Cartier Philanthropy was interested in meeting long-term sustainable development goals.

In Community A, the INGO contracted with local organizations to build a gravity-fed water distribution system including a spring catchment, chlorination, and distribution via four kiosks. This system was intended to provide >7,500 people with safe drinking water. In Community B, a water distribution system was built with a filtration and chlorination system to provide more than 10,000 people with safe drinking water from 10 kiosks and 800 private connections. In both communities, water committees were established and trained with the goal of ensuring effective management and maintenance of water infrastructure. Additionally, it was planned that the recently established (in 2009) Haitian government agency La Direction Nationale de l'Eau Potable et de l'Assainissement (DINEPA) would oversee water system operations and management after project completion. Note the communities are not named herein to protect informant confidentiality, and are generally representative of communities needing safely managed water.

Achieving sustained effectiveness of water systems in low- and middle-income countries (LMICs) is difficult. Less sustainable systems are older, further from urban areas, and lack funding, accountability, regular maintenance, access to supplies, technical expertise, community management systems, functional water committees, gender equity in water committees, fee collection systems, trust between users and system operators, and user demand and usage (Foster 2013; Rayner et al. 2016a, 2016b). More sustainable water systems were ones including appropriate technologies, management schemes, fee collection, and regular repair and maintenance (Fisher et al. 2015; Bartram & Cronk 2017). In complex emergency situations specifically (such as in Haiti with successive disasters, epidemics, and political emergencies), successes in sustainable water infrastructure include training of mechanics, supporting water committees, selecting simple technologies, and providing support after initial implementation (ACF 2007). Lack of engagement with local institutions and local capacity building were identified as barriers to sustained effectiveness because they led to technical failures (in system management) and social failures (in not understanding social relationships and community dynamics).

In this evaluation, we assessed the sustained effectiveness of the Cartier Philanthropy project in 2016, 2 years after project completion in 2014, using both international and donor metrics.

To assess sustained effectiveness, we conducted one mixed-method evaluation, including infrastructure assessments, household surveys, key informant interviews (KIIs), and financial analysis. Data were triangulated across mixed methods in analysis to determine international and donor metrics were met. This evaluation was approved by the Tufts University (1516-20) and Haitian Institutional Review Boards (1512007). Written informed consent was obtained before each key informant interview and household survey.

Infrastructure assessments

In Communities A and B, each source, reservoir, and kiosk were visited with a community member informant involved with installation and assessed. The following data were collected: global positioning system (GPS) point, system operational capacity and condition data, availability of spare parts, consumables information, and water samples from kiosks when available.

Household surveys

Household surveys were carried out in Haitian Creole by trained Haitian enumerators. In each community, enumerators selected houses by beginning at a kiosk, then visiting every other house until 10 household surveys were completed (maximum 140 households). In Community B, an additional 80 households were randomly selected from a list of 228 households that paid private connection deposits (maximum 80 households). At each house, a survey was conducted, consisting of 77 questions on demographics, drinking water knowledge and practices, drinking water access and storage, and knowledge about the project. Enumerators followed a survey guide with close-ended questions that offered several options to choose from, questions that required enumerators to observe participants' actions after cues were given (i.e. ‘Can you show me where you wash your hands?’), and open-ended questions with associated probes. During the survey, a sample of stored household drinking water was collected. Survey data were entered into Microsoft Excel (Redmond, WA, USA) and analyzed using Stata 10.1 (College Station, TX, USA).

Water quality testing

Water samples from kiosks and households were analyzed for free chlorine residual (FCR) and E. coli. FCR was tested using a Hach® ColorWheel Test Kit (range 0.0–3.5 mg/L) and DPD-1 tablets (Loveland, CO, USA). Results ≥0.2 mg/L were considered to meet international guidelines (WHO 2011). To test E. coli, 100 mL samples were collected aseptically in Whirl-Pak™ bags with sodium thiosulfate (Nasco, Ft. Atkinson, WI, USA), stored on ice, and tested within 12 h of collection. Samples were tested using membrane filtration: sample water was diluted appropriately with sterile buffered water and then filtered aseptically through a 45-μm Millipore filter (Billerica, MA, USA) using a portable Millipore filtration stand. Filters were then placed in a plastic petri dish with a mColiBlue24® media soaked pad, and incubated at 35–37 °C for 24 h in a portable incubator. For quality control, sterile buffered water controls were tested every 20 plates, and 10% of samples were duplicated. Results were converted to CFU (colony forming units)/100 mL, and categorized according to World Health Organization (WHO) risk categories (WHO 2011).

Key informant interviews

KIIs were conducted with persons involved in project implementation, operation, and management, including members of the Water Committees. Interviews were semi-structured, with the interview guide consisting of 31 open-ended questions including personal background, water and sanitation knowledge, role in the project, opinions about project implementation, and capacity building activities of the project. Interview responses were recorded on paper forms, entered into Microsoft Excel, and coded into emergent themes in Excel by staff who speak Haitian Creole fluently. Key quotations were extracted and included herein.

Financial analysis

During KIIs, water committee financial reports were collected. Data were analyzed for revenue compared with expenses, revenue source distribution, and cash balance at the last report.

Sustained effectiveness

To assess sustained effectiveness, outcome metrics (co-developed between Cartier Philanthropy and Tufts University) were assessed, including relevance, effectiveness, efficiency, sustainability, and impact. Relevance was measured using responses from KIIs and the percentage of households using water provided by the system (from household surveys). Effectiveness was measured using infrastructure assessments to determine what was operational with detectable FCR and no detectable E. coli at the time of the survey. Efficiency was measured using household surveys as the percentage of households using water provided by the system with detectable FCR and no detectable E. coli. Sustainability was measured as the (1) percentage of infrastructure with sufficient spare parts, consumables, and ongoing operation and maintenance; (2) percentage of communities with operational water committees (KIIs, financial analysis), (3) percentage of communities with operational bank accounts with user fees (financial analysis), (4) percentage of communities with DINEPA or Mayoral involvement or support (KIIs), and (5) local organization empowerment as a result of program (KIIs). Impact was intended to be measured as improvements in water quality and health outcomes.

From March 22 to April 19, 2016, we carried out 20 infrastructure assessments, 199 household surveys, 30 KIIs, and collected two water committee financial report assessments. Results are presented below.

Infrastructure assessments

In Community A, a water distribution system was installed, including a capped spring flowing into a concrete reservoir (both in good condition) then through a distribution system to four kiosks. During the assessment, one of four kiosks (25%) were operational, and the one operational kiosk lacked stock of sufficient spare parts and chlorine consumables. Informants who visited the infrastructure systems with researchers stated that: two of the non-operational kiosks would function if valves were opened; there was at least one kiosk where water never reached due to potential unsound hydraulic system design; and/or, kiosks were closed because the population stopped paying for water and/or kiosk operators were not receiving their monthly stipend. Sample water tested from the operational kiosk had non-detectable FCR and was ‘medium risk’, with 21 E. coli CFU/100 mL.

In Community B, a spring box was built in the mountains, and water flowed via a sand filtration system and reservoir with a chlorination system into the community. The spring box and reservoir infrastructure appeared in good condition; however, the sand filtration system had not been recently cleaned. The distribution system was designed to provide water for 10 kiosks and household connections. No kiosks (0%) were operational during the evaluation because: water did not reach the kiosks; there were broken pipes and/or tanks; and/or, they were not opened by operators. Thus, no water samples were collected. Additionally, the non-operational kiosks also lacked sufficient spare parts and chlorine consumables in stock. Informants stated that 7 of 10 (70%) kiosks could be operational as is, or with minor repairs, that pressure decreased once private connections were installed, that there was superfluous air in the water system, and that spring box water was diverted to supply ‘The Fountain’, a water point for mountainous populations. The Fountain had continuous water flow, was in good condition except for a broken PVC pipe, and had non-detectable FCR and non-detectable E. coli.

Household surveys

Demographics

Overall, 199 households surveys were conducted, including 10 around each kiosk in Community A (40) and Community B (100), and 59 private connections in Community B-private. Average educational attainment was 5.7, 7.6, and 8.3 years in Communities A, B, and B-private, respectively (Table 1). Household cell phone ownership was 48% in Community A, and 92–97% in Communities B and B-private.

Table 1

Demographics, WASH knowledge and practices, and water access

CommunityA (n = 40)B (n = 100)B-private (n = 59)
Demographics 
 Mean (SD) people per household 5.1 (2.4) 6.9 (3.5) 5.2 (2.4) 
 Female respondent, % (n48% (40) 72% (100) 56% (59) 
 Respondent attended school, % (n58% (40) 81% (99) 95% (59) 
 Mean (SD) highest grade, (n5.7 (2.4) (23) 7.6 (3.1) (75) 8.3 (3.7) (47) 
 Reports female head-of-household can read, % (n35% (40) 57% (100) 81% (59) 
 Reports male head-of-household can read, % (n48% (40) 60% (100) 76% (59) 
 Connected to electrical grid, % (n0% (40) 56% (99) 91% (56) 
 Has at least one cellular phone, % (n48% (40) 92% (92) 97% (59) 
 Has at least one motorcycle, % (n0% (40) 15% (92) 44% (59) 
WASH Knowledge and Practices 
 Believes water can make you sick, % (n100% (37) 94% (99) 97% (59) 
 Believes water in house is safe to drink, % (n97% (37) 92% (93) 88% (58) 
 Top reported ways to prevent diarrhea 
  Drink safe water, % (n50% (40) 76% (100) 80% (59) 
  Wash hands, % (n35% (40) 54% (100) 47% (59) 
  I don't know, % (n40% (40) 14% (100) 10% (59) 
 Has on plot or shared latrine, % (n28% (40) 94% (100) 98% (59) 
 Has dedicated hand washing place, % (n0% (39) 9% (96) 31% (51) 
 Reports washes hands after using toilet, % (n40% (40) 73% (100) 75% (59) 
Water Access, Treatment, and Storage 
 Water storage container covered, % (n85% (34) 93% (98) 92% (53) 
 Provided drinking water sample, % (n75% (40) 93% (100) 85% (59) 
 Households with drinking water from system, % (n60% (40) 58% (100) 42% (59) 
 Reported treated at household, % (n17% (30) 23% (93) 18% (50) 
  With chlorine product, % (n80% (5) 95% (21) 100% (9) 
  With filter, % (n0% (5) 5% (21) 0% (9) 
  Don't know/Other, % (n20% (5) 0% (21) 0% (9) 
 Households with drinking/non-drinking water from system % (n63% (40) 72% (100) 92% (59) 
System Use 
 Report paying for water, % (n20% (40) 45% (100) 90% (59) 
 Mean monthly payment for water (SD), n 25 (0) n = 8 157 (60.0) n = 36 199 (53.3) n = 53 
 Received information about the system, % (n50% (40) 35% (100) 48% (59) 
 Could name contact for questions, % (n26% (38) 33% (96) 60% (53) 
 Heard about hygiene activities, % (n43% (40) 57% (98) 45% (58) 
 Participated in hygiene activities, % (n28% (40) 7% (99) 12% (58) 
System Perceptions 
 Comments about project 
  It's a good thing, % (n45% (40) 43% (100) 41% (59) 
  Before we didn't have access to water, % (n22% (40) 33% (100) 25% (59) 
  The source is protected now, % (n38% (40) 10% (100) 0% (59) 
  The water is treated, % (n22% (40) 10% (100) 0% (59) 
  Water isn't treated/safe, % (n16% (40) 14% (100) 30% (59) 
  It's not well managed, % (n22% (40) 29% (100) 25% (59) 
  A distribution schedule is needed, % (n0% (40) 22% (100) 28% (59) 
  Water is not given fairly, % (n0% (40) 20% (100) 18% (59) 
  Distribution is not frequent enough, % (n0% (40) 44% (100) 46% (59) 
  It's too expensive, % (n22% (40) 14% (100) 25% (59) 
CommunityA (n = 40)B (n = 100)B-private (n = 59)
Demographics 
 Mean (SD) people per household 5.1 (2.4) 6.9 (3.5) 5.2 (2.4) 
 Female respondent, % (n48% (40) 72% (100) 56% (59) 
 Respondent attended school, % (n58% (40) 81% (99) 95% (59) 
 Mean (SD) highest grade, (n5.7 (2.4) (23) 7.6 (3.1) (75) 8.3 (3.7) (47) 
 Reports female head-of-household can read, % (n35% (40) 57% (100) 81% (59) 
 Reports male head-of-household can read, % (n48% (40) 60% (100) 76% (59) 
 Connected to electrical grid, % (n0% (40) 56% (99) 91% (56) 
 Has at least one cellular phone, % (n48% (40) 92% (92) 97% (59) 
 Has at least one motorcycle, % (n0% (40) 15% (92) 44% (59) 
WASH Knowledge and Practices 
 Believes water can make you sick, % (n100% (37) 94% (99) 97% (59) 
 Believes water in house is safe to drink, % (n97% (37) 92% (93) 88% (58) 
 Top reported ways to prevent diarrhea 
  Drink safe water, % (n50% (40) 76% (100) 80% (59) 
  Wash hands, % (n35% (40) 54% (100) 47% (59) 
  I don't know, % (n40% (40) 14% (100) 10% (59) 
 Has on plot or shared latrine, % (n28% (40) 94% (100) 98% (59) 
 Has dedicated hand washing place, % (n0% (39) 9% (96) 31% (51) 
 Reports washes hands after using toilet, % (n40% (40) 73% (100) 75% (59) 
Water Access, Treatment, and Storage 
 Water storage container covered, % (n85% (34) 93% (98) 92% (53) 
 Provided drinking water sample, % (n75% (40) 93% (100) 85% (59) 
 Households with drinking water from system, % (n60% (40) 58% (100) 42% (59) 
 Reported treated at household, % (n17% (30) 23% (93) 18% (50) 
  With chlorine product, % (n80% (5) 95% (21) 100% (9) 
  With filter, % (n0% (5) 5% (21) 0% (9) 
  Don't know/Other, % (n20% (5) 0% (21) 0% (9) 
 Households with drinking/non-drinking water from system % (n63% (40) 72% (100) 92% (59) 
System Use 
 Report paying for water, % (n20% (40) 45% (100) 90% (59) 
 Mean monthly payment for water (SD), n 25 (0) n = 8 157 (60.0) n = 36 199 (53.3) n = 53 
 Received information about the system, % (n50% (40) 35% (100) 48% (59) 
 Could name contact for questions, % (n26% (38) 33% (96) 60% (53) 
 Heard about hygiene activities, % (n43% (40) 57% (98) 45% (58) 
 Participated in hygiene activities, % (n28% (40) 7% (99) 12% (58) 
System Perceptions 
 Comments about project 
  It's a good thing, % (n45% (40) 43% (100) 41% (59) 
  Before we didn't have access to water, % (n22% (40) 33% (100) 25% (59) 
  The source is protected now, % (n38% (40) 10% (100) 0% (59) 
  The water is treated, % (n22% (40) 10% (100) 0% (59) 
  Water isn't treated/safe, % (n16% (40) 14% (100) 30% (59) 
  It's not well managed, % (n22% (40) 29% (100) 25% (59) 
  A distribution schedule is needed, % (n0% (40) 22% (100) 28% (59) 
  Water is not given fairly, % (n0% (40) 20% (100) 18% (59) 
  Distribution is not frequent enough, % (n0% (40) 44% (100) 46% (59) 
  It's too expensive, % (n22% (40) 14% (100) 25% (59) 

WASH knowledge and practices

Most respondents knew water can make you sick (94–100%), thought their water was safe to drink (88–97%), and covered their stored water (85–93%) (Table 1). Access to on-plot or shared latrines was 28% in Community A, and 94–98% in Communities B and B-private. Having a dedicated handwashing place was 0% in Community A, 9% in Community B, and 31% in Community B-private.

Water access, treatment, and storage

In total, 63, 72, and 92% of households had stored household INGO system water during the survey in Community A (from the single operational kiosk), B (from The Fountain or neighbors household connections), and B-private (from private connections), respectively (Table 1). However, fewer households, 60, 58, and 42%, reported using that water for drinking in Communities A, B, and B-private, respectively.

System use

In Community A, 20% of respondents reported paying an average 25 HTG (USD 0.40) per month for water (80% reported not paying) (Table 1). In Communities B and B-private, 45 and 90% of respondents reported paying an average 157 HTG (USD 2.60) and 199 HTG (USD 3.30), respectively. Overall, 14–25% of respondents reported water costs were too high.

In Communities A, B, and B-private, 50, 35, and 48% of respondents reported having received information about the INGO system, and 26, 33, and 60% could name someone to contact if they had questions, respectively (Table 1). Overall, 43, 57, and 45% of respondents had heard about hygiene activities, with 38, 7, and 12% reporting having participated, in Communities A, B, and B-private, respectively.

System perceptions

Overall, 45, 43, and 41% from Communities A, B, and B-private, respectively, reported believing that the INGO project was a good project, because there was no water before, the source is protected, and water is treated (Table 1). However, 14–30% of respondents stated the water is not safe and the system is not well managed, and in Communities B and B-private, respondents stated a distribution schedule is needed, distribution is not frequent enough, and water is not given fairly. 17, 23, and 18% of respondents reported treating their stored drinking water sample at home in Communities A, B, and B-private, respectively (Table 1). Of those respondents reporting treatment, 80, 95, and 100% of respondents, respectively, reported household chlorination.

Water quality

FCR was detected in 0% of household water samples in Community A, regardless of reported household water treatment. In Communities B and B-private, FCR was detected only in samples from households reporting household chlorination.

Geometric mean E. coli in household stored drinking water from the INGO system was 10.9, 33.5, and 9.5 CFU/100 mL without household treatment, and 11.3, 10.7, and 1.1 CFU/100 mL with reported household treatment, in Communities A, B, and B-private, respectively.

When categorized by WHO risk category, 4, 10, and 36% of household samples from the INGO system met the WHO guideline value of <1 E. coli CFU/100 mL in Communities A, B, and B-private water samples, respectively. An additional 42, 26, and 32% of household samples from the INGO system were <10 E. coli CFU/100 mL, leading to 46, 36, and 68% of household samples from the INGO system (in Communities A, B, and B-private, respectively) considered ‘low risk’.

Summary

Overall, survey results show that demographics were different across the study populations, with respondents in Communities B, and especially B-private, having higher education, knowledge, and indicators of wealth than in Community A. While 63, 72, and 92% of respondents in Communities A, B, and B-private could provide a sample of the INGO system water from their household during the evaluation, only 60, 58, and 42%, respectively, reported using that water for drinking, and a number of households treated that water before drinking. No sample from a household with INGO system water not treated with household water treatment met standards of safe drinking water: detectable FCR; and, non-detectable E. coli per 100 mL sample.

Key informant interviews

Nine themes emerged in KIIs with project and community leaders; each described below.

General feedback

Community A informants expressed appreciation for the project, stating the water system is too important to fall into disrepair, and they appreciated water being treated, improved access in remote areas, and employment opportunities. Community B informants said the water system was an ‘exceptional asset’ and a ‘gwo, gwo bagay’ (very, very big thing), and appreciated ‘anyone can drink the water’, employment opportunities, and time gained to earn money and attend school.

Engagement of authorities

In rural areas, DINEPA delegates water system management to water committees, and has guidelines on water committee establishment process, structure, and operation. In Community A, DINEPA was described as a ‘supplier of knowledge’, especially for leading water committee trainings. Informants also stated local political leaders were involved, but their lack of support was an implementation weakness. In Community B, DINEPA also led trainings and assisted water committees with activities (e.g. submitting reports). Informants stated the Mayor's office was involved, but lacked technical expertise, which prevented them from effectively engaging in the project.

Community engagement

In Community A, informants stated there was little community involvement in community meetings or to discuss water system maintenance. Informants expressed frustration that local workers with adequate skills were not hired for the project. In Community B, community members participated in constructing the water system, providing land, and attended community meetings.

Capacity building and empowerment

In Community A, the water committee received technical, management, public health, and social training from DINEPA and a local NGO. Informants expressed they did not receive sufficient training and material resources to function properly, stating the water committee was: ‘like children sent to school by their parents without being fed’ and ‘like orphans, like children without mothers and fathers’. In Community B, the water committee received accounting, management, and technical training from DINEPA and a local NGO. In Community B, informants reported acquiring useful skills in the trainings.

Water committees

Community A informants perceived the water committee as being ineffective at organizing community meetings and convincing the population to pay. An informant stated the current members ‘are there, just looking, not contributing actively’. Community B informants described the water committee as dysfunctional, with individual respondents stating ‘tèt anba’ (upside down), ‘kraze’ (broken), ‘jèsyon zero’ (zero management), and ‘the committee doesn't exist’. Both committees also reported significant inter-personal conflicts as well.

Operations and maintenance

Community A informants stated some residents were trained to operate the chlorinator and test FCR, and a kiosk operator was supplied chlorine tablets. During the evaluation, it had been a ‘while’ (but <1 year) since the system had been chlorinated, and 3–4 months since there had been chlorine tablets. In Community B, informants reported issues regarding who should cover the chlorine manager's transportation and chlorine purchase costs.

Recommendations

Community A informants recommended centering human relationships and community culture during project development, and having more time and input from local authorities. Community B informants recommended: hiring an engineer to address technical problems; clarifying who is responsible for specific costs; facilitating better communication between stakeholders; and, implementing different methods of encouraging people to pay for water.

Sustainable financing

In Community A, system operation was intended to be funded by user fees. Informants stated that, when the system was first implemented, about half the population paid. However, while it is Haitian policy that populations pay for water, some informants stated that: people did not want to, or could not afford to, pay. Informants attributed this to knowledge gaps and that water was free in nearby areas. In Community B, system operation was intended to be funded by user fees, but people with private connections gave free water to their neighbors. Additionally, people did not pay because they believed water should be free, the fee was too high, service quality was poor, and finances were perceived as mismanaged.

Financial analysis

In Community A, during the evaluation in March/April 2016, financial reports from the beginning of system operation in November 2014 to March 2015 were obtained that included trends in revenues, expenses, and revenue distribution throughout the reporting period (Figure 1). Water fees of 25 HTG (0.57 USD) per household were intended to be collected from 906 households per month. Revenue exceeded expenses in Community A during four of five reporting periods, and in January 2015, the month with the largest amount of revenue, when 254 households paid water fees. Total revenue exceeded total expenses by 43%, resulting in a cash balance of 6,790 HTG (142.35 USD) on March 31, 2015.
Figure 1

Community A revenue and expenses over time.

Figure 1

Community A revenue and expenses over time.

Close modal
In Community B, during the evaluation in March/April 2016, monthly financial reports from beginning of project operations in July 2014 to December 2015 (most current) were obtained from water committees, and included monthly revenue and expenses, monthly income by source, and average revenue source and expense category (Figure 2). Total revenue increased from the start of fee collection until peaking in March 2015 and decreasing thereafter. Overall, revenue exceeded expenses in 5 of 18 months. Monthly water fees were 200 HTG (4.41 USD) and made up 61% of revenue, one-time private connection fees were 600 HTG (13.23 USD) and made up 38% of revenue, and kiosks were the remaining 1% of revenue. The cash balance on December 31, 2015 was 16,485 HTG (363.59 USD). When comparing monthly revenue to monthly expenses (Figure 2(c)), monthly expenses ranged from a high of 49,345 HTG (1,087.48 USD) in September 2014 to a low of 7,797 HTG (139.46 USD) in December 2015. Salaries accounted for 63% of expenses, followed by 19% for repairs. Expenses declined over time because of salary reductions. Total revenue exceeded total expenses by 11% over the reporting period. If one-time private connection fees are excluded, 69% of recurring costs were recovered through recurring monthly water fees and kiosk revenue, sufficient to cover salaries and some repairs.
Figure 2

Community B financial information, including: (a) revenue and expenses; (b) monthly income by source over time; and, (c) average revenue source and expense category.

Figure 2

Community B financial information, including: (a) revenue and expenses; (b) monthly income by source over time; and, (c) average revenue source and expense category.

Close modal

Overall, for the time period, records were available, revenue was sufficient to operate and maintain the systems (when connection fees were included), as total revenue exceeded total expenses. Thus, there was some willingness to pay for the INGO systems, and lack of payments or decrease in payments occurred over time (as discussed in the KII section above) due to lack of trust in the water committee (including inter-personal conflicts), unwillingness (of some) to ever pay for water, and of the remaining population to continue to pay for unreliable water as system reliability declined.

Sustained effectiveness

Quantitative and qualitative data were analyzed for five key outcome metrics.

For relevance, in both communities, the installed infrastructure was highly appreciated. Positive comments by informants were the description as ‘gwo, gwo bagay’ (a very, very big thing) for the population and one informant stating, ‘I don't know how to say thank you to Cartier. I don't know where we would be today without them – we might not have water’. Overall, 63–92% of respondents had household stored water from the INGO system during the evaluation. However, only 42–60% of households used INGO system water for drinking, and many treated that water before drinking.

For effectiveness, only one of 14 kiosks was operational, and the Fountain and some household connections. It was unclear whether systems were designed correctly, and if all kiosks could be operational. No sample (0/2, 0%) from INGO system infrastructure met standards of having detectable FCR and non-detectible E. coli.

For efficiency, no sample (0/117, 0%) from a household with INGO system water that was not treated using household water treatment had detectable FCR and non-detectable E. coli.

For sustainability, neither INGO system (0/2, 0%) had sufficient spare parts, chlorine consumables in stock, an ongoing operation and maintenance plan, and/or an operational water committee. While in both communities, water management committees had been formed, received training, and created reports; neither water committee was fully functioning during the evaluation, and both reported within committee inter-personal conflicts. Additionally, both committees reported operational bank accounts, and analysis found sufficient fees collected to maintain average system maintenance for a time. However, lack of community trust in the water committee's ability to manage funds led to lack of sustainable fee collection and ongoing maintenance. Both INGO projects had local government involvement and support, and it was reported that local organizations were empowered by gaining project management skills such as prioritization and time management.

For impact, improvements in water quality and health outcomes were not able to be assessed.

Two years after systems were operational, in 2016, we carried out one mixed-methods evaluation to assess the sustained effectiveness of water systems implemented in Haiti in two communities, installed by a donor in response to the 2010 earthquake and cholera outbreak. We found that the projects were generally appreciated by key informant interviewees, and 42–60% of household survey respondents near the installed sources in Communities A, B, and B-private reporting relying on the INGO water system as their primary drinking source. However, neither system was reaching the populations as intended, and only one of four kiosks in Community A and none of ten kiosks in Community B were operational during the evaluation of two years after installation. Household connections and one alternate source were available in Community B. No sample met the WHO's safe drinking water guidelines of detectable FCR and non-detectable E. coli, except in households reporting chlorine-based household water treatment.

Operations and maintenance issues included both technical and systemic issues. Technical issues such as lack of water pressure and air in the system likely prevented water from reaching kiosks and private connections. Systemic issues included negative feedback loops stemming from lack of trust. Due to a feeling they should not need to pay for water, lack of trust in the water committee (including with financial management), and subsequent unreliability of the kiosk water distribution more and more community members did not feel they should pay for water, which led to supplies running short and undercompensated staff, which then led to poorer service and management, ultimately reinforcing community sentiments of not wanting to pay for unreliable services. The World Bank highlights this negative feedback loop in WASH interventions in LMIC, when water infrastructure implementers lack expertise and resources to reach sustained effectiveness, resulting in low-quality services at a price that consumers are not willing to pay (Goksu et al. 2019). This is exacerbated because DINEPA was founded in 2009, and while informants mentioned DINEPA provided value to the program, it is not clear that DINEPA could reasonably be expected to assume program operations. While the systems could have been brought to operation with minor repairs, and there was user payment for a time, the system did not meet initial high expectations and much of it fell into disrepair.

Sustained effectiveness of water infrastructure interventions is more difficult to achieve without adequate governance. Inadequate governance is common in complex and emergency-impacted communities in Haiti and South Sudan (Koski-Karell et al. 2016; Erismann et al. 2019). However, data from Bolivia, Ghana, Kenya, Peru, and Zambia shows, in the absence of governance, community engagement can be a means to implement successful small-scale water infrastructure interventions (Whittington et al. 2009; Kelly et al. 2017). When prioritized, community engagement can foster a sense of ownership, empowerment, and social capital, and create meaningful social and economic opportunities or community members, all of which can enable long-term sustainability (Kelly et al. 2017). Community engagement requires resources, time, and community trust, however, which can be difficult to achieve in complex emergency contexts like Haiti (Quintanilla 2015). Ongoing factors in Haiti such as extreme poverty, political volatility, government mistrust, and weak institutional capacity all likely impacted execution and sustainability of this project.

Our results are limited by lack of clinical records to assess whether water system installation lowered diarrheal disease rates. As shown in previous studies, it is difficult to conduct rigorous epidemiological evaluations of water supply improvements as people live, go to school, use water resources, and receive medical attention in different catchments (Matheson et al. 2012; deReil et al. 2018). Additionally, although to our knowledge this study was unique in evaluating actual water committee financial reports, individual bank account transactions were not available for analysis. Evaluating financial records of water infrastructure interventions can provide information about implementing sustainable revenue collection models (Jones 2010; Foster & Hope 2016). Lastly, funding was allocated only to evaluate this project once in 2016, so we could not compare our results to results from other projects implemented in Haiti during this time or this project across times. As this intervention was conducted after the 2010 earthquake and cholera outbreak in Haiti, the majority of assistance was allocated to short-term relief versus long-term projects. However, as this project was implemented and evaluated years after the earthquake, some may not consider this an earthquake-response project. Additionally, we feel these limitations did not impact the results of our study, and recommend further research on health impact of infrastructure and water committee financial management.

Based on our results and pre-existing literature, we recommended to Cartier Philanthropy, and we recommend to implementers and governments considering building infrastructure after emergencies in complex contexts, to: incorporate a budget for adaptations/contingencies into the initial project design to increase flexibility; conduct scoping evaluations to identify key stakeholders and develop sector and community engagement plans; invest heavily in community engagement and institutional capacity strengthening; budget for operations and maintenance; plan an exit strategy, including a transition phase where the system is operated and managed locally but still has external support; and, conduct monitoring and evaluation activities during and post-project.

These recommendations, which came from Haiti, are relevant to implementing infrastructure across complex emergency contexts, as the world strives to meet the SDGs by 2030, and provide safely managed drinking water for all.

We would like to thank the anonymous implementer of this project, who allowed us to evaluate the intervention, review the manuscript, and provide feedback.

All relevant data are included in the paper or its Supplementary Information.

The authors declare there is no conflict.

ACF–International Network & ACF-France, Technical Department
2007
How to Make WASH Projects Sustainable and Successfully Disengage in Vulnerable Contexts: A Practical Manual of Recommendations and Good Practices Based on a Case Study of Five ACF-IN Water, Sanitation, and Hygiene Projects
.
Bartram
J.
&
Cronk
R.
2017
Factors influencing water system functionality in Nigeria and Tanzania: a regression and Bayesian network analysis
.
Environmental Science & Technology
51
.
https://doi.org/10.1021/acs.est.7b03287.
deRiel
E.
,
Puttkammer
N.
,
Hyppolite
N.
,
Diallo
J.
,
Wagner
S.
,
Honoré
J. G.
,
Balan
J. G.
,
Celestin
N.
,
Vallès
J. S.
,
Duval
N.
,
Thimothé
G.
,
Boncy
J.
,
Coq
N. R. L.
&
Barnhart
S.
2018
Success factors for implementing and sustaining a mature electronic medical record in a low-resource setting: a case study of iSanté in Haiti
.
Health Policy and Planning
33
(
2
),
237
246
.
https://doi.org/10.1093/heapol/czx171
.
Erismann
S.
,
Gürler
S.
,
Wieland
V.
,
Prytherch
H.
,
Künzli
N.
,
Utzinger
J.
&
Peterhans
B.
2019
Addressing fragility through community-based health programmes: insights from two qualitative case study evaluations in South Sudan and Haiti
.
Health Research Policy and Systems
17
(
1
),
20
.
https://doi.org/10.1186/s12961-019-0420-7
.
Financial Tracking Service
2011
Haiti Emergencies for 2011: List of All Humanitarian Pledges, Commitments and Contributions in 2011
.
United Nations (UN) Office for the Coordination of Humanitarian Affairs (OCHA)
,
New York
.
Fisher
M. B.
,
Shields
K. F.
,
Chan
T. U.
,
Christenson
E.
,
Cronk
R. D.
,
Leker
H.
,
Samani
D.
,
Apoya
P.
,
Lutz
A.
&
Bartram
J.
2015
Understanding handpump sustainability: determinants of rural water source functionality in the Greater Afram Plains region of Ghana
.
Water Resources Research
51
(
10
),
8431
8449
.
https://doi.org/10.1002/2014WR016770
.
Foster
T.
2013
Predictors of sustainability for community-managed handpumps in sub-Saharan Africa: evidence from Liberia, Sierra Leone, and Uganda
.
Environmental Science & Technology
47
(
21
),
12037
12046
.
https://doi.org/10.1021/es402086n
.
Foster
T.
&
Hope
R.
2016
A multi-decadal and social-ecological systems analysis of community waterpoint payment behaviours in rural Kenya
.
Journal of Rural Studies
47
,
85
96
.
https://doi.org/10.1016/j.jrurstud.2016.07.026
.
Gelting
R.
,
Bliss
K.
,
Patrick
M.
,
Lockhart
G.
&
Handzel
T.
2013
Water, sanitation and hygiene in Haiti: past, present, and future
.
The American Journal of Tropical Medicine and Hygiene
89
(
4
),
665
670
.
https://doi.org/10.4269/ajtmh.13-0217
.
Goksu
A.
,
Bakalian
A.
,
Kingdom
B.
,
Saltiel
G.
,
Mumssen
Y.
,
Soppe
G.
,
Kolker
J.
&
Delmon
V.
2019
Reform and Finance for the Urban Water Supply and Sanitation Sector
.
World Bank Group
,
Washington DC
.
Institute for Health Metrics and Evaluation (IHME)
2019
GBD Results
.
Global Burden of Disease Study
.
Jones
S.
2010
Community financing of handpump maintenance: a case study in rural Mali . Paper Presented at the IRC Symposium ‘Pumps, Pipes and Promises: Costs, Finances and Accountability for Sustainable WASH Services’, in The Hague, The Netherlands from 16–18 November. IRC
.
Kelly, E., Lee, K., Shields, K. F., Cronk, R., Behnke, N., Klug, T. & Bartram, J. 2017 The role of social capital and sense of ownership in rural community-managed water systems: qualitative evidence from Ghana, Kenya, and Zambia. Journal of Rural Studies 56, 156–166. https://doi.org/10.1016/j.jrurstud.2017.08.021.
Koski-Karell
V.
,
Farmer
P. E.
,
Isaac
B.
,
Campa
E. M.
,
Viaud
L.
,
Namphy
P. C.
,
Ternier
R.
&
Ivers
L. C.
2016
Haiti's progress in achieving its 10-year plan to eliminate cholera: hidden sickness cannot be cured
.
Risk Management and Healthcare Policy
9
,
87
100
.
https://doi.org/10.2147/RMHP.S75919
.
Matheson
A. I.
,
Baseman
J. G.
,
Wagner
S. H.
,
O'Malley
G. E.
,
Puttkammer
N. H.
,
Emmanuel
E.
,
Zamor
G.
,
Frédéric
R.
,
Coq
N. R.
&
Lober
W. B.
2012
Implementation and expansion of an electronic medical record for HIV care and treatment in Haiti: an assessment of system use and the impact of large-scale disruptions
.
International Journal of Medical Informatics
81
(
4
),
244
256
.
https://doi.org/10.1016/j.ijmedinf.2012.01.011
.
Ministere Sante Publique et de la Population (MSPP)
2020
Rapport du Réseau National de Surveillance: Choléra (p. 17) [Government]. Direction d'Epidémiologie, des Laboratoires et de la Recherche (DELR)
.
Quintanilla
J.
2015
Ten Lessons on Communicating with Communities in Complex Emergencies; Humanitarian Practice Network
:
Rayner
J.
,
Murray
A.
,
Joseph
M.
,
Branz
A.
&
Lantagne
D.
2016a
Evaluation of household drinking water filter distribution programs in Haiti
.
Journal of Water, Sanitation and Hygiene for Development
6
(
1
),
42
54
.
https://doi.org/10.2166/washdev.2016.121
.
Rayner
J.
,
Yates
T.
,
Joseph
M.
&
Lantagne
D.
2016b
Sustained effectiveness of automatic chlorinators installed in community-scale water distribution systems during an emergency recovery project in Haiti
.
Journal of Water, Sanitation and Hygiene for Development
6
(
4
),
602
612
.
https://doi.org/10.2166/washdev.2016.068
.
Whittington
D.
,
Davis
J.
,
Prokopy
L.
,
Komives
K.
,
Thorsten
R.
,
Lukacs
H.
,
Bakalian
A.
&
Wakeman
W.
2009
How well is the demand-driven, community management model for rural water supply systems doing? Evidence from Bolivia, Peru and Ghana
.
Water Policy
11
(
6
),
696
718
.
https://doi.org/10.2166/wp.2009.310
.
WHO
2011
Guidelines for Drinking-Water: Fourth Edition Incorporating the First Addendum
, 4th edn.
World Health Organization
,
Geneva, Switzerland
.
World Health Organization (WHO) & UNICEF
2020
WASH Dashboard. Joint Monitoring Programme
.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying, adaptation and redistribution, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/).