Using propensity score matching to evaluate the impact of household water treatment on childhood diarrhea: evidence from water-stressed rural settings Open Access

Water is fundamental for linking the sustainable development goals (SDGs), with SDG 6 specifically focusing on issues of water and sanitation (SIWI 2023). Water insecurity (i.e., the inability to obtain sufficient, reliable, and safe water for human well-being) poses risks and stress for humans worldwide, threatening the SDGs agenda (Tarrass & Benjelloun 2012). It is estimated that 367 million people use unimproved sources, and 122 million get their drinking water directly from surface water sources, such as rivers, lakes, and ponds, in 2020. Approximately 19% of the world's population relies on water sources that are unsafe to drink and unavailable when needed (WHO/UNICEF 2021).

The global health burden of infectious diseases is devastating, with an estimated 1.6 million annual deaths from diseases associated with lack of access to safe drinking water, inadequate sanitation, and poor hygiene (Prüss-Ustün et al. 2019). In addition to unimproved drinking water, fecal contamination of the environment (e.g., open defecation) and poor personal hygiene, mainly handwashing with soap (i.e., fecal-oral transmission pathways), play a role in increasing the incidence of diarrheal disease (Goddard et al. 2020). As a result, the current progress on water-related goals and targets must be improved to catch up with the SDGs, particularly in low- and middle-income countries (LMICs) (SIWI 2023).

In sub-Saharan Africa, about 386 million people lacked basic drinking water services in 2020 (WHO/UNICEF 2021). Inadequate, unreliable, and unsafe drinking water contributes to about 40% of the population-attributable fraction and results in 16.8 disability-adjusted life years due to infectious diseases like diarrhea (Prüss-Ustün et al. 2019). For instance, according to the Child Health and Mortality Prevention Surveillance network, about 95% of child deaths in eastern Ethiopia were related to infectious diseases, of which diarrhea contributed 10.5% as the immediate cause of death (Madrid et al. 2023).

Point-of-use water treatment, extending beyond basic water, sanitation, and hygiene (WASH) services, is needed to prevent diarrheal diseases (Ngasala et al. 2020; Wolf et al. 2022). Studies documented the effectiveness of adopting various water treatment approaches in reducing waterborne diseases. For example, Wolf et al. conducted a meta-study about the effectiveness of WASH interventions on the risk of diarrheal disease in LMICs. Of the interventions, water treated at the point of use could reduce the risk from 50 to 66% (filtration: n = 23 studies; risk ratio (RR) 0.50; solar treatment: n = 13; RR 0.63; and chlorination: n = 25; RR 0.66) (Wolf et al. 2022). Another meta-study in LMICs reported that boiling water had a significant protective effect against diarrhea disease-causing pathogens (Cohen & Colford 2017). A systematic review of WASH interventions also identified that point-of-use water filtration (RR = 0.47) and water disinfection (RR = 0.69) reduced the risk of acute diarrhea among under five children (Darvesh et al. 2017).

Although studies have indicated the protective impact of adopting household water treatment (HWT) methods against diarrhea, the literature presents inconsistent findings. For instance, the following experimental and non-experimental studies, such as those by Arnold et al. (2009); Boisson et al. (2013); Enger et al. (2013), have reported contradicting results. The most common justifications outlined by the studies for the lack of protective effect of HWT against diarrhea are poor compliance and inconsistent use. Moreover, studies have shown water quality interventions could be protective against diarrhea in the short term (i.e., during the intervention or sometime after the end of the intervention); however, the effectiveness might not be consistent in the long term (Arnold et al. 2009; Clasen & Boisson 2016).

Understanding the impacts of point-of-use water treatment on child health outcomes, like diarrhea, is critical for guiding water-related policies in water-stressed settings in low-income countries like Ethiopia. This study aims to examine the causal effects of adopting point-of-use water treatment on reducing the prevalence of diarrhea in children under five, using propensity score matching (PSM) approaches. The study is the first to employ PSM to estimate the preventive effect of HWT adoption (i.e., as a treatment variable) on the prevalence of childhood diarrhea in rural settings. We hypothesized that there is a causal relationship between adopting HWT and a lower prevalence of childhood diarrhea. This study focused on three commonly practiced HWT methods: chlorination, filtration, and boiling.

The household survey was collected using a structured questionnaire (see the Supplementary Material Table S1) coupled with onsite and laboratory water quality tests. Face-to-face interviews were conducted among mothers/caregivers of children under five. We primarily look for mothers to interview, as they are the principal carriers of water and play one of the most significant roles in ensuring household water security while caring for their children. The study collected detailed information on socio-economic status, water supply service provision, HWT practices, water use and handling behavior, under-five child health, sanitation service, etc.

The sample size for the household survey was determined by using Epi Info software version 7.2.5 by considering the following assumptions: a 95% confidence level, an 80% power with a one-to-one ratio between households with different water sources, and 15.1 and 28.1%, prevalence of childhood diarrhea among households utilizing improved and unimproved water sources, respectively (Manalew & Tennekoon 2019). Considering a 1.5 design effect and a 10% non-response rate, the final sample size was 521 households.

A simple random sampling technique (using the lottery method) was employed to select nine study Kebeles¹. Households in these Kebeles fetch water from similar sources and are not far from each other. We present the sampling procedure in the supplementary material (Figure S1). First, the number of households sampled in each Kebele was determined by proportional allocation depending on the total households in the Kebele. The study households are then selected using systematic random sampling by calculating the interval value. This random sampling of households helps to reduce the self-selection bias that arises when respondents state their treatment status (see Section 2.4).

We used the KOBO Toolbox mobile application to collect household survey data. Eight enumerators, all holding MSc degrees in Environmental Health, were recruited. They underwent 5 days of training, which included rehearsing, questionnaire testing through a pretest on 5% of the sample, and conducting the water quality assessment.

Thirty-six water samples from sources were collected by considering the primary source of drinking water for the study households from nine Kebeles. For water samples from point-of-use, we took 20% of the sample households (n = 104) to capture the water quality at home. Water samples were collected directly from the source outlets of public standpipes, boreholes, and hand-dug wells. For spring water, samples were taken from the point where people fetch water for drinking. Point-of-use samples were collected from households' drinking water storage containers using their drinking cups. The samples were collected by experienced data collectors and analyzed by the School of Environmental Health laboratory technician at Haramaya University. All samples for fecal coliform analysis were collected onsite using 150 ml sterilized plastic bottles and transported to the laboratory using an ice bag for analysis.

A standard method for microbial quality analysis was applied, following Wagtech (2011). The membrane filtration method, using the Wagtech water testing kit, was performed to identify the presence of coliform bacteria. Thermotolerant (fecal) coliforms were measured as an indicator of fecal pollution and the potential presence of pathogens (Odonkor & Ampofo 2013). The fecal contamination was reported by the number of colony forming units (CFU) per 100 ml water sample. For highly contaminated samples, such as turbid samples and those with colonies that are too many to count, a 1:10 dilution was used, and the test was repeated. Physicochemical tests were done onsite using pH and turbidity meters. We measure the free or residual chlorine using a tablet reagent, diethyl-p-phenylenediamine, by a color comparator.

We used a PSM method to estimate the effect of adopting HWT on childhood diarrhea. PSM is a widely used non-randomized method to account for self-selection bias² (Dehejia & Wahba 2002). Self-selection bias occurs when respondents decide whether or not to participate. This may cause endogeneity problems, leading to biased model estimation (Bareinboim & Tian 2015). In this case, ordinary least squares (OLS) lead to biased estimates, as they do not consider the problem of sample self-selection and do not provide useful counterfactual analysis. PSM is an alternative approach to experimental studies, which seeks to create a comparable control group that can serve as a reasonable counterfactual to remove self-selection bias due to observable differences between treatment and control households (Heinrich et al. 2010).

In observational studies, the treatment and control groups differ regarding their implementation of treatment methods and other characteristics such as socio-economic, demographic, and environmental factors. The principle of PSM is to match each treated household with control households based on similar observable characteristics (Dehejia & Wahba 2002). Unlike the OLS and instrumental variable techniques, the PSM technique does not assume the functional form, normal distribution of unobserved covariates, and finding instrumental variables for the specification of the outcome equation. The requirement in PSM is a set of measurable covariates for matching and determining the treatment's causal effects on the outcome variable (Heckman & Vytlacil 2007). The PSM steps are illustrated in Figure S2.

Two basic assumptions have to be satisfied to validate the outputs of PSM: the conditional independence assumption (CIA) and the common support condition (CSC) (Becker & Ichino 2002). The CIA³ states that given a set of observable covariates, potential outcomes are independent of treatment assignment (i.e., independent of whether households made the treatment decision or not). This assumption implies that the selection is based only on observable characteristics of the study households, and the expected outcome in the absence of treatment does not depend on treatment status (Lechner 2001). The CSC entails the existence of sufficient overlap in the characteristics of the treated and control households to find adequate matches (i.e., for every treated household, there are credible control households) (Heinrich et al. 2010).

Different matching methods pose a tradeoff between bias and variance depending on the number of observations discarded from the analysis. Various approaches to implementing PSM analysis include nearest neighbor matching, caliper or radius matching, and kernel and local linear matching (Heinrich et al. 2010). Here, we employed the three broadly used matching algorithms: nearest neighbor matching, radius matching, and kernel matching.

Households adopting at least one water treatment method (Table 1) are classified as a treated group, and those not practicing any water treatment options are classified as a control group. We computed the effect of individual methods on children's diarrhea to understand and compare the methods. Propensity score analysis was undertaken to match households who adopt HWT methods (treated groups) with non-adopters (control groups) who have similar or close propensity scores (predicted probability values), which are constructed based on observable characteristics. We used the ‘psmatch2’ function in STATA 18 software to estimate the propensity scores and perform the matching.

This represents the difference between the expected risk of childhood diarrhea prevalence for households that use drinking water treatment methods and the expected risk of childhood diarrhea prevalence for those households that do not treat drinking water.

The treatment variable is HWT practice using the three appropriate and commonly used methods: chlorination, filtration, and boiling. Households practicing point-of-use water treatment and those not practicing were identified in the study (Table 1). The treatment takes the value of one if the household adopts at least one treatment method and zero otherwise.

Children's health can be captured using several indicators; one of the most common indicators regarding safe water, sanitation, and hygiene services is the prevalence of diarrhea, specifically among children (WHO/UNICEF 2005). Childhood diarrhea was measured by mothers'/caregivers' self-reported prevalence of diarrhea within the previous 2 weeks for children under five⁴ (Alebel et al. 2018). Diarrhea was defined as three or more loose or watery stools in a 24-h period or a single stool with blood or mucus (WHO/UNICEF 2005). The youngest child from each household is included to provide information in the study, as they typically serve as the index child in similar studies (Alebel et al. 2018; Merid et al. 2023).

The control variables (Table S2) were selected based on a review of related literature by considering their confounding effects on the treatment and outcome variables. These variables were broadly classified into socio-economic and demographic, water handling and use behavior at the household level, and WASH factors.

Of the 521 households, most (93.5%) respondents were married, and 61.4% did not attend formal education. The respondents had an average age of 29.9 years and an annual aggregate income of 41,225.4 Ethiopian birr. The household wealth index was constructed using principal component analysis with eight household assets and divided into three parts by employing the parameter availability of assets in the household (Table 2).

Based on the household survey, it is found that 163 (31.3%) households practice at least one of the following water treatment methods: chlorination, filtration, and boiling. Of the households, 19.5% practice chlorination to treat their drinking water, and 14.6 and 9.6% practice filtration and boiling, respectively. This finding was comparable with previous studies in different regions of Ethiopia. For instance, a study in eastern Ethiopia, Oromia region, reported a 31% consistent use of point-of-use water chlorination (Geremew et al. 2019). Another study in the same region indicated that 30.3% of households dependent on unimproved water sources adopted three main HWT methods: boiling, cloth filtration, and chlorine-based products (Eticha et al. 2022). In addition, a study in the Southern region reported that 34.5% of households adopted one or more water treatment methods (Sisay et al. 2022).

However, the findings from the present study are incomparable with some other reports. For instance, a low HWT adoption (6%) was reported in the rural Amhara region (Azage et al. 2021). On the other hand, a very high HWT implementation (76.3%) was reported in the capital city of the Amhara region, Bahar Dar city (Birara et al. 2018). The main disparity in the findings could be due to differences in the community's perception of water quality and the presence of water safety intervention in the area.

About 49% of the studied households mostly depend on unimproved water sources for domestic use, including drinking, and 46% have to travel more than 30 min roundtrip to fetch water from the source. Only 6% of the sample water sources had a bacteriological quality that confirmed the WHO classification for safe drinking water (zero CFU per 100 ml). Regarding sanitation, nearly one-fourth of the households lacked a toilet facility in their yard, and more than half (54.9%) had inadequate or unimproved toilet facilities (Table S3).

The water samples were categorized based on the WHO risk categories for fecal coliform concentrations, measured as colonies per 100 ml (WHO 2011). Out of 104 household water samples, approximately 1.9% met the WHO conformity for bacteriological water quality, while 33.7 and 41.3% were classified as low- and intermediate-risk, respectively. Of the sampled households, 41 employed one or a combination of HWT methods, and 65.9% of the water samples exhibited a decrease in bacteriological colony count (CFU) compared to the corresponding source samples. Statistical analysis using the Wilcoxon signed-rank test (since the data is not normally distributed) demonstrated a significant difference (z = 3.94, p = 0.001) between the median values of bacteriological quality at the source and household levels.

About 25.3% (95% CI, 21.5%, 29.1%) of the last-born under-five children experienced diarrhea in the last 2 weeks before the study period. Over three-quarters of the children experiencing diarrhea reside in households that had not implemented any HWT methods. The presence of elevated levels of fecal coliforms, such as Escherichia coli, markedly increases the occurrence of diarrheal disease and affects child growth (Hanif et al. 2024).

The study determines the factors influencing the adoption of HWT methods. Here, the response variable was HWT adoption, and the explanatory variables were listed in Table 3. The model results indicated that housewife mothers had a higher probability of practicing HWT methods compared with mothers working in agriculture. Households that sourced water from low- and intermediate-risk water sources were less likely to adopt HWT than those from high-risk sources. Higher household income also influences the adoption positively. Moreover, attending WASH training, having good hand hygiene practices, and having improved toilet facilities had a significant positive association with adopting HWT methods (Table 3). This adds valuable information to the existing literature regarding the determinants of HWT adoption (Sisay et al. 2022; Daniel et al. 2023).

We run a balance test to check the quality of the matching algorithms (Table S4). Standardized bias (SB) was one of the tests used to check the balance in the model, and it was less sensitive to the sample size. Based on Caliendo & Kopeinig (2008), the SB is considered large when above 20%; in that case, the matching shows an imbalance issue. In the present match, the SB of the nearest neighbor matching (28.9%) was large, which indicates a major imbalance in the propensity scores of the treated and control groups. In contrast, radius and kernel matching had SB values of 15.3 and 13.1%, respectively, which can ensure sufficient balance to make causal inferences about treated and control groups. The p-value also indicated that the mean bias was insignificant, different from zero after matching; this means no significant differences between the treated and control groups after the samples were matched.

Table 4 shows the estimation of the average effect of practicing at least one HWT method (i.e., chlorination, boiling, or filtration) on those households that adopt the methods (i.e., the ATT). The impact of HWT practice on the outcome variable was computed using three commonly used matching algorithms: nearest neighbor matching, radius matching, and kernel matching, which were estimated to compare matching balance and quality. The PS of households adopting water treatment ranges between 0.066 and 0.676 (mean = 0.37, SD = 0.136) and 0.054 to 0.680 (mean = 0.28, SD = 0.133) otherwise. A bootstrap of 50 replications estimated the standard errors. Before bootstrapping, the observation off-support was removed, and the ATT was re-estimated to ensure no new common support problems existed in the propensity scores' estimation.

The findings of the ATT estimation showed that HWT adopters had a statistically significant effect (p < 0.05) in reducing under-five child diarrhea prevalence compared than non-adopter households based on all matching algorithms. The average values of ATT indicated that households with at least one or a combination of HWT methods had 10.5%age points (ATT =−0.105) less than non-users. The ATT values range from −0.113 and −0.116 for kernel and radius matching, respectively, to −0.086 for nearest neighbor matching (Table 4). The protective effects of HWT against diarrhea were consistent with randomized controlled trial (RCT) studies in Ethiopia. Mengistie et al. conducted an RCT in Kersa district, eastern Ethiopia, to assess the effectiveness of drinking water chlorination using 1.2% sodium hypochlorite on diarrheal disease reduction among children under five. The authors indicated that a 58% reduction (incidence risk ratio (IRR) of 0.42) in the incidence of diarrhea was observed in the intervention group when compared with the control group (Mengistie et al. 2013). A similar study in rural villages of Dire Dawa city administration, eastern Ethiopia, reported a 36% (IRR of 0.64) reduction of diarrheal disease observed in households that adopt sodium hypochlorite water disinfection (Solomon et al. 2020). Moreover, several RCT studies in LMICs corroborate the present study regarding the impact of HWT on diarrhea reduction. For example, a study in Kenya (Crump et al. 2005), Zambia (Pickering & Davis 2012), and Rwanda (Kirby et al. 2017).

Non-randomized studies also reported the protective effects of safe water against diarrheal disease. A PSM study in the Philippines using three rounds of National Demographic and Health Survey data estimated the ATT of accessing piped water and sanitation services. The authors reported that access to piped water reduces the incidence of diarrhea in children under-five by as much as 4.5% (Capuno et al. 2015). A recent study investigating the impact of safe water and decent sanitation in rural areas of 27 LMICs stated that safe water at home had a 7.4% reduction in childhood diarrhea compared to households with unsafe water at home (Merid et al. 2023). Efforts should focus on enhancing the quality of water at the point of use rather than solely concentrating on improving the source of drinking water to reduce diarrhea effectively (Husain & Das 2023).

As it is important to understand the impact of individual treatment, each method was examined separately. After following a similar PSM analysis approach, households adopting chlorination have shown a statistically significant (p < 0.01) effect in reducing childhood diarrhea by 17.6, 14.0, and 14.7%age points in the case of nearest neighbor, radius, and kernel matching, respectively (Table S6). However, the effect is not statistically significant for boiling and filtration. One explanation for this might be the small number of HWT adopters for filtration (14.6%) and boiling (9.6%), which compromises the matching process, as not all observations got a good match from the control groups. About thirteen percent of the observations were off-support for the adoption of filtration. This further reduces the number of treated groups.

Some studies also contradict the protective effects of HWT in preventing diarrheal disease. Studies reported poor compliance and inconsistent use, which may explain the lack of protective effect of HWT against diarrhea (Boisson et al. 2013; Enger et al. 2013). A 1-year randomized, double-blinded study in the rural Democratic Republic of Congo that looked at the effect of household-based filtration devices on diarrhea reduction reported that the device improved water quality. However, it did not reduce diarrhea (Boisson et al. 2010). With a similar randomized study but with a larger sample size from urban and rural households, a study in India concluded chlorination using sodium dichloroisocyanurate tablets could not be protective against diarrheal illness. The authors backed their findings that the low compliance and reduction in water source contamination might have contributed to the absence of a significant effect (Boisson et al. 2013). Moreover, a post-intervention study, 3 years after an HWT and handwashing behavioral change intervention in rural Guatemala, found that HWT practices were minimally sustained after 6 months. They reported that after the intervention, HWT adoption did not bring any difference in the prevalence of childhood diarrhea in the intervention and control groups (Arnold et al. 2009). In the present study, unlike the combined impacts of water treatment methods, the analysis for individual methods was not significantly effective in reducing diarrhea except for chlorination. As pointed out by previous studies, inconsistent use might be one reason for the lack of protection, though we did not investigate the frequency and constant use of HWT methods.

Based on individual treatment effect analysis, we found that only households adopting chlorination have shown a statistically significant effect in reducing childhood diarrhea. The potential reason for this could be that residents know the proper use of chlorine-containing products, as chlorination is the main focus of WASH training by the health extension workers (HEWs). The training is accompanied by a demonstration, mainly utilizing chlorine solutions like ‘Wuha agar.’ As HEWs mentioned during key informant interviews, this product is sometimes distributed to households for free. Our finding also indicated chlorination is the commonly applied method to purify water in the study area. In addition, compared to boiling and filtration, chlorination requires a lower treatment time before consumption, which might increase its consistent use and efficiency, unlike waiting until the boiling point is reached or regularly cleaning filtration materials.

When interpreting the results, it is important to consider the following limitations. Although we carefully selected and included the essential matching covariates that are unlikely to be affected by participation, there might be variables that are not included in the matching. This is the constraint of matching methods. The other is that even though the enumerators were trained and conducted a pilot study to minimize bias regarding measuring self-reported health outcomes and adopting HWT methods, the study might still be affected by self-reported bias. In addition, as this was a cross-sectional study, we failed to measure and consider the frequency and consistency of HWT use, which could have strengthened the findings. Moreover, we did not specifically examine the specific methods used, such as the type of filtration medium. Future studies might consider studying these impacts.

Despite numerous years of preventive interventions, such as water treatment at the household level, childhood diarrhea was still the second leading cause of death in Ethiopia. It is unclear what could reduce diarrhea's prevalence, and scientific evidence showed inconsistent findings. Using the PSM method, we found that adopting at least one or a combination of HWT methods had a protective effect on childhood diarrhea. The most common water treatment methods in the study area were chlorination, filtration, and boiling. The individual treatment effect implied that households adopting water chlorination were effective in reducing childhood diarrhea. However, filtration and boiling could not result in a significant reduction in diarrheal prevalence. This suggests the need to empower and train resource-limited communities in the effective use of boiling and filtration methods. This study pointed out the importance of designing a strategy for the wide-scale use of treatment methods, particularly point-of-use water chlorination, to reduce the burden of diarrheal diseases among children under five and ensure water security in water-stressed rural areas of Ethiopia.

We express our gratitude to Ghent University's special research fund (BOF) for generously funding this study. We acknowledge Haramaya University, College of Health and Medical Sciences, for providing ethical approval for the proposal and offering material support. Additionally, our thanks extend to the School of Environmental Health Laboratory personnel (Haramaya University) for their valuable assistance in conducting water quality analysis.

This research was funded by the special research fund (Bijzonder Onderzoeksfonds – BOF) of Ghent University, Belgium.

H.G. and B.D. conceptualized the study and developed the methodology; H.G. contribution in software; B.D. validated the process; H.G. rendered support in formal analysis and investigated the work; H.G. wrote the original draft preparation; H.G., M.G., and B.D. writing and reviewed and edited the article; B.D. and M.G. supervised the whole work. All authors have read and agreed to the current version of the manuscript.

The study obtained ethical approval from Haramaya University, College of Health and Medical Sciences, Institutional Health Research Ethics Review Committee (IHRERC) (Ref. No. IHRERC/126/2022).

All relevant data are included in the paper or its Supplementary Information.

The authors declare there is no conflict.