Validity of an abbreviated Individual Water Insecurity Experiences (IWISE-4) Scale for measuring the prevalence of water insecurity in low- and middle-income countries

AUC

area under the curve

GWP

Gallup World Poll

HWISE-12

12-item Household Water Insecurity Experiences

HWISE-4

4-item Household Water Insecurity Experiences

IWISE-12

12-item Individual Water Insecurity Experiences

IWISE-4

4-item Individual Water Insecurity Experiences

WISE

Water Insecurity Experiences

ROC

receiver operating characteristic

Growing problems with physical water availability (e.g., Mekonnen & Hoekstra 2016) and quality (Damania et al. 2019) point to a mounting global water crisis. These issues have consequences for access (physical, economic, and otherwise), acceptability, safety, and/or stability of water for domestic uses (Rosinger & Young 2020; Young 2021), i.e., water insecurity (Jepson et al. 2017).

Although these data and regular headlines suggest a global water crisis, there is a noted absence in high-resolution data about who is experiencing water insecurity (High-Level Panel on Water 2018). For example, global indicators of drinking water access have not been disaggregated beyond subnational regions (e.g., urban vs. rural) and household-level wealth quintiles (UNICEF and WHO 2021), such that we lack data on disparities in physical water availability or water infrastructure by household characteristics (e.g., size) or individual characteristics (e.g., gender, age, employment status). Furthermore, indicators of water availability and drinking water services do not capture information on experiences with water access, use, and stability beyond that for drinking water. Because difficulties with water for cooking, bathing, and cleaning can also affect many aspects of well-being, this has meant that we have not had a complete picture of the human consequences of water insecurity (Young et al. 2021b).

To more holistically capture human experiences with sub-constructs of water access, use, and stability for domestic needs, the Household Water Insecurity Experiences (HWISE) and Individual Water Insecurity Experiences (IWISE) Scales were developed (Young et al. 2019a, 2019b, 2021a). Their internal consistency, construct validity, and cross-context equivalence have been demonstrated for multiple low- and middle-income countries. Both scales comprise 12 items about the frequency with which households (HWISE) or individuals (IWISE) experience water-related problems. The HWISE and IWISE scale items are similarly phrased, though the former was validated using a recall period of 4 weeks while the latter used a recall period of 12 months, thereby incorporating seasonal variation in precipitation and temperature. Scale items cover the same sub-constructs, which include water-related psychosocial distress, insufficient water for consumption, and water-induced disruptions in hygiene and other daily activities. The WISE scales or subsets of their items have been valuable for understanding how water insecurity relates to a range of issues, including socioeconomic and gender inequalities, food insecurity, psychosocial distress, hygiene, human development, and health (Young et al. 2019b; Brewis et al. 2020; Hannah et al. 2020; Staddon et al. 2020; Stoler et al. 2020, 2021; Venkataramanan et al. 2020; Rosinger et al. 2021; Duignan et al. 2022; Mao et al. 2022; Wutich et al. 2022).

Although administration of the WISE scales is straightforward and can be completed in approximately 3 min, there are circumstances in which a shorter scale may be preferred or necessary, including when survey space is limited or for telephone surveys, where lengthy surveys are too burdensome for respondents. As such, an abbreviated, four-item, one-minute HWISE scale (HWISE-4) was developed and shown to have suitable predictive accuracy and criterion, convergent, and discriminant validity (Young et al. 2021c).

An individual version of the WISE scale is useful for being able to examine intra-household variation in water insecurity, identify disparities in water insecurity across individual-level characteristics (e.g., gender), and/or explore relationships between water insecurity and individual-level health outcomes (Young et al. 2021a). However, the validity and psychometric properties of a shortened version of the IWISE Scale have yet to be examined. Therefore, the purpose of this study was to assess the internal consistency, cross-country equivalence, and construct validity of a four-item version of the IWISE (herein referred to as IWISE-4) Scale, as well as to evaluate how accurately it could estimate individual water insecurity prevalence when compared with the 12-item scale.

The cross-sectional data for validating IWISE-4 comes from the 2020 Gallup World Poll (GWP), which administered the IWISE-12 module to 45,555 civilian, non-institutionalized individuals ≥15 years old (herein referred to as adults) in 31 low- and middle-income countries from four regions: Sub-Saharan Africa, North Africa, Asia, and Latin America between September 2020 and February 2021. Details on the implementation and validation of the full 12-item IWISE Scale (henceforth referred to as IWISE-12) have been detailed elsewhere (Young et al. 2021a). Data were also collected on perceived water quality and basic socio-demographic characteristics.

Data were collected following GWP's standard protocol for obtaining consent from participants, de-identified, and made available to the authors. The authors were not involved with the consent or data collection process.

The IWISE-12 Scale consists of 12 questions that query the frequency that individuals experienced life-disrupting water-related problems in the previous 12 months (Young et al. 2021a). The items pertain to psychosocial distress related to water (worry, anger, shame), water-induced disruptions in hygiene practices (inadequate water for handwashing, bathing, laundering), and issues pertaining to water consumption (not having desired amounts of water to drink, having to change foods eaten, going to sleep thirsty) (Supplementary Table S1). Other questions ask about disruptions in daily schedule and water supply. Response options are ‘never’ (scored as 0), ‘1–2 months’ (1), ‘some, not all, months’ (2), or ‘almost every month’ (3). Scores for the 12 items are summed; the IWISE-12 scores range from 0 to 36. A score of ≥12 has been used as a cut-point for defining water insecurity for both the IWISE (Young et al. 2021a) and HWISE (Young et al. 2019a) Scales.

We began by assessing the suitability of the same subset of items that were selected for the HWISE-4 (Supplementary Table S1) (Young et al. 2021c). These items pertained to worry about not having enough water for all needs (‘worry’), having to change plans because of problems with water (‘plans’), not being able to wash hands after dirty activities because of problems with water (‘hands’), and not having enough water to drink (‘drink’). We also performed the suite of analyses described below on several other four-item subsets; each subset (listed in a footnote on Supplementary Table S1) was selected to contain items representing the range of most to least frequently affirmed experiences (Young et al. 2021a). As no other subset was superior to the subset used for HWISE-4 in terms of correctly classifying individuals’ water insecurity status or estimating the prevalence of WI, we used the initial set of items, i.e., ‘worry’, ‘plans’, ‘hands’, and ‘drink’, to maintain consistency across the household- and individual-level abbreviated versions of the WISE scales (Supplementary Table S1).

Of the 45,555 individuals who were surveyed by the GWP with the IWISE module, 1,585 respondents were excluded due to missing one or more IWISE items, resulting in an analytic sample of 43,970 from 31 countries.

The validity of IWISE-4 was examined using a variety of methods (overview in Supplementary Table S2). First, we compared the internal consistency of the IWISE-4 items to that of the IWISE-12 items, by comparing the Cronbach's alpha for each scale by country and on average across the full sample.

Second, using the same alignment method in Mplus and threshold of <25% of non-equivalence among item parameters (Asparouhov & Muthén 2014; Muthén & Asparouhov 2014) used to assess the measurement equivalence of IWISE-12 (Young et al. 2021a), we investigated if IWISE-4 was cross-country equivalent.

Third, we assessed the predictive accuracy of the IWISE-4 scale by regressing the continuous IWISE-12 score on the continuous IWISE-4 score by country. We also used these models to examine the root mean square errors to assess the additional absolute error introduced when using IWISE-4 instead of the IWISE-12.

Fourth, we compared the prevalence of water insecurity established using the full IWISE-12 scale to the prevalence of IWISE-4 scores of ≥3, ≥4, and ≥5, to determine which cut-point provided the minimum absolute difference in water insecurity prevalence estimates compared with IWISE-12.

Fifth, we created receiver operating characteristic (ROC) curves that plotted the true positive rate (sensitivity) in relation to the false positive rate (1-specificity) for each potential IWISE-4 cut-point (0–12). The area under the ROC curve, representing the probability that IWISE-4 can accurately predict water insecurity as defined by IWISE-12, was used as a measure of overall accuracy of IWISE-4. To identify which IWISE-4 cut-point most accurately predicted water insecurity (WI), we examined the sensitivity, specificity, positive and negative predictive values, and percent of individuals correctly classified as water secure or insecure. The cut-point with the highest sensitivity, specificity, and largest percentage of individuals correctly classified in most countries was the cut-point we selected.

Sixth, we plotted the sensitivity, specificity, and percentage of individuals whose water insecurity status was correctly classified by an IWISE-4 score ≥4 in relation to country prevalence of water insecurity (IWISE-12 scores ≥12) to determine if there was a relationship between correct classification and water insecurity prevalence.

Finally, we used data on respondents’ reported satisfaction (satisfied or dissatisfied) with their water quality to test construct validity. This is the same variable used to test construct validity in the validation of the IWISE-12 Scale (Young et al. 2021a). Specifically, we used simple logistic regression models to estimate the odds of water quality dissatisfaction in relation to water insecurity defined by the IWISE-12 Scale (cut-point ≥12) and the IWISE-4 Scale (cut-point ≥4), and compared the coefficients and confidence intervals for each country and in the overall pooled sample (adjusting for country).

All analyses were completed using Stata (v17.0, StataCorp, College Station, TX). Sampling weights were not used in these analyses, as the analyses were used to assess the validity of the instrument, and not to provide nationally representative estimates of water insecurity prevalence.

When examining internal consistency by country, Cronbach's alpha for the IWISE-4 items ranged from 0.67 to 0.91, with a mean of 0.76 across countries (Supplementary Table S3). The Cronbach's alpha values for IWISE-4 (range: 0.87–0.96, mean: 0.92) had a wider range and were lower than those for the IWISE-12.

Results from the alignment method used to test approximate measurement equivalence confirmed that only 12.9% of the item parameters were non-equivalent (data not shown), well below the <25% non-equivalence threshold used as criteria for establishing approximate measurement invariance (Asparouhov & Muthén 2014; Muthén & Asparouhov 2014). This confirms that IWISE-4 is approximately cross-country equivalent, similar to the IWISE-12 (Young et al. 2021a) and HWISE-12 (Young et al. 2019b) scales.

The predictive accuracy of IWISE-4 relative to IWISE-12 was high across countries. The coefficients from individual country models ranged from 2.46 to 2.85, with a mean of 2.65 across countries (Table 1). The root mean square errors ranged from 1.19 to 3.54, with a mean of 2.68. IWISE-4 scores explained 83–95% (mean: 88%) of the variation in IWISE-12 scores across countries.

Relative to IWISE-4 cut-points of ≥3 and ≥5, the cut-point of ≥4 provided water insecurity prevalence estimates that most closely approximated (i.e., had the minimum absolute difference from) the prevalence estimates using IWISE-12 score of ≥12 in most countries (Supplementary Table S4). The cut-point of ≥4 overestimated the prevalence of water insecurity by a mean of 3.4 percentage points (range: 0.1–8.2) across countries and by 2.8 percentage points when pooling the data. This cut-point was more accurate than a cut-point of ≥3, which overestimated water insecurity by a mean of 11.1 percentage points (range 1.4–16.8) and a cut-point of ≥5, which underestimated water insecurity by a mean of −5.2 (range: −0.5 to −11.8) percentage points.

The 4-item version of the IWISE Scale, the IWISE-4, is a suitable, cross-country equivalent approach to estimating the individual prevalence of water insecurity when the use of the full 12-item IWISE Scale is not possible. Based on the ROC curves, the IWISE-4 was able to predict water insecurity as defined by IWISE-12 with ≥95% accuracy in all the countries tested; an IWISE-4 score of ≥4 correctly classified water insecurity status of 87.1–98.5% individuals across countries. The accuracy of the IWISE-4 was comparable to that of HWISE-4, which correctly classified water insecurity status of >91% of households across study sites using a cut-point of ≥4 (Young et al. 2021c).

The predictive accuracy, as measured by the average absolute error (i.e., root mean square error), was 2.68 points on average, the equivalent of less than one IWISE-12 item being affirmed at the frequency of ‘almost every month’. A similar range of root mean square errors (2.13–2.68) was reported when regressing HWISE-4 on HWISE-12 (Young et al. 2021c). That the internal consistency of 0.76 for IWISE-4 was less than that for the IWISE-12 (0.92) was expected because eight fewer items were used. The range of Cronbach alpha values of 0.66–0.91 was similar to the range of 0.79–0.93 reported for HWISE-4 (Young et al. 2021c).

Compared with the IWISE-12, the IWISE-4 captures fewer manifestations of water problems and therefore should not be used to differentiate degrees of water insecurity. The fewer experiences of water insecurity captured by the IWISE-4 could explain the slightly lower negative predictive value (i.e., higher proportion of false negatives) in countries with a higher prevalence of water insecurity (i.e., more potential water problems). Notwithstanding, IWISE-4 provided water insecurity prevalence estimates within a few percentage points of the IWISE-12 estimates and correctly classified the water insecurity status of >87% of individuals across countries with both low and high water insecurity prevalence. Moreover, the IWISE-4 indicator of water insecurity predicted nearly the same odds of water quality dissatisfaction as IWISE-12 in most countries, including those with low (e.g., China) and high (e.g., Cameroon) water insecurity prevalence. This suggests that the construct validity of the IWISE-4 Scale holds across distinct contexts, but investigators may wish to use IWISE-12 to better understand the array of water problems experienced by individuals in more water insecure regions.

The IWISE-4 slightly overestimated water insecurity compared with IWISE-12. However, given the wide range of prevalence across countries (4.3–68.1%), the overestimation of 3.4% on average was small and with little consequence for comparisons across countries. For countries using IWISE to monitor WI, using the IWISE-12 or IWISE-4 consistently would facilitate comparisons over time. Furthermore, data from IWISE-12 could be used to calibrate the prevalence of the IWISE-4 (i.e., adjust the prevalence of IWISE-4 so it was exactly comparable to that of IWISE-12 in that country). Collection of IWISE-12 data in a subsample could also be useful for calibration in countries suspected to have higher water insecurity and/or understand the severity of water insecurity.

Information on individuals’ experiences with water access, use, and stability is imperative for making informed decisions about where to invest resources to improve water security for those most in need, as well as to understand the impact of interventions and natural shocks.

The IWISE-12 Scale captures a comprehensive array of water problems and can be used for establishing prevalence and differentiating degrees of individual water insecurity. When limited resources prevent the implementation of the IWISE-12 Scale, the IWISE-4 Scale can provide suitable, cross-country equivalent estimates of water insecurity prevalence.

We gratefully acknowledge the support of the Carnegie Corporation; the support of the American people provided to the Feed the Future Sustainable Intensification Innovation Lab (SIIL) through the United States Agency for International Development Cooperative Agreement AID-OAA-L-14-00006; and Northwestern University, including from the Northwestern Institute on Complex Systems, the Global Poverty Research Lab, Weinberg College, the Office for Research, Leslie and Mac McQuown via the Center for Engineering Sustainability and Resilience, Global Health Studies, and the Center for Water Research. We are also grateful to the HWISE-Research Collaborative Network community for their intellectual vibrancy and Gallup Poll for their support throughout data collection and analysis. Finally, we would like to acknowledge Joshua Miller and Godfred Boateng for their review of an earlier draft of this manuscript.

Data can be made publicly available upon reasonable request; readers should contact the corresponding author for details.

The authors declare there is no conflict.