Indicator-based rural water service sustainability assessment: a review Open Access

Access to safe drinking water is a fundamental and essential human right, essential to health (Bos et al. 2018). People without access to water services have less opportunity to realize their potential (Watkins 2006). Access to an improved water source significantly reduces waterborne diseases (Armah 2014; Pullan et al. 2014). The formulation and implementation of international policies such as the Millennium Development Goals (MDGs) and the Sustainable Development Goals (SDGs) has made significant progress in access to safe drinking water over the past three decades. From 2000 to 2017, the proportion of the population with access to safely managed water service increased from 61% to 71%. At the same time, the proportion of the rural population with access to safely managed water services has increased from 39% to 53% (WHO & UNICEF 2019).

The presence of water supply systems in rural areas does not mean that people have access to reliable and sustainable water services (Majuru et al. 2012; Penn et al. 2017). Indeed, drinking water supply in rural areas faces problems and challenges related to inadequacies in maintenance operations and financial management generated by the service (Molinos-Senante et al. 2019). These deficiencies are the result of low population density (Wedgworth et al. 2014), incompetence of water point committees (ACF 2007), and low incomes of populations (Dhungana & Baral 2016; Githae et al. 2018). In addition, water service managers are not regulated, giving rise to poor quality of services (Galindo & Palerm 2016).

Several indices have been developed to analyze the sustainability of water resource management in order to help decision-makers identify actions to be taken and investments to improve water services received by people. These include the Canadian Water Sustainability Index (Policy Research 2007), Index of Drinking Water Adequacy (Seetharam & Rao 2010), the Sustainable Water Governance Index (Iribarnegaray & Seghezzo 2012), equity index in water and sanitation (Luh et al. 2013), Sustainability Index Tool (Schweitzer et al. 2014), WASH performance Index (Cronk et al. 2015), Sustainable Water Management Index (SWMI) (Maiolo & Pantusa 2019), and the Index of water quality service (Molinos-Senante et al. 2019).

Previous literature reviews of the indexes for the analysis of water resource management have been conducted, including by OECD (2008), Juwana et al. (2012) and Schweitzer et al. (2014). However, the number of indices developed each year is constantly increasing (Bandura 2006). This paper is therefore intended to update the review of literature on the indices to analyze water resources management with particular attention to clues to assess the sustainability of water services in rural areas in order to identify knowledge limits and lines of research.

Water service refers to all the activities and means put in place to provide people with equitable access to quality water, in sufficient quantities and at an affordable price (pS-Eau 2014). In practice, there is a trend to confuse the service itself, generally defined in terms of the quantity of water, of a given quality, accessible to users, and the system (material) used to provide it. However, the difference between the system and the service is fundamental. The system is the means used to provide the service. For example, a drill equipped with a manual pump provides one type of service, while a home water distribution system provides another (Moriarty et al. 2010).

Pezon et al. (2012) define a water service as the provision of water in accordance with a set of key indicators. This definition emphasizes the concept of level of service, which is a term used to describe and differentiate the quality of service received by users. According to these authors, four components or principles are used to characterize water services. These include the amount of water, water quality, accessibility and the reliability of the service.

The concept of sustainability applied to the natural environment was introduced by the World Commission for Environment and Development (WCED 1987). The original definition of sustainability appears to be an intergenerational compromise to meet current needs without compromising the ability of future generations to meet their own. Thus, action is considered sustainable when it is economically profitable, has limited negative impacts on the natural environment, responds equitably to societal expectations and is enshrined in a context of good governance (Edjossan-Sossou 2015).

This definition has been widely used in many industries and has been adapted to an operational definition. In water services management, Pezon (2006) proposed a normative and functional definition that is the most widely accepted. According to Pezon, a water service is sustainable if it distributes water that meets standards and ensures the renewal of its capital on the basis of a rate acceptable to users.

However, this definition does not allow policies and managers to anticipate what is uncertain and what may call into question service sustainability. In response to these shortcomings, Brochet (2014) proposed a definition that takes into account elements in terms of external sustainability. According to Brochet (2014), a drinking water service is sustainable if, while respecting the legal framework in force, it is able to ensure the renewal of its capital, without external financing and adapt to possible contextual variations, on the basis of a rate acceptable to subscribers.

The development stages of an index (Figure 1) presented in this section are a synthesis of the methodologies of the construction of indices proposed by Nardo et al. (2005), OECD (2008), and Juwana et al. (2012).

Components and indicators are the main elements of an Index. Therefore, their selection is extremely important. They are generally selected through a literature review of existing evaluation frameworks (Chaves & Alipaz 2007; Juwana et al. 2010). Ideally, components and indicators are selected on the basis of relevance, analytical strength, speed, accessibility, comparability and consistency (OECD 2008). The relevance of the data is a qualitative assessment of the value provided by these data. Speed reflects the length of time between data availability and the event or phenomenon they describe, while data accessibility reflects the ease with which data can be located and accessed from original sources. The consistency of the data reflects the extent to which they are logically connected and mutually consistent.

In this section, only the most widely used methods of standardizing the values of indicators will be presented.

Overall, there are two broad categories of methods for weighting components and indicators: statistical methods and participatory methods (Nardo et al. 2005).

The main statistical methods used for weighting indicators include Principal Component Analysis (PCA), Factor Analysis (FA), and the Unobserved Component Model (UCM).

Principal Component Analysis (PCA), and in particular Factor Analysis (FA), brings together individual indicators that are collinear to form a composite indicator that captures as much as possible of the information common to individual indicators (OECD 2008; Juwana et al. 2012). In general, PCA/FA assigns weights based on the charge factor of each indicator to the final index. The use of PCA/FA to determine weights involves four steps (Figure 2).

The first step is to analyze the correlation of indicators. If there is no correlation, it is likely that the indicators do not share common factors. Second, in cases where indicators are not correlated, indicators are assigned equal weights. However, indicators are unlikely to have any correlation. If the correlation existed, the second step is to identify common factors, representing the groups of indicators. In PCA, the factors are known as components. The third step is to determine the contribution of each indicator to its corresponding factor using the factor load analysis. Next, the final step is to calculate weights based on the factor load analysis and common factors.

The UCM method assumes that index indicators depend on other unknown factors (OECD 2008). These factors are labelled as unobserved components (Harvey & Koopman 2000; Nardo et al. 2005). Dependence on unwanted components, as well as errors associated with each index indicator, is indicated by the variance of each indicator. To use this method, the first step is to calculate the variance of each index indicator. Then the sum of the variance of the other indicators is calculated. The weight of an indicator increases as the variance of this indicator decreases and the sum of the variance of the other indicators increases (Nardo et al. 2005).

In participatory methods, weights are given on the basis of expert judgments. The main participatory methods include Budget Allocation (BA), and the hierarchical analysis method. In the BA method, experts in an area described by a set of indicators are asked to allocate a budget among the indicators based on their subjective experience and judgment of the relative importance of each indicator. Weights are determined from the average budget allocated to each indicator.

The analytical hierarchy process (AHP) is a multi-criterion analysis technique used in different fields such as customer services (Kwong & Bai 2002) and water conservation (Zhang 2009). It facilitates the decomposition of a problem into a hierarchical structure and ensures that the qualitative and quantitative aspects of the problem are simultaneously taken into account in the evaluation process during which opinions are systematically extracted by the average of peer comparisons (OECD 2008). Weights are determined through peer-to-peer comparisons of identified indicators (Juwana et al. 2012).

In index development, aggregation is done sequentially as illustrated in Figure 3 assuming the sustainability index has dimensions or components, indicators, and sub-indicators. In general, there are two main methods of aggregation in the development of indices: the arithmetic method and the geometric method.

Several judgments are issued during the various stages of the construction of the index including the selection of indicators, standardization, weighting and aggregation. These judgments generate uncertainties that affect the final index that may lead to a challenge to the robustness of the final index. The combination of uncertainty and sensitivity analysis allows the index to be robust and to improve its transparency. There are several methods that allow for the analysis of uncertainties. However, this section will address the Monte Carlo method as described by Nardo et al. (2005).

The Monte Carlo analysis consists of defining a probabilistic function combining the index's variables. The aim is to generate N combinations from X_i random variables and analyze the impact of each combination on the result variable, i.e. on the value of the final index. X_i samples can be obtained from several randomization methods such as simple random sampling, and stratification sampling, etc. (Saltelli et al. 2000). From an arbitrary threshold, it is possible to determine the characteristics of this density function from the number N simulations obtained.

The sensitivity analysis of the index can be done by building a confidence interval or by decomposition of variance. Sensitivity analysis by building a confidence interval involves combining a confidence interval with a margin of error set beforehand. The next step is to vary the different methods and to assess the sensitivity of the index to the change. Sensitivity analysis by decomposition of variance allows the strength of the index to be assessed. The analysis consists of assessing the contribution of each input variable (indicator) to the total variance of the index on the one hand, and detecting the part explained by the interactions between the different inputs on the other. This decomposition allows the building of indexes of index sensitivity, a quantitative measure of sensitivity.

The water sustainability indexes in the literature can be categorized as global water sustainability indexes and regional water sustainability indexes. Global water sustainability indexes have a general scope and can be applied to all contexts without much change. Regional water sustainability indexes are region-specific and cannot be applied to other contexts without modification.

However, the main issue commonly raised with WPI that must be addressed by futures studies is the selection of the appropriate number of components and subcomponents (Salameh 2000; Sullivan 2002; Molle & Mollinga 2003; Sullivan et al. 2003). Selecting the components means choosing to make certain aspects visible and therefore to make others invisible according to the logic of action of the WPI and certain social compromises.

The Index of Drinking Water Adequacy (IDWA) is a monitoring tool designed to assess and compare countries' performance in providing adequate drinking water to their citizens. The IDWA allows for country-to-country comparisons and helps determine which components of access to safe drinking water are low and require priority attention (Seetharam & Rao 2010). The IDWA consists of five components: resource, use, quality, capacity and access. Each component of the IDWA receives the same weighting factor. The final value of the IDWA is the average of the standardized values of the five components. However, the main issue commonly raised with IDWA and that must be addressed by future studies of this index is the selection of the appropriate weighting method. In fact, some think that quality is so important that giving it a weight equal to all other components is not acceptable.

The index was applied with successful results to the Rio Grande transboundary basin by Sandoval-Solis et al. (2011). However, further research is needed to estimate and evaluate water management of the basin under different hydrologic conditions, considering the alteration of the hydrological cycle due to climate change. Further research is also needed to estimate future demands and their evaluation in the planning simulation model.

EI was applied to 56 States in 2010 in order to measure state progress in realizing substantive equality for the right to water (Luh et al. 2013). This study demonstrated that EI score is not dependent on factors such as achieved level of coverage or gross national income. However, different weightings to SI, PI and OI components may be addressed by futures studies in order to take into account the relative importance of components, and sensitivity analysis is required to determine which sample weights would fit best from both human and legal standpoints. In addition, the four indicators used to construct the Equity Index were selected based on the availability of data, and as more data becomes available, additional carefully selected indicators should be included in the index and will help to provide a more representative EI value.

The Water, Sanitation and Hygiene (WASH) Performance Index was proposed by Cronk et al. (2015) in order to meet the challenges posed by sustainable development, in particular universal access to water, sanitation and hygiene services. The index ranks countries based on water and sanitation performance and on implantation of the right to water and sanitation. The WASH Performance Index employs frontier analysis to monitor the human right to water and sanitation across countries and over time. It was applied by Cronk et al. (2015) to all the UN country members. They found that progress toward equity in sanitation is significantly associated with governance indicators and suggested that enabling an environment for WASH contributes to progress in sanitation equity.

The CWSI was applied to six community case studies in Canada (Policy Research 2007) with a very successful result in gauging the usefulness and practicality of the index. However, Canada does not face the same water challenge as the rest of the world, particularly developing countries in Africa and parts of Asia. So, the CWSI needs further refinement and development before applying to another country.

Iribarnegaray & Seghezzo (2012) developed the Sustainable Water Governance Index (SWGI) for the city of Salta. The SWGI connects governance and sustainability concepts with their roles in water and sanitation management systems (WSMS). The categories used for the construction of the SWGI for the city of Salta, Argentina, are presented in Figure 4.

However, subjective estimations are prone to continuous changes and depend on the personal experience and knowledge of the members of the assessment team. Translation of opinions into semi quantitative measures is not without risks, and it must be facilitated by experts for it to be meaningful. This translation can be done by future studies by means of convergence participatory techniques or other tools used for environmental and social assessment.

The Water and Sanitation Sustainability Index (WASSI) was proposed by Iribarnegaray et al. (2012). WASSI is divided into three generic sub-indexes: Place, Permanence, and People. The ‘Place’ sub-index assesses the relationship of the management system with its environment and the biophysical and cultural territory on which it operates. The Permanence sub-index assesses the short-, medium- and long-term aspects of the WSMS, with indicators that reflect local capacity to solve problems, improve the management system and ensure coverage of basic human needs. WASSI's ‘Person’ sub-index places greater emphasis on the personal aspects of the management system, with indicators that highlight the human dimension of water management in times of scarcity and unequal access to water and sanitation services.

The Sustainable Cities Water Index (SCWI), developed by Arcadis, explores three aspects that make up robust, efficient and healthy water environments to develop an indicative ranking of cities. The index is a tool to help inform future water improvement and long-term sustainability (Arcadis 2016). Arcadis (2016) breaks down water sustainability into three fundamental elements (Figure 5). The SCWI was applied to 50 global cities from 31 countries across all continents of the world by Arcadis (2016). They found that cities need to make greater investment to improve their resiliency to extreme weather events and unforeseen water shortages.

The Water, Sanitation and Hygiene (WASH) Index was proposed by Tsesmelis et al. (2020) in order to facilitate WASH related assessments in refugee camps, by capturing and reflecting the actual WASH conditions and provide the necessary information for efficient program planning and implementation. The index is based on three components: water, sanitation and hygiene. The weighting of the main and sub-indices is achieved through the application of a Radial Basis Function (RBF) Network. The aggregation of each individual indicator is based on the ‘Weighted Arithmetic Mean’.

The index was applied in Europe and specifically in twelve humanitarian camps in Greece by Tsesmelis et al. (2020). This application showed that it can describe the encountered conditions at the refugee camps well. However, the index did not include access of vulnerable and marginalized groups to WASH services and some facilities like schools, and health centers are not included. Thus, the WASH Index has still to undergo further development in order to address these issues.

Table 1 presents a number of other tools that have been developed to trigger improvements to programme design or take remedial actions.

Various indices have been developed to measure the sustainability of water services. Most of the models reported in the literature are very complex, and sustainability assessments focus on water resource management. Thus, the indicators for an integrative, contextualized and prospective analysis of the sustainability of water services in rural areas remain embryonic, if not non-existent. Indeed, most of the indices presented in the literature focus on one aspect of service sustainability, including water resource management (WPI and CWSI), water governance (IDWA, SWGI, WASSI, EI), water-sanitation-hygiene (WASH Performance Index), water management (Water Management Sustainability Index) and the Service Quality Index. Future studies should therefore be carried out in order to develop indices allowing an integrative, contextualized and prospective analysis of the sustainability of water service in rural areas.

These indices refer to the current or past situation of the aspect of the service they are measuring. Their ability to give a valid picture of sustainability in a coherent time horizon is therefore limited. This difficulty therefore encourages futures studies to seek to incorporate a long-term approach that is not only retrospective but also forward-looking. In addition, these indices are based on a forecasting logic of the investments that water service managers at all levels will need to make and can be expected to ensure the sustainability of the service. However, the sustainability of water services cannot be reduced to predictable aspects alone (Brochet 2014). It must include elements of adaptation to external shocks, which some authors (Walker et al. 2004) have theorized in terms of resilience. Indeed, the approaches developed do not allow us to anticipate what is uncertain and which may call into question the sustainability of services. These indices focus on the actions of service managers and users at all levels as if they were the only actors to interact with the drinking water system. However, other participants can help to change the sustainability of the service, whether in response to natural events (climate change, natural disasters, etc.) or events of anthropogenic origin (crises, changes in consumption patterns, etc.). Therefore, it is essential future studies do not provide a static analysis of sustainability, but a dynamic analysis including contextual parameters and long-term forecasts.

In addition, the indices in the literature are made up of indicators that are not directly applicable to rural areas in Sub-Saharan Africa. They need to be adjusted to reflect contextual parameters before being applied – hence the interest of futures studies.

This systematic review highlights the importance of the development of a dynamic analysis of sustainability of rural water services including contextual parameters and long-term forecasting. Findings suggest that indices present in the literature do not provide an integrative, contextualized and prospective analysis of the sustainability of water services. In addition, these indices select only a few evaluation criteria deemed relevant. In other words, these indices make the choice to make visible certain phenomena and therefore make others invisible according to the logic of action of each and to certain social compromises. Future studies should address these shortcomings and develop a new index to provide an integrative, contextual and prospective analysis of the sustainability of rural water services.

All relevant data are included in the paper or its Supplementary Information.