Developing a novel tool for assessing water service sustainability in rural areas of sub-Saharan Africa

Premature failure of water infrastructure and poor service levels experienced by end-users in rural areas in sub-Saharan Africa has resulted in an increasing emphasis on sustainability in recent years. Partly as a response to these challenges, a number of tools were developed to help understand and improve water services. These tools include the WASH Life-Cycle Assessment (McConville & Mihelcic 2007), the Sustainability Check (UNICEF 2008), the Sustainability Assessment Tool (SAT; Aguasan Group 2010), the Sustainability Snapshot (Carter et al. 2011), the Sustainable Index Tool (SIT; USAID/Rotary International 2012), the WASHCost Tool (WASHCost 2012), the Sustainability Monitoring Framework (SMF; DWA 2013), the WASH Sustainability Sector Assessment Tool (Schweitzer et al. 2014), and the Water, Sanitation and Hygiene Bottleneck Analysis Tool (WASH BAT; WASH BAT 2018).

The WASH Life-Cycle Assessment consists of a matrix, the dimensions of which are defined as the five sustainability factors and the five project life stages (McConville & Mihelcic 2007). The Sustainability Check assesses the sustainability of the WASH infrastructure using five weighted factors: institutional, social, financial, technical, and sanitation (UNICEF 2008). The SAT is used to determine the sustainability of the program interventions and is based on six components: economic, environmental, institutional, knowledge, social, and technological (Aguasan Group 2010). The Sustainability Snapshot determines the financial and technical capacity of the community-managed water system. The tool is based on three components: financial, technical, and spare parts/equipment (Carter et al. 2011). The SIT assesses the sustainability of the services provided by WASH project interventions. The index is based on five components (institutional, management, financial, technical, and environmental), but no weighting is introduced into the scoring (USAID/Rotary International 2012).

The WASHCost tool is an open-source tool designed to aid users to effectively plan, budget, manage, and evaluate the delivery of water and sanitation services using a lifecycle costs approach. The tool helps stakeholders consider what the expected capital and recurrent expenditures will be for different technologies and service levels (WASHCost 2012). The SMF assesses the presence or absence of factors with a proven impact on sustainability. The SMF is based on five components: financial, institutional, environmental, technical, and social (FIETS) (DWA 2013). The WASH Sustainability Sector Assessment Tool provides a better understanding of the program design, priorities, and decision-making within the context of the sector level as opposed to the individual project level, as well as identifying key weaknesses or bottlenecks. The WASH BAT has been designed for use by governments and development partners for WASH sector strengthening. It enables a systematic identification of factors (or ‘bottlenecks’) that prevent the achievement of sustainable service delivery within national or subnational WASH targets and helps stakeholders to define activities aimed at removing the root causes of these bottlenecks (WASH BAT 2018).

Although there are successful experiments in the implementation of these tools, they do not provide an integrative, contextualized, and prospective analysis of the sustainability of rural water services (Mvongo et al. 2021a). This difficulty, therefore, encourages the development of a new index to provide an integrative, contextual, and prospective analysis of the sustainability of rural water services. A new water sustainability index, called the Water Service Sustainability Index (WSSI), was developed using the analytical hierarchy process (AHP; Mvongo et al. 2021b). This was specifically developed with the involvement of local water stakeholders and based on natural and socioeconomic characteristics of the Mvila Division. This index is able to obtain information on rural water services, and prioritize problems related to service management with priority actions to improve water service management. Therefore, this paper aimed to develop a tool for implementing the WSSI.

Dimensions and indicators are the main constituents of the WSSI. They were selected through a literature review on existing evaluation frameworks (Mvongo et al. 2021b). The WSSI final framework has 21 indicators grouped into six dimensions (Table 1).

The AHP was used to weighting dimensions and indicators because of its simplicity. It facilitates the decomposition of the problem into a hierarchical structure and ensures that the qualitative and quantitative aspects of the problem are taken into account simultaneously in the evaluation process in which expert judgments are taken into account through an even comparison matrix (Saaty 1980; Siekelova et al. 2021). The weights of the various indicators were established by following the steps and procedures recommended by Saaty (1990).

The development of the WaSAT was done using Microsoft Excel and Open as App. Microsoft Excel was used for programming the WSSI calculations, whereas Open as App was used to convert the Excel File into an application. Figure 1 shows the different stages of creating the WaSAT application.

The WaSAT is a systematic questioning tool according to six dimensions (economic, environmental, social, technical, institutional, and governance) which makes it possible to assess to what extent the water service management policy implemented in a locality or a municipality promotes the achievement of the sustainability objectives of the service and universal access to drinking water. The tool can be used as an Excel spreadsheet or as an application. The tool is intended for municipal authorities, officials of ministry in charge of rural water supply, water engineers, and researchers.

Table 3 presents the scoring grid used by the WaSAT. This grid allows the evaluation of sustainability levels on the basis of scoring. These scores are based on a five-point scale (fully met (score of 100%); met to a high degree (score of 75%); met acceptably (score of 50%); met to a low degree (score of 25%); and not met (score of 0%)) depending on the criteria met by the service and described in Table 3.

The WaSAT as an Excel spreadsheet (Supplementary Material 1) is composed of seven sheets or pages (Menu, Welcome, Description of indicators, Normalization of indicators, Evaluation, Summary, Interpretation, and Contact). Some pages (Welcome, Description of indicators, Normalization of indicators, and Interpretation) present the modalities of use, while others (Evaluation and Summary) allow to calculate the WSSI and to present the calculations in the form of a graph. Figure 2 shows the WaSAT menu page.

The ‘Welcome’ page briefly presents the WaSAT and the context in which this tool was developed. The ‘Description of indicators’ page presents the dimensions and indicators used in the calculation of the WSSI, while the ‘Normalization of indicators’ page presents the criteria for evaluating qualitative indicators. The ‘Evaluation’ page is used to calculate the WSSI, and the ‘Summary’ page presents the summary of the results and the associated graphs. The ‘Interpretation’ page is devoted to WSSI interpretation methods, while the ‘Contact’ page presents information on the authors of the WaSAT.

The WaSAT as an application is made up of eight pages. The first page entitled ‘Summary’ presents the graphic summary of the results, while page 2 entitled ‘WSSI’ presents the summary of the calculation of the WSSI. Pages 3, 4, 5, 6, 7, and 8, respectively, present the summaries of the calculations of the sub-indexes of the economic, environmental, social, technical, institutional, and governance dimensions. Figure 3 shows the WaSAT as a web application.

The summary page is the page that opens when you start the application. It presents a graphic summary of the results of the evaluation. The graphs presented are radar types and are seven in number. The ‘WSSI’ page gives the numeric values of the performance of the index dimensions and the value of the index. The ‘Economic’ page presents two columns of results: a column that presents the performance values of the indicators of the economic dimension (dark gray color) and a column that presents the evaluation of each indicator on the basis of the scoring grid (gray color). Only the indicator evaluation data can be changed, while the performance values are calculated automatically by multiplying the evaluation results by the weight of the indicator. This page works much like a calculator. The other pages (environmental, social, technical, institutional, and governance) have the same configuration as the ‘Economic’ page.

The WaSAT provides a solid baseline on the sustainability of water services at the village level in rural areas and identifies priority actions to be taken to move services toward sustainability. The tool focuses on rural water services in sub-Saharan Africa in a tropical climate context. The data used by this tool are collected at the level at which services are provided (household or community level). The proposed tool enriches the range of tools developed in the literature to improve access to drinking water and sanitation.

The innovative aspect of the work is the presentation of a new indicator as an adaptation of an existing one because of the need to adapt it to the specificities of rural areas in sub-Saharan Africa. Specifically, the first innovative aspect is to take climate change into account through the climate risk indicator. Climate change is having an impact on the availability of water resources, specific per capita consumption, quality of service, the state of water infrastructure, and the costs of investment and operation of the service (pS-Eau, Acqua-OING, AFD, GRET & AESN 2013). In addition, the risks associated with climate change can comprehensively challenge the sustainability of water services. The second is the introduction of forecasting that is not only retrospective but also forward-looking. For example, climate risk takes into account not only past and present climate variations, but also future climate variations. The same is true of the service's self-financing capacity, which, while analyzing the service's pricing policy, takes into account the price revision due to current and future inflation.

The tool differs, however, from most of the similar tools presented in the literature which focus on the evaluation of projects or programs in the rural water and sanitation sector [WASH Life-Cycle Assessment by McConville & Mihelcic (2007); SAT of Aguasan Group (2010); USAID/Rotary International Sustainability Index Tool (SIT) 2012; WASHCost Tool from WASHCost (2012); SMF by DWA (2013)] or which focus on a sector analysis [Sustainability Snapshot by Carter et al. (2010); WASH Sustainability Sector Assessment Tool from IRC/Aguaconsult (2013); Rural Water and Sanitation Information System (SIASAR) de SIASAR (2016)].

However, the tool has two main limitations. The first is a major drawback to using the WaSAT as an Excel file. These include the number of Excel files to use. Indeed, if a municipality has several water services, each service will have to correspond to an Excel file for an evaluation. This could lead to errors and loss of data due to handling. This difficulty is taken into account in using the WaSAT as an application that allows assessment data to be centralized in a single database. However, using it comes at a cost and requires an internet connection. This makes it difficult to access and use by rural communities in Africa given the precariousness of living conditions in rural areas.

Second, given that the WaSAT is still in the early stages of its development, there is a need to systematically validate its future results. This is necessary to ensure that the tool is linked to the actual durability of the service. However, such an analysis is beyond the scope of this study and would require the collection of data on the water services actually provided in rural areas. Although this is an expensive process, it would increase confidence in the use of the WaSAT. In addition, the tool should be more flexible and appropriate for use in urban and peri-urban areas – not limited to rural interventions – and incorporate data from different levels, for example, going beyond the single level of direct intervention to include higher level enabling environment considerations.

The WaSAT provides a solid baseline on the sustainability of water services at the village level in rural areas and identifies priority actions to be taken to move services toward sustainability. There is a major drawback to using the WaSAT as an Excel file, however. These include the number of Excel files to use. Indeed, if a municipality has several water services, each service will have to correspond to an Excel file for an evaluation. This could lead to errors and loss of data due to handling. This difficulty is taken into account in using the WaSAT as an application that allows assessment data to be centralized in a single database. However, using it comes at a cost and requires an internet connection. This makes it difficult to access and use by rural communities in Africa given the precariousness of living conditions in rural areas. The application of the WSSI will shed light on the state of play of the sustainability of water services and measures aimed at improving water management. The tool is being implemented in eight municipalities in the Mvila Division (Southern region of Cameroon) with quite promising results.

The tool will be implemented in the Mvila Division (Southern region of Cameroon) in collaboration with the eight municipalities of the Division.

All relevant data are included in the paper or its Supplementary Information.