Illuminating utility benchmarking data with analysis and consumer feedback – insights from Nepal

Goal 6 of the United Nations Sustainable Development Goals (SDG) 2016–2030 is comprised of a series of targets on water supply and sanitation. Drinking water issues are encompassed in Target 6.1 which reads: ‘By 2030, achieve universal and equitable access to safe and affordable drinking water for all’ (United Nations General Assembly 2015). The Government of Nepal has also set national targets to fulfil SDG Goal 6.1 (NPC 2015). One of the measures to determine if SDGs have been achieved in the water context is to establish benchmarking procedures. In a number of countries, some important experiences have been accumulated so far (Parena & Smeet 2001). As a way to monitor and accelerate efforts toward achieving national targets, the Government of Nepal has also started a benchmarking program. Water supply service in Nepal is operated and maintained mainly by water users and sanitation committees (WUSCs) in rural areas and small towns. The water supply systems in the larger and older towns (i.e., historic towns established a long time ago) are managed by the Nepal Water Supply Corporation (NWSC) or by the Water Supply Management Board (WSMB). In the Kathmandu Valley specifically, they are managed by the Kathmandu Valley Water Supply Management Board (KVWSMB) and operated by Kathmandu Upatyaka Khanepani Limited (KUKL). With the technical and financial support of the Japan International Cooperation Agency (JICA), the benchmarking program was conducted in 26 towns where the water systems are managed by the NWSC (23), WSMB (2), and KVWSMB/KUKL (1). The population of these 26 towns ranges from 3,200 to 300,000 in 25 of the towns and 2,560,000 in Kathmandu valley (i.e., the capital city of Nepal). To support policy improvement and better investment in infrastructures related to larger town water supply in Nepal, this paper aims to describe the current status of town water supply in Nepal and to reveal the correlations between key performance indicators, drinking water quality, and customers' opinions on water supply.

Figure 1 shows the locations of the 26 targeted towns, which are widely dispersed geographically throughout Nepal. The benchmarking activities were conducted subsequent to previous benchmarking undertaken in the fiscal year 2013–2014 (SEIU 2015), which targeted mainly water supply in small towns. The performance of each town service provider was evaluated with the nine key performance indicators as shown in Table 1. These indicators were selected considering availability and significance to the water supply sector of Nepal, although the International Benchmarking Network for Water and Sanitation Utilities (IBNET) adopted 12 categories of indicators (IBNET 2018) and three of them (i.e., pipe network performance, assets, and process indicators) were not included in this study. The primary reason is that the available information/data were limited with most of the water providers in Nepal. Moreover, a field survey of this study provided training for staff from each service provider to record key performance indicators every year. Thus, we tried to avoid indicators that required complicated calculations or caused additional workload for the staff.

The study team, as delegates of the Government of Nepal, visited each water supply provider and held an orientation meeting to train the staff to calculate the performance indicators that would be important for this study. During the orientation meeting, the staff were asked to complete a data sheet concerning institutional information (basic information about the provider), the number of the staff in each occupational category, water distribution systems, water coverage and service connection, customer service, water production, and revenue and expenses based on official documents (e.g., list of staff, record of pump operation hours, list of customers, billing documents for water tariffs, and accounting documents). The study team checked the provided data, and calculated the performance indicators on site, and shared the finalized results with the provider staff.

Water samples were collected from 21 major sources and 103 randomly selected taps in the Kathmandu valley as well as all water sources and five randomly selected taps in each of the other 25 selected towns. Figure 2 shows the tested water quality parameters. All 28 parameters in National Drinking Water quality Standard (NDWQS) in Nepal were tested for sources in the 26 towns, and a selection of 12 parameters was tested for taps in 25 of the towns, while a selection of 10 parameters was tested for taps in the Kathmandu valley. Parameters for source water quality were selected to identify the potential risk to the drinking water in Nepal, while parameters for tap water quality were selected rather for practical use of water quality testing.

Six parameters – pH, turbidity, free residual chlorine (FRC), Escherichia coli, electrical conductivity and temperature – were tested on-site with a field kit (DelAgua kit), and samples were transferred to a private laboratory (Water Engineering and Training Centre (P.) Ltd) in Kathmandu for testing of the remaining parameters.

Benchmarking is important for improving the quality and safety of water supply systems (Neunteufel et al. 2010). Often overlooked, stakeholder (user) feedback is also an important tool in water supply systems decision-making (Dziedzic & Karney 2015), even though the end-users are the ultimate evaluators of the water systems in their daily lives. Therefore, a consumer survey was conducted, as well as evaluation with key performance indicators to confirm the perceptions of service and satisfaction level from the users' side. For the sample selection, the service area of each town was divided into five clusters based on the water supply system structure and the geographical conditions in order to collect unbiased opinions from the whole service areas. Interviews were conducted for the randomly selected 50 households by door-to-door visit covering 10 households from each cluster in each town. The questions concerning consumer's satisfaction include four criteria – reliability of supply time, cleanliness of water, complaints response, and sufficiency of tap water. The respondents were asked to choose one answer from three choices – good, moderate, and bad. The answers were customized so that they made sense for each question (e.g., always, sometimes, and never were the answer options regarding the sufficiency of tap water). The customer satisfaction rate for each criterion was calculated based on the ratio of those who answered the good category for each question.

Production cost was 2.5–23.6 (average 12.4) Nepal Rupees per cubic metre (NRs/m³) (Note: 1 NRs ≒ 0.009 USD). These large differences among towns may come from geographic differences, such as mountainous and plain locations. The collection ratio was generally good, ranging from 0.7 to 1.0 (average: 0.9). The operating ratio was 0.24 to 2.8 (average: 1.3), showing significant differences between each town. Moreover, a deficit of financial status was found in operation and maintenance in 14 out of 26 towns. Only one town fulfilled the expected operating ratio value (0.5) set by the Government of Nepal. Finally, service hours varied from 2 to 12 (average: 6.5) h/day. The expected number of service hours set by the Government of Nepal is 8 h/day, though all the selected towns adopt intermittent water supply due to the shortage of source water.

Operating ratio is the most critical parameter concerning water supply in Nepal because more than half of the studied water supply providers demonstrated a financial deficit in operation and maintenance. This means that sound management of the water supply in those towns was quite difficult – eventually, the water supply might not be able to continue for the future. Thus, correlation analysis was implemented to investigate the causal relationship between the operating ratio and other factors such as the remaining eight performance indicators and the water quality test results, as well as the number of connections that can describe the actual scale of the water supply for each town.

According to the results of the analysis, four factors were significantly correlated with the operating ratio while other factors did not show significant correlation with the operating ratio. The correlation coefficient R was calculated using a simple regression analysis. The highest correlation was found with staff ratio (R = 0.74, P < 0.001) representing positive high correlation. This is because the personnel cost dominated the total expenditure of water supply providers, reaching 57% on average. Number of connections, water coverage, and metered ratio showed negative moderate correlation with operating ratio (i.e., R = −0.63, P < 0.001, R = −0.56, P < 0.01 and R = −0.43, P < 0.05). Based on these results, it is presumed that improving these four factors (i.e., decreasing the staff ratio and increasing the number of connections, the water coverage, and the metered ratio) may contribute to improving the operating ratio. The NRW ratio and other performance indicators were not found to be significantly correlated with the operating ratio and, therefore, it is presumed that improving these factors is unlikely to contribute significantly to a more efficient operating ratio in the target towns.

The results of water quality tests in the Kathmandu valley and the other 25 towns are shown in Figure 3. The vertical axis of Figure 3 shows the ratio of water samples that met NDWQS for each category at source and tap. A comparison of physical and chemical water quality test results between source and tap is not meaningful because the number of parameters tested in source and tap water was different. The parameters not meeting NDWQS in source water were color, pH, turbidity in physical parameters, aluminum, ammonia, arsenic, iron, manganese, nitrate in chemical parameters, and E. coli as a biological parameter while those in tap water were color, pH, turbidity in physical parameters, ammonia, iron, nitrate, nitrite in chemical parameters, and E. coli as a biological parameter and FRC. The physical and chemical water quality at both source and tap fulfilled NDWQS in the range of 78–98% for 26 of the towns. While biological water quality was critically poor at the taps in all 26 towns: only 44% and 46% of samples met the NDWQS in the Kathmandu valley and in the other 25 towns, respectively. Test results demonstrated the situation to be particularly critical in rural areas where 82.2% of household water samples were E. coli-contaminated (NPC 2017). FRC was also generally poor, meeting the NDWQS in only 10% of the samples at taps.

There are two possible reasons for the biological contamination in general. First, poor maintenance of the FRC by providers could cause the low quality of water at the tap; second, intrusion of contaminated water to the pipelines through leakage points could be a cause of contamination of tap water. It is beyond our scope of the study to conclude and not able to do so with the currently available data. A further data collection (e.g., the ratio of leakage in NRW and the number of leakage points) is necessary to investigate which is the dominant factor of the biological contamination in each area.

Although serious arsenic contamination in ground water has been reported in the Terai region (Maharjan et al. 2006) where most of the target towns are located and where ground water is their main water source, only one water source showed an arsenic concentration (0.075 mg/L) higher than NDWQS. The reason for the low arsenic contamination observed may be that the aquifer used for urban water supply, which is relatively deep, is separated from the shallower aquifer used for the hand tube wells that have been observed to be contaminated (Islam et al. 2008).

Next, we investigated the result of the consumer survey. Service quality of water supply is a vital factor from the perspectives both of public health and customer satisfaction. In the consumer survey, user satisfaction was based on four criteria: reliability of supply time, cleanliness of water, complaints response, and sufficiency of tap water. Overall, the customer satisfaction rate as Good Service calculated based on the results of four questions ranged from a maximum of 97% to a minimum of 14% (average: 55%). Respective results of the answer for four criteria were the following: reliability of supply time (good – 55%, moderate – 39%, bad – 10%), cleanliness of water (good – 53%, moderate – 38%, bad – 9%), complaints response (good – 53%, moderate – 38%, bad – 9%), and sufficiency of tap water (good – 62%, moderate – 29%, bad – 9%). In addition to the four criteria, users were asked about their method for preparing water for drinking (using filters or boiling). The results were that 59% of users drank water directly (i.e., no filtering or boiling), whereas 11% boiled water and 30% filtered water before drinking it.

Correlation between the percentage of users drinking their water directly (direct drinking ratio) and physical water quality was analyzed. Judging from the results (R = 0.72, P < 0.001) representing significant and high correlation, it is presumed that users may avoid direct drinking when they notice worse physical water quality due to aesthetic problems with the tap water. Most of the physical contamination in the surveyed towns was in the form of turbidity, which is visible and tangible.

In a similar fashion, correlation between chemical water quality and direct drinking ratio was analyzed, and the results (R = 0.56, P < 0.01) – representing significant and moderate correlation – show that worse chemical water quality could be a cause of a lower direct drinking ratio. Similar to the results for physical water quality, most of the chemical contamination in the surveyed areas was from iron, which can be observed in the smell and color of the tap water; therefore, users may avoid direct drinking due to aesthetics.

Finally, correlation between biological water quality and direct drinking ratio was analyzed and no significant correlation was found between them (R = 0.36, P > 0.05). It can be inferred that biologically deteriorated water is unlikely to be identified by users aesthetically and, therefore, users may not refrain from direct drinking of tap water with biological contamination. A number of tap water sources were contaminated physically, chemically, and/or biologically based on the results of water quality tests in this study. Nevertheless, it is very difficult for users to judge the water quality if they do not have specific measures to test the water quality accurately. In general, water supply providers in Nepal do not announce to the users if the tap water is drinkable. Thus, the consumers have to determine whether they should use a filter, boil the water, or drink it directly based on the experiences in their daily lives. It is presumed that biological contamination is less noticeable and, thus, difficult to avoid with user countermeasures, compared with physical and chemical contamination, even though 54–56% of tap water contained E. coli (faecal contamination) in the target towns.

The quantity of supplied water is also a very important factor for the service level of water supply. The performance indicators for the quantity of water from the provider's side are service hours and water production rate. As for the water production rate, we redefined this as water consumption rate (Water consumption rate = water production rate*(100 − NRW ratio)/100) for the following analysis because the actual amount of water received by the customer is the amount remaining after the reduction of the quantity of NRW from the total produced water. The consumer survey results for water sufficiency were expected to be correlated with both of these indicators. Nevertheless, the correlations between water sufficiency and both service hours and water consumption rate were not significant (R = 0.33, P > 0.05 and R = 0.16, P > 0.05), respectively. On the other hand, the correlation between two survey items – water sufficiency and reliability of water supply time – was statistically significant and moderate (R = 0.62, P < 0.001). The results may mean that customer satisfaction with water sufficiency is driven by the predictability of the water supply time rather than the total amount of water and/or service hours.

The following conclusions and recommendations are based on the study results. First, the operating ratio – the ratio of operating costs to sales revenue – was significantly correlated with the number of staff, the number of connections, the proportion of the population connected to the water supply (water coverage), and the percentage of connections that are metered. Therefore, improving performance related to these four indicators through business plan making and implementation may accelerate improvement in the operating ratio. Second, biological contamination is less noticeable to users compared with physical and chemical contamination. Thus, it may be difficult for users to countermeasure against biological contamination by using filters or boiling, and more than half of the samples of tap water were biologically contaminated. There may be need to initiate activities to manage FRC and leakage reduction to prevent biological contamination, as well as to share information between users and service providers about water quality test results (usually done by the service providers). Third, the correlation between customer satisfaction on water sufficiency and the reliability of supply time was significant, whereas the correlations between customer satisfaction on water sufficiency and both the number of hours of service and water consumption rate (i.e., amount of supplied water) were not significant. Therefore, priority should be given to accuracy of supply time in a country with intermittent water supply such as Nepal. Fourth, it seems that the effort to conduct such an interactive assessment from the provider's and customer's side, including key performance indicators, water quality, and a consumer survey, is beneficial for evaluating risks to the water supply and identifying appropriate responses both at the policy level and in the field. Specifically, these responses are: more effective and efficient policy and strategy making for urban water supply at the national level; institutional and infrastructure improvement plans for each water provider; and enhancement of user satisfaction, which will ultimately be vital for a sustainable water supply.