Illuminating utility benchmarking data with analysis and consumer feedback – insights from Nepal

The Government of Nepal, with the support of the Japan International Cooperation Agency (JICA), conducted a benchmarking activity with performance indicators, water quality tests, and a consumer survey for 26 larger and older town water supply services in the country. Performance indicators varied widely between the different towns: number of staff/1,000 connection (staff ratio), for example, ranged from 2.8 to 15.3, whereas water supply coverage ranged from 13.8% to 98.4%, and operating ratios which indicate financial performance ranged from 0.24 to 2.8. Critically poor biological water quality was found in water quality tests that 55% of tap water samples were Escherichia coli-contaminated. Overall, customer satisfaction ranged between 14% and 97%. Analysis showed that the operating ratio was significantly correlated with the staff ratio, the number of water supply connections, the water coverage, and the metered ratio. Results of water quality tests and the consumer survey revealed risk in direct drinking of tap water and a need to improve tap water quality by managing free residual chlorine and leakage reduction. Customers’ satisfaction with water sufficiency was found to be significantly correlated with their perception of the reliability of supply time rather than the number of supply hours and water consumption rate. doi: 10.2166/washdev.2019.159 om https://iwaponline.com/washdev/article-pdf/9/2/356/583182/washdev0090356.pdf 2019 R. Ogata (corresponding author) JICA Jordan Office, Amman, Jordan E-mail: ogata.ryuji2@gmail.com N. Khatri Thecho, Lalitpur, Nepal R. Ogata M. Sakamoto The University of Tokyo, Kashiwa, Chiba, Japan


INTRODUCTION
Goal 6 of the United Nations Sustainable Development Goals (SDG) 2016-2030 is comprised of a series of targets on water supply and sanitation. Drinking water issues are encompassed in Target 6.1 which reads: 'By 2030, achieve universal and equitable access to safe and affordable drinking water for all' (United Nations General Assembly ). The Government of Nepal has also set national targets to fulfil SDG Goal 6.1 (NPC ). One of the measures to determine if SDGs have been achieved in the water context is to establish benchmarking procedures. In a number of countries, some important experiences have been accumulated so far (Parena & Smeet ). As a way to monitor and accelerate efforts toward achieving national targets, the Government of Nepal has also started a benchmarking program. Water To support policy improvement and better investment in infrastructures related to larger town water supply in Nepal, this paper aims to describe the current status of town water supply in Nepal and to reveal the correlations between key performance indicators, drinking water quality, and customers' opinions on water supply. Figure 1 shows the locations of the 26 targeted towns, which are widely dispersed geographically throughout Nepal.

METHODOLOGY
The benchmarking activities were conducted subsequent to previous benchmarking undertaken in the fiscal year 2013-2014 (SEIU ), which targeted mainly water supply in small towns. The performance of each town service provider was evaluated with the nine key performance indicators as shown in Table 1. These indicators were selected considering availability and significance to the water supply sector of Nepal, although the International Benchmarking Network for Water and Sanitation Utilities (IBNET) adopted 12 categories of indicators (IBNET ) and three of them (i.e., pipe network performance, assets, and process indicators) were not included in this study. The primary reason is that the available information/data were limited with most of the water providers in Nepal. Moreover, a field survey of this study provided training for staff from each service provider to record key performance indicators every year. Thus, we tried to avoid indicators that required complicated calculations or caused additional workload for the staff.
The study team, as delegates of the Government of Nepal, visited each water supply provider and held an orientation meeting to train the staff to calculate the performance indicators that would be important for this study. During the orientation meeting, the staff were asked to complete a data sheet concerning institutional information (basic information about the provider), the number of the staff in each occupational category, water distribution systems, water coverage and service connection, customer service, water production, and revenue and expenses based on official documents (e.g., list of staff, record of pump operation hours, list of customers, billing documents for water tariffs, and accounting documents). The study team checked the provided data, and calculated the performance indicators on site, and shared the finalized results with the provider staff.
Water samples were collected from 21 major sources and 103 randomly selected taps in the Kathmandu valley as well as all water sources and five randomly selected taps in each of the other 25 selected towns. Figure 2 shows the tested water quality parameters. All 28 parameters in National Drinking Water quality Standard (NDWQS) in Nepal were tested for sources in the 26 towns, and a selection of 12 parameters was tested for taps in 25 of the towns, while a selection of 10 parameters was tested for taps in the Kathmandu valley.  Parameters for source water quality were selected to identify the potential risk to the drinking water in Nepal, while parameters for tap water quality were selected rather for practical use of water quality testing.
Six parameters -pH, turbidity, free residual chlorine (FRC), Escherichia coli, electrical conductivity and temperaturewere tested on-site with a field kit (DelAgua kit), and samples were transferred to a private laboratory (Water Engineering and Training Centre (P.) Ltd) in Kathmandu for testing of the remaining parameters.
Benchmarking is important for improving the quality and safety of water supply systems (Neunteufel et al. ).
Often overlooked, stakeholder (user) feedback is also an important tool in water supply systems decision-making (Dziedzic & Karney ), even though the end-users are the ultimate evaluators of the water systems in their daily lives. Therefore, a consumer survey was conducted, as well as evaluation with key performance indicators to confirm the perceptions of service and satisfaction level from the users' side. For the sample selection, the service area of each town was divided into five clusters based on the water supply system structure and the geographical conditions in order to collect unbiased opinions from the whole service areas. Interviews were conducted for the randomly selected 50 households by door-to-door visit covering 10 households from each cluster in each town. The questions concerning consumer's satisfaction include four criteriareliability of supply time, cleanliness of water, complaints response, and sufficiency of tap water.
The respondents were asked to choose one answer from three choicesgood, moderate, and bad. The answers were customized so that they made sense for each question (e.g., always, sometimes, and never were the answer options regarding the sufficiency of tap water). The customer satisfaction rate for each criterion was calculated based on the ratio of those who answered the good category for each question.

RESULTS AND DISCUSSION
The evaluation of the water supply systems using the perform- (AverageDailyWater Production À AverageDailyWater Consumption (Billing))=AverageDailyWaterProductionÃ100 The average daily water production was calculated based on pump capacity and operation hours for the cases where ground water or surface water was used as a water source.
For cases where gravity flow from surface source water was used as the water source, the calculation was based on the elevation difference and the diameter and length of the transmission pipe. The average daily water consumption was calculated based on the water bills.
Production cost was 2.5-23.6 (average 12.4) Nepal Rupees per cubic metre (NRs/m 3 ) (Note: 1 NRs ≒ 0.009 USD). These large differences among towns may come from geographic differences, such as mountainous and plain locations. The collection ratio was generally good, ranging from 0.7 to 1.0 (average: 0.9). The operating ratio was According to the results of the analysis, four factors were significantly correlated with the operating ratio while other factors did not show significant correlation with the operating ratio. The correlation coefficient R was calculated using a simple regression analysis. The highest correlation was found with staff ratio (R ¼ 0.74, P < 0.001) representing positive high correlation. This is because the personnel cost dominated the total expenditure of water supply providers, reaching 57% on average. Number of connections, water coverage, and metered ratio showed negative moderate correlation with operating ratio (i.e., R ¼ À0.63, P < 0.001, R ¼ À0.56, P < 0.01 and R ¼ À0.43, P < 0.05). Based on these results, it is presumed that improving these four factors (i.e., decreasing the staff ratio and increasing the number of connections, the water coverage, and the metered ratio) may contribute to improving the operating ratio. The NRW ratio and other performance indicators were not found to be significantly correlated with the operating ratio and, therefore, it is presumed that improving these factors is unlikely to contribute significantly to a more efficient operating ratio in the target towns.
The results of water quality tests in the Kathmandu valley and the other 25 towns are shown in Figure 3. The vertical axis of Figure 3 shows the ratio of water samples  pipelines through leakage points could be a cause of contamination of tap water. It is beyond our scope of the study to conclude and not able to do so with the currently available data. A further data collection (e.g., the ratio of leakage in NRW and the number of leakage points) is necessary to investigate which is the dominant factor of the biological contamination in each area.
Although serious arsenic contamination in ground water has been reported in the Terai region (Maharjan et al. ) where most of the target towns are located and where ground water is their main water source, only one water source showed an arsenic concentration (0.075 mg/L) higher than NDWQS. The reason for the low arsenic contamination observed may be that the aquifer used for urban water supply, which is relatively deep, is separated from the shallower aquifer used for the hand tube wells that have been observed to be contaminated (Islam et al. ).
Next, we investigated the result of the consumer survey.
Service quality of water supply is a vital factor from the perspectives both of public health and customer satisfaction.
In the consumer survey, user satisfaction was based on four criteria: reliability of supply time, cleanliness of water, complaints response, and sufficiency of tap water. Overall, the customer satisfaction rate as Good Service calculated based on the results of four questions ranged from a maximum of 97% to a minimum of 14% (average: 55%).
Correlation between the percentage of users drinking their water directly (direct drinking ratio) and physical water quality was analyzed. Judging from the results (R ¼ 0.72, P < 0.001) representing significant and high correlation, it is presumed that users may avoid direct drinking when they notice worse physical water quality due to aesthetic problems with the tap water. Most of the physical contamination in the surveyed towns was in the form of turbidity, which is visible and tangible.
In a similar fashion, correlation between chemical water quality and direct drinking ratio was analyzed, and the results (R ¼ 0.56, P < 0.01)representing significant and moderate correlationshow that worse chemical water quality could be a cause of a lower direct drinking ratio. Similar to the results for physical water quality, most of the chemical contamination in the surveyed areas was from iron, which can be observed in the smell and color of the tap water; therefore, users may avoid direct drinking due to aesthetics.
Finally, correlation between biological water quality and direct drinking ratio was analyzed and no significant corre- The quantity of supplied water is also a very important factor for the service level of water supply. The performance indicators for the quantity of water from the provider's side are service hours and water production rate. As for the water production rate, we redefined this as water consumption rate (Water consumption rate ¼ water production rate*(100 À NRW ratio)/100) for the following analysis because the actual amount of water received by the customer is the amount remaining after the reduction of the quantity of NRW from the total produced water. The consumer survey results for water sufficiency were expected to be correlated with both of these indicators. Nevertheless, the correlations between water sufficiency and both service hours and water consumption rate were not significant (R ¼ 0.33, P > 0.05 and R ¼ 0.16, P > 0.05), respectively. On the other hand, the correlation between two survey itemswater sufficiency and reliability of water supply timewas statistically significant and moderate (R ¼ 0.62, P < 0.001). The results may mean that customer satisfaction with water sufficiency is driven by the predictability of the water supply time rather than the total amount of water and/or service hours.

CONCLUSIONS
The following conclusions and recommendations are based on the study results. First, the operating ratiothe ratio of operating costs to sales revenuewas significantly correlated with the number of staff, the number of connections, the proportion of the population connected to the water supply (water coverage), and the percentage of connections that are metered. Therefore, improving performance related to these four indicators through business plan making and implementation may accelerate improvement in the operating ratio. Second, biological contamination is less noticeable to users compared with physical and chemical contamination.
Thus, it may be difficult for users to countermeasure against biological contamination by using filters or boiling, and more than half of the samples of tap water were biologically contaminated. There may be need to initiate activities to manage FRC and leakage reduction to prevent biological contamination, as well as to share information between users and service providers about water quality test results (usually done by the service providers). Third, the correlation between customer satisfaction on water sufficiency and the reliability of supply time was significant, whereas the correlations between customer satisfaction on water sufficiency and both the number of hours of service and water consumption rate (i.e., amount of supplied water) were not significant. Therefore, priority should be given to accuracy of supply time in a country with intermittent water supply such as Nepal.
Fourth, it seems that the effort to conduct such an interactive assessment from the provider's and customer's side, including key performance indicators, water quality, and a consumer survey, is beneficial for evaluating risks to the water supply and identifying appropriate responses both at the policy level and in the field. Specifically, these responses are: more effective and efficient policy and strategy making for urban water supply at the national level; institutional and infrastructure improvement plans for each water provider; and enhancement of user satisfaction, which will ultimately be vital for a sustainable water supply.