Investigation of water utility efficiency features: production variables and the DEA model choices

The quality of utility management is determined by the achievement of targets and service level benchmarks settings through regulated water tariffs. Improved revenue collection efficiency indicates non-revenue water (NRW) reduction and good water service delivery. Reducing NRW improves a utility's operating environment, economics, and most importantly, social trust (Gidion et al. 2019a; AL-Washali et al. 2020). To simulate the quality of services provided in a non-competitive operating environment (Pinto et al. 2017b), the performances of urban water utilities (UWUs) are compared using production variables referred to as key performance indicators (KPIs) (Singh et al. 2014; Thanassoulis & Silva 2018; Gidion et al. 2019b). Efficiency measurement involves the choice of a parametric or non-parametric benchmarking method (Lampe & Hilgers 2015; Gidion et al. 2022). Some parametric methods, such as stochastic frontier analysis (SFA) use a set of production variables (input and output), as does data envelopment analysis (DEA). It is well known that Charnes et al. (1978) and Banker et al. (1984) developed efficiency benchmarking methods with constant returns to scale (CRS) and variable returns to scale (VRS), respectively, which have been extended in various forms based on efficiency frontiers required by comparable decision-making units (DMUs). The extensions of DEA are classified mainly based on the production variables used, the technologies employed, and the number of analysis stages. The production variables can be categorised as absolute numbers or as ratios if the denominators of the variables are different (Hollingsworth & Smith 2003; Hatami-Marbini & Toloo 2019).

The majority of publications available in the water industry use input-oriented DEA for absolute data (Pinto et al. 2017a; Cabrera et al. 2018), which are available in CRS and VRS, and a few publications use DEA for ratio data (Gidion et al. 2019a), which are only available in VRS (Hollingsworth & Smith 2003). Recently, efficiency analysis using DEA has gained prominence in the water industry and is used to improve utilities' management capabilities (Pastor & Aparicio 2010). However, a few researchers use undesirable production variables to benchmark a DMU in the developed DEA models. For example, a recent publication by Lombardi et al. (2019) and lo Storto (2020) used absolute and ratio variables to generate utility efficiency using CRS and VRS DEA models developed to analyse absolute numbers, respectively.

Although a study by Romano & Guerrini (2011) employed DEA for absolute data to determine the efficiency of variables most often used to generate utility efficiency, UWU regulators use absolute data to a limited extent to rank the performance of a UWU (Berg 2010; Mugisha 2011). Regulators use KPIs to rank utility efficiency, which are ratio data in the efficiency measurement industry (Thanassoulis & Silva 2018). In general, the majority of studies extend DEA for absolute data to examine management features (Lombardi et al. 2019; lo Storto 2020). Using absolute numerical data to assess efficiency does not reflect the practical efficiency of a UWU (Alegre et al. 2016); therefore, assessing utility efficiency using ratios highlights the significant operational differences among UWUs (Gidion et al. 2019a, 2019b; Gidion et al. 2022). This study extends the use of DEA for ratio data proposed by Emrouznejad & Amin (2009) and Hatami-Marbini & Toloo (2019) and compares the utility efficiencies generated using DEA for absolute and ratio data.

DEA for absolute and ratio data are never compared to assess its suitability in water service delivery analysis, which is why most recent publications use DEA for absolute data to measure efficiency in the water industry (Lombardi et al. 2019; Wu et al. 2019; Xie et al. 2020). Furthermore, the types of input and output variables used to assess the efficiency of a UWU in DEA for absolute numbers do not allow assessing UWUs of different operation scales together (Romano & Guerrini 2011; Guthrie et al. 2017; Gidion et al. 2019a). It is well known that a large-scale UWU provides more information that is not likely to be readily available in a small-scale UWU (Haider et al. 2013) and using KPIs available in both large and small UWUs provides sound information to rank UWUs of different scales together.

This study presents DEA for absolute numbers and ratios to assess UWU efficiency. DEA for ratios uses absolute and ratio variables (KPIs) as inputs/outputs. Empirical studies such as Romano & Guerrini (2011) and Lombardi et al. (2019) examined the sustainability of UWU in efficiency analysis and concluded that the efficiency of UWU differs among firms with dissimilar scale characteristics (Worthington 2014). Based on the benchmarking technology of DEA for ratios, this study has a different perspective, as the assessment shows that the methodology ignores scale groups during the analysis and provides homogeneous efficiency results through the same input/output variables used, removing the scale size limitation of UWUs. KPIs provide an equivalent assessment of utility performance (Vilanova et al. 2015; Alegre et al. 2016) and thus, provide the opportunity for small-scale UWUs to be compared with large-scale UWUs and apply common management strategies to improve UWU efficiency.

During efficiency assessment, the number of measuring units should be more than three times the number of production variables to avoid DEA evaluation failure that produces efficiency results suffering from discrimination (Eskelinen 2017; Li et al. 2017); this means that even an underperforming unit can become efficient (Wagner & Shimshak 2007; Toloo et al. 2015). Thus, reducing the number of production variables to less than one-third of the measuring units reduces the chance that many units will become efficient. The selection of production variables should be done by experts because using unreliable or omitting relevant production inputs/outputs in the DEA analysis makes a large number of units efficient (Ruggiero 2005; Lombardi et al. 2019). Variable selection can be done using judgement screening, statistical tests, or DEA-based analysis to determine the importance of the original and influential input/output with the greatest impact (Golany & Roll 1989). However, in a limited environment, selecting production variables based on expert judgement alone is acceptable (Wagner & Shimshak 2007; Eskelinen 2017). Utility performance improvement depends on schemes that take into account the selection of input and output variables that focus directly on operating costs and, at the same time, on the quality of service provided; the majority of available publications in the water industry rely on production variables that have been widely used in previous studies, without understanding that time and technological changes represent a new working environment that nullifies the importance of some production variables. A few studies use correlation and regression analysis to determine the potential of selected production variables before DEA assessment. A highly correlated variable is considered redundant to avoid biased measurement (Ruggiero 2005; Wagner & Shimshak 2007; Lombardi et al. 2019).

The current study recommends the use of correlation analysis and analyses the efficiency of UWU using DEA for ratios. Thus, the omission of highly correlated variables depends on the assessment of the significance of the variable, because not all highly correlated variables are redundant, sometimes they are an indicator to examine a variable more closely (Golany & Roll 1989; Wagner & Shimshak 2007). Some researchers in the water industry have used ratios for assessing utility efficiency in DEA for absolute data. For example, Table 1 lists some publications that use DEA for absolute numbers (Equation (1)) and analyse production sets that include ratios and absolute numbers. The results analysed were used to derive observations and suggest measures to improve performance.

The confusion is due to the nature of a utility's operating environment, which provides information in the form of ratios to rank a utility's performance (Thanassoulis & Silva 2018) and DEA models in absolute data. The use of ratio variables in DEA for absolute data technically leads to incorrect results (Hollingsworth & Smith 2003; Emrouznejad & Amin 2009). In this study, absolute variables that make ratios are used to assess the efficiency of a utility using DEA for absolute data and DEA for ratios, respectively; the absolute and ratio information generated is verified based on the results of correlation analysis. In the case of more than one redundant variable, the study uses judgement and knowledge to determine the similarity and importance of the variables in relation to utility efficiency- and management-specific features.

Using standard DEA for absolute numbers, Wagner & Shimshak (2007) proposed a stepwise variable selection approach that involves efficiency analysis by variables redundant with constant units till the number of efficient units is reduced with a reduced number of production variables. This study arguably again criticises the fact that practitioners cannot rely upon the proposed approach, because a utility cannot be measured using one input and output variable over several production sets of variables used during service delivery and conclude on managerial improvement. Gidion et al. (2019a) developed a network-DEA model for ratios analysing utility efficiency in the same way as Wagner & Shimshak (2007); however, the recent study benchmarks utility efficiency with units and production variables that are redundant. Gidion et al. (2019a) study included utilities with zero values in the production variables and benchmarked them together with utilities that outperformed targets in their production variables; the analysis benchmarks a utility under a yardstick competition regime while still allowing for the examination of efficiency- and management-specific features. The two studies are presented to illustrate an example of two different DEA models that use different production variables in their analysis. DEA for absolute numbers uses a linear programme technology, while DEA for ratios uses a non-linear programme technology to benchmark utility efficiency. DEA benchmarks utility efficiency in input or output orientations; the input DEA model aims to minimise input and increase outputs, while the output orientation aims to minimise output and maximise input (Charnes et al. 1978; Banker et al. 1984).

Equations (1) and (2) present VRS DEA models for absolute numbers and ratios, respectively, employed to compare the efficiencies of a UWU. Thus, consider a set of z UWUs, each consuming i input to generate r output in a limitation that Let and ⁠, respectively, represent the vectors of m consumed input ratio(s) and s output ratios. and represent a numerator and denominator of the input (⁠⁠), and and represent a numerator and denominator of the output (⁠⁠) for ⁠. In the DEA model, and composed of the known input ratio and output-ratio vectors, respectively, of the target UWU₀, is a vector describing the percentages of other producers used for constructing the virtual producer, and is the producer's efficiency score.

These input-oriented DEA models have been used to investigate managerial features in drinking water companies, while the output-oriented DEA models are mostly used to measure the efficiency of wastewater companies. The VRS form of DEA is mostly used to investigate the efficiency of UWUs; this is due to the nature of UWUs operating under VRS (Berg & Lin 2008; Berg 2010). In practice, DEA for absolute numbers has been extended in many forms compared to DEA for ratios; the standard DEA models developed by Charnes et al. (1978) and Banker et al. (1984) are the original models. Studying the efficiency of a UWU requires an appropriate DEA model that examines the efficiency- and managerial-specific features, wrong models have been used to investigate managerial efficiencies (Singh et al. 2014; Lombardi et al. 2019). In this study, the standard input-oriented VRS DEA models for absolute and ratio data are used to ascertain their features and reliability in benchmarking utility efficiency; the two kinds of DEA models have been previously employed in UWU efficiency assessment. However, in a limited condition, DEA for ratio data have used. This study analyses a utility efficiency using DEA for absolute data developed by Banker et al. (1984) and DEA for ratio data extended by Emrouznejad & Amin (2009) and later modified by Hatami-Marbini & Toloo (2019).

The study focuses on Tanzanian regional utilities to produce a set of utility characteristics that significantly identify the proper DEA model suitable to assess and investigate efficiency- and management-specific features of a utility. This study employs the same data used in the Gidion et al. (2019b) study, which analysed 32 utilities with 10 production variables. Due to the reason that the assessment focuses on comparing DEA for absolute numbers and ratios, the analysis considers two kinds of data to investigate UWU's efficiencies- and management-specific features. Gidion et al. (2019b) used information published by the EWURA (2018). EWURA (Energy and Water Utilities Regulatory Authority) is an autonomous multi-sectoral regulatory authority established under Act Cap 414′ of the laws of Tanzania. Recently, the EWURA monitored the performance of 94 utilities that are geographically distributed and publicly owned, as shown in Tables 2 and 3.

The regulator uses empirical analysis to rank the performance of 94 utilities; during ranking, regional and national projects UWUs are ranked together, whereas district towns UWUs are ranked separately. District towns UWUs are considered small-scale UWUs and regional and national projects UWUs are considered large-scale UWUs. In our analysis, we are comparing the efficiencies of regional and national projects UWUs to investigate the quality of DEA for absolute and ratio variables in the UWU efficiency investigation. The KPIs used by the EWURA to rank UWUs are also published in the Alegre et al. (2016) study; using the input/output (I/O) coding provided in this study to simplify the analysis, Alegre et al. (2016) defined the KPIs employed in Gidion et al. (2019b) as follows:

• (I)A: NRW (%) is the ratio of the difference between the volume of water produced and the actual revenue volume of water to the volume of water produced by a utility. The difference between the volume of water produced and the actual revenue volume of water is referred to as an absolute number.

• (I)B: Working ratio (–) is the ratio of operational expenses to operational revenue. To generate a working ratio, depreciation, interests, and debit services are excluded from the operational expenses. Operational expenses are an absolute variable.

• (I)C: Operating ratio (–) is the ratio of operating costs to operational revenues. To generate an operating ratio, depreciation, interests, and debit services are included in the analysis of the operational costs. The operating revenue is an absolute variable.

• (I)D: Personnel expenditure (%) is the personnel expenditure expressed as a percentage of total revenue collection. Personnel expenditure is an absolute variable.

• (I)E: Staff/1,000 connections (FTE/1,000 Con.) is an indicator that measures the ratio of staff distribution per available connections/customers. The number of staff is an absolute variable.

• (O)A: Proportion of the population served with water (%) is the percentage of the population connected to water service compared to the total population of the area with water service extension. Population served is an absolute variable.

• (O)B: Average hours of supply (%) is the percentage of average daily hours used to pressurise the system. The actual average water supply hours are counted as an absolute variable.

• (O)C: Metering ratio (%) is the ratio of metered customers to total customers registered in a utility database expressed in percentage. The number of active metered connections is an absolute variable.

• (O)D: Revenue collection efficiency (%) is the ratio of revenue collected to the billed amount expressed in percentage. The revenue collected is counted as an absolute variable.

• (O)E: Water quality compliance (%) is the ratio of the number of samples that have passed a water quality test to the total samples collected. The number of samples that passed the test is an absolute variable.

The numerator of the KPI is an absolute number used in DEA for absolute data analysis, and the ratio variable that results in KPI analysis is the data used in DEA for ratio analysis. Because DEA for absolute numbers uses part of the information used in DEA for ratios, the study applies correlation analysis generated from the KPIs to identify redundant variables in the analysis, as previously discussed. The information obtained from the correlation analysis of the KPI variables is presented in Table 4.

Table 4 presents the correlation matrix for 10 ratio variables; the analysis shows that the correlations between input-to-input and output-to-output correlations are positive, while the correlations between inputs and outputs are negative. The positive observations signify that the independent and dependent variables increase in value simultaneously, while the negative observation signifies that the independent variable increases while the dependent variable decreases in value (Clemen & Reilly 1999; Yang et al. 2019). The correlation identity guide from Yang et al. (2019) helps to determine the degree of correlation of the variables analysed. If the correlation degree is 0.000, it means that the correlation degree is independent, if the correlation value is between 0.000 and 0.300, the correlation degree is a micro correlation. If the correlation coefficient is between 0.300 and 0.500, the correlation degree is termed a real correlation, if the correlation coefficient is between 0.500 and 0.800, the correlation degree is a significant correlation. Note that the correlation range is between +1 and −1, if the coefficients range above 0.800 is termed as highly correlated. Referring to the analysis in Table 4, the majority of the correlations of the variables fall within the micro- and real-correlation degrees. None of the variables are highly correlated. However, variable (O)D, which is the revenue collection efficiency should be considered redundant because it is the opposite of NRW; reducing NRW will increase revenue collection efficiency and vice versa. Furthermore, variables (I)B and (I)C use the same numerator name differentiated with either depreciation, interests, and debit services are included or excluded thus, to avoid confusion in the analysis variable (I)C is counted as redundant. Thus, our new variables to use in DEA for absolute numbers and ratios are indicated in Table 5.

Investigating utilities management-specific features starts after comparing the degree of achievement of established performance indicators (PIs) and best practices. The International Water Association (IWA) and other water industry stakeholders have established several PIs, and regulators summarise some of them and rank utilities' performance. Regulators use alternative techniques that involve empirical analysis by using the PIs as input variables and identifying a utility with excellent, medium, and low performance on selected PIs in a competitive scenario (Davis 2011; Chen & Chen 2014). However, the approaches are prone to human error and favouritism (Gidion et al. 2019b). DEA for absolute data cannot use PIs as input variables, and when they are used in the analysis, the results are not reliable because the convexity axiom is not satisfied; DEA for ratio data overcomes the convexity problem when analysing UWU with ratio data (Hatami-Marbini & Toloo 2019). Aggregating UWUs and assessing their efficiency using PIs opens up the possibility for a researcher to identify a utility that outperforms the level of several PIs compared to other utilities, where a weak-performing utility will have a chance to learn how to improve the underlying poorly rated PIs. Regulators use the same approach to examine through empirical analysis. and propose measures for performance improvement. The efficiency assessment proposed in this study does not rely on assumptions, thus avoiding human errors. Figure 1 shows the trends of efficiency assessment using DEA for absolute and ratio data; DEA for ratios significantly outperformed DEA for absolute data in efficiency benchmarking and has added value to improve the economic dimension of a utility. Examining the level of PIs achieved through efficiency benchmarking and proposing measures to improve performance improves not only economic factors but also environmental and social trust factors, which are indicators of utility sustainability (Molinos-Senante et al. 2016).

Multidimensional efficiency assessment can be achieved by aggregating a set of selected PIs, referred to as production variables; representativeness, relevance, reliability, sensitivity, understandability, comparability, and transparency are the criteria recommended in the PIs selection (Vilanova et al. 2015; Molinos-Senante et al. 2016). Due to the nature of the utility operating environment, which includes the provision of water services in an area without other competitors, the use of PIs in efficiency assessments provides holistic utility management in different environments (Marques & Monteiro 2001; Haider et al. 2016). The role of PIs in utility efficiency assessment remains important (Alegre et al. 2016; Molinos-Senante et al. 2016); management complexity is determined upfront after efficiency assessment by combining efficiency score and production variables (Xie et al. 2020). Based on the researcher's preferences, PIs are classified into different groups (Marques & Monteiro 2001; Molinos-Senante et al. 2016); however, based on the importance of sustainability of services, PIs are classified into economic, environmental, and social dimensions (Vilanova et al. 2015). Global technological progress is on the rise and the importance of economic management has improved the utility's operating environment and social trust. Advances in the design of pumps that operate optimally and cost-effectively are improving the operating environment. However, the increase of water losses in the system slows the efforts. Efficiency assessment has the general objective of improving performance in utility management, and efficiency scores and production variables quantify specific efficiency measures. While efficiency scores rank a utility's position within a group of DMUs, PIs are used to make decisions about the efficiency achieved and to verify the effectiveness of optimisation measures that an efficient utility uses to improve efficiency that are appropriate for an inefficient utility. In this way, the adaptation of management sharing techniques between utilities to improve efficiency is encouraged; the measure leads utilities to work together as a group without regard to the environmental barrier (Korhonen & Syrjänen 2004; Lejarraga & Müller-Trede 2017).

According to Berg (2010), regulators have used empirical methods to rank utility performance and identify specific management feature improvements; empirical methods are prone to human error. DEA is a non-parametric efficiency assessment method; human error does not exist during the analysis and the selection of the DEA model for UWU efficiency assessment depends on available production data. The assessment shows that the use of inappropriate variable data in the analysis leads to incorrect efficiency results that cannot be relied upon in decision-making. Production variables and choosing the right DEA model in efficiency assessment is one of the best approaches during efficiency assessment. Using ratio data in DEA for absolute numbers increases the number of efficient UWUs; thus, underperforming UWUs identified efficiently due to incorrect data selection can also be included in efficiency- and management-specific feature investigations.

The majority of available publications investigate UWU management features using DEA for absolute data. However, DEA for absolute data has been extended to analyse ratios, which overcomes the convexity problem in DEA for absolute data when ratio data are used. The assessment shows that DEA for ratios achieves efficiency gains that can be relied upon. The use of DEA for ratios helps researchers and regulators identify efficiency- and management-specific features based on efficiency outcomes and production sets (KPIs) that are used. DEA for ratio data has been extended in the water industry and compares the efficiency of UWUs in a competitive scenario, as the empirical analysis does (Gidion et al. 2019a, 2019b, 2022).

The assessment of this study is limited to variables and DEA model choices; the remark helps researchers and practitioners to conclude the best management features for efficient utility management. Future researchers should explore efficiency- and management-specific features using proper production variables and the DEA model.

All relevant data are available from an online repository or repositories. https://data.mendeley.com/datasets/2mz55y3b4w.

The authors declare there is no conflict.