Abstract

Over the past two decades, performance assessment based on performance indicators (PI) has been one of the areas showing the greatest advancement in the water sector. This practice may be defined as any approach that aims to measure effectiveness and/or efficiency of an activity or process using PI, supporting the decision-making process and encouraging the continuous improvement of a water utility. Despite the potential benefits, few projects and initiatives of performance assessment based on PI have been undertaken regarding storm water systems. The present article aims to review studies that have been carried out in the area to provide a state of the art on the subject, understand the applicability of the existing methodologies and their limitations, and to point out future directions. The review revealed that most of the research works were of limited scope, focusing on single storm water systems or some specific aspects of their performance, and also gave an inappropriate definition and selection of PI, compromising their further application. The identified gaps emphasise the need for a new methodology to provide a reference framework in the field. Such methodology should be objective, standardised and well-adapted to different contexts, driven by objectives and criteria and based on PI and targets.

Introduction

Over the past decades, performance assessment has become a common practice in the water sector (Cabrera Jr & Pardo, 2008), with a huge number of projects and initiatives being undertaken (Alegre & Cabrera, 2011). It has essentially arisen due to the need to create artificial competition mechanisms and promote effectiveness and efficiency in this sector of monopolistic nature (Alegre, 1999). The main areas in the urban sector in which this practice has been carried out are: drinking water production, water supply, drinking water storage and distribution, wastewater collection and treatment (Vieira, 2009).

It was predominantly in the late 1990s that important efforts at developing performance assessment systems based on PI took place, the regulators of water services being the main drivers (Alegre et al., 2009). Some of the first initiatives were conducted by, for example, the Office of Water Services (Ofwat, 1996a, 1996b); the International Water Association (IWA) (Alegre et al., 2000; Matos et al., 2003); the American Water Works Association Research Foundation (AWWARF, 1996); the World Bank (Yeppes & Dianderas, 1996); the Asian Development Bank (Mcintosh & Yñoguez, 1997); the six cities group from Scandinavian countries (Stockholm, Gothenburg, Malmo, Copenhagen, Oslo and Helsinki) (Adamsson, 1997); and the Malaysian Water Association (MWA, 1996). Also highlighted, more recently, are the ISO24500 standards (ISO 24510, 24511, 24512) (ISO, 2007a, 2007b, 2007c), which are for voluntary application and provide guidelines for the assessment of services related to the management of water supply and wastewater systems, having as their objective the continuous improvement of the level of service provided to the user/customer at optimal cost (Rohrhofer et al., 2008).

According to the ISO24500 standards (ISO, 2007a, 2007b, 2007c), assessment is a ‘process, or the result of this process, comparing a specified subject matter to relevant references’. Regarding this, performance assessment is any approach that permits evaluation of the efficiency and/or the effectiveness of a process or activity using performance metrics (Alegre & Coelho, 2012), such as PI.

Due to the inherent complexity of water services, with multiple constraints and drivers (macroeconomic, social and environmental), the use of performance assessment systems can bring benefits to water utilities. Some of those benefits are (van den Berg & Danilenko, 2011; Vilanova et al., 2015; Alegre et al., 2016): measuring the quality of service and the utility's effectiveness and efficiency; supporting the decision-making process; identifying areas for improvement; making transparent the comparison between objectives; providing benchmarking with similar utilities in the country or region or with standards of international good practice; aiding dissemination of results via marketing; and encouraging utilities to provide an improved service. Apart from water and wastewater utilities, other entities can benefit from the use of performance assessment based on PI, such as: national or regional policy-making bodies, consumers, regulatory agencies, financing bodies and also multilateral organisations (Matos et al., 2003; Alegre et al., 2016).

Performance assessment systems may also be part of broader management approaches, such as: quality management programmes, risk management, benchmarking and infrastructure asset management (IAM) (Baptista & Alegre, 2009). In the case of benchmarking, performance assessment using PI plays a key role in the process.

Benchmarking is meant to improve performance through systematic search and adaptation of leading practices (Cabrera Jr et al., 2011) by comparing PI with reference standards (Marques & De Witte, 2010). It is frequently carried out in more mature organisations, and it results from the need for transparent and standardised information with which to compare utilities' performance (Cabrera Jr et al., 2011). This has led to an increased emphasis on measurement of results, on transparency and on accountability (van den Berg & Danilenko, 2011). IWA has made an important contribution to benchmarking initiatives through the Benchmarking and Performance Assessment Specialist Group, which is an international forum responsible for the development of IWA frameworks of performance indicators (Matos et al., 2003; Alegre et al., 2016) and benchmarking for water and wastewater services. The standardised approaches used to define PI led to a broad consensus on water utilities worldwide (Nafi et al., 2015), especially in European countries (Marques & De Witte, 2010). Besides the IWA contributions, there are other benchmarking initiatives worldwide based on performance assessment. Some examples are: the South East Asia Water Utility Network (SEAWUN); the Union of African Water Suppliers' Water Utilities Partnership (WUP); the Pacific Water and Wastes Association (PAWWA); the Association of Water and Sanitation Regulatory Entities of the Americas (ADERASA); and the World Bank's International Benchmarking Network for Water and Sanitation Utilities (IBNET).

Even though performance assessment based on PI has been revealed to be a powerful tool within the water sector, not all systems are at the same level of application. While water supply and wastewater systems have been the main beneficiaries of the advances in the performance assessment field, such practice has not been fully adopted for storm water systems.

Storm water systems are a relevant part of the urban infrastructure. Their main functions are to collect storm water to reduce flooding, to avoid property damage and health risks occurring, and to prevent potential environmental impacts (Butler & Davies, 2011). In the context of climate change and rapid urbanisation, with changes in rainfall patterns, land uses and sea level, the ability of systems to comply with performance requirements may be severely affected. Furthermore, these systems demand great financial effort for installation, maintenance, rehabilitation and repair works, creating great challenges for system management.

Given this, promoting a wide application of performance assessment based on PI to storm water systems is of utmost importance. It would help to attain many of the recognised benefits already observed for water and wastewater systems. Above all, it would make a contribution to knowing storm water systems' behaviour and their vulnerabilities to support decisions and improve their management.

It is, then, essential to know what has been the current research in the field and its main gaps, to define a suitable approach to help change the paradigm. In this regard, the main objectives of the research article are: (1) reviewing PI that have been used or proposed for storm water systems; (2) discussing their applicability; (3) identifying limitations and barriers for a wider application; and (4) presenting current and future work to assess storm water systems' performance. It is important to note that this review's focus is the storm water system and not the service, excluding aspects related to the management activity of water utilities (e.g., management of human resources, financial performance, level of response to users' complaints). A brief description of the different types of storm water systems and their functions is presented as well as concepts related to performance assessment in the water sector.

Storm water systems

For many years, storm drainage systems have been designed and implemented with the sole purpose of removing storm water from the cities as quickly as possible to avoid flooding occurrences (Butler & Davies, 2011). These conventional systems basically convey runoff water from impervious streets and pavements through drainage channels and underground pipes, discharging it into streams and rivers (Charlesworth, 2010; Zhang et al., 2017).

Conventional pipe systems can be classified as combined and separate systems. In combined systems, both wastewater and storm water are collected in the same pipe network, whereas in separate systems these two types of water are conveyed by two distinct pipe networks. Some towns have mixed systems, with parts of the network being combined and other parts being separate.

Combined systems were the most widely implemented systems serving populations for decades. Around the 1950s, in Europe and in the USA, these systems were perceived as a menace to receiving waters due to the combined sewer overflows (CSOs) (Toffol, 2006). These structures were used to divert excess storm water into a nearby receiving water, thus being a great source of pollution (Lau et al., 2002). For this reason, it started to be recommended to consider separate sewers for the construction of new systems in order to reduce the frequency of CSOs (Butler & Davies, 2011). But as time went by and more studies were published, the notion that storm water was ‘clean’ and that a separate system would have a great impact on pollution reduction per se, started to be put aside.

Despite their high representativeness in developed countries, conventional storm water systems have been revealed to be inadequate to cope with either the demands resulting from both widespread urbanisation and climate change's foreseeable effects on rainfall patterns and sea level rise, or with environmental concerns, due to their limited capacity and flexibility (Zhou, 2014). As a response, over recent decades, a vast number of alternative drainage solutions have been conceived (Winz et al., 2011; Zhang et al., 2017). There are several designations in different parts of the world (Fletcher et al., 2015): Low Impact Development (LID); Water Sensitive Urban Design (WSUD); Integrated Urban Water Management (IUWM); Sustainable Urban Drainage Solutions (SUDS); Best Management Practices (BMPs), among others. Hereafter, these solutions will be referred to as SUDS.

SUDS are expected to minimise the impacts of development on the quantity and quality of the runoff, and maximise amenity and biodiversity opportunities (Woods-Ballard et al., 2007; Charlesworth, 2010). They can be cost-effective, at least in terms of investment if not operation and maintenance, when compared with conventional systems (Revitt et al., 2008). The benefits of SUDS may be maximised if they are considered from the beginning of the development planning process and throughout – influencing site layout and design, and the use and characteristics of urban spaces (Woods-Ballard et al., 2015).

SUDS design should rely on the concept of ‘management train’, consisting of a sequence of SUDS techniques that aim to reduce flow rates and volumes and minimise pollution (Woods-Ballard et al., 2015; Graham, 2016). These techniques need to be gradually applied from prevention, source control, site control through to regional control. Prevention and source control should be considered before site or regional controls (Woods-Ballard et al., 2007).

As examples of source control techniques, there are green roofs, permeable surfaces, bio retention areas, filter strips and local infiltration. The most common site control techniques are detention and retention basins, swales and urban ponds, and at a regional scale, retention ponds and wetlands are some of the options (Graham, 2016).

Nevertheless, it is important to bear in mind that these systems may have some drawbacks, such as in the case of extreme events. Under such events, the water volume reduction may be very limited and sensitive to the size and duration of rainfall event, type of soil and texture (Zhou, 2014). Due to this, some authors suggest that it would be more advantageous to promote the integration of SUDS and traditional drainage solutions to enhance their synergy (Zhou, 2014), before making radical shifts in the urban drainage panorama.

Performance indicator systems

Performance indicators (PI) are a fundamental component of any performance assessment system, whose final goal is to provide information on systems' behaviour to support their management. The ISO 24500 standards (ISO, 2007a, 2007b, 2007c) propose a methodology to develop performance assessment systems for water and wastewater services, in which PI definition constitutes one of the main steps. Regardless of the focus area within the water sector, the proposed methodology comprises: (1) identification of physical, management and/or service components; (2) definition of objectives; (3) definition of criteria; (4) definition of PI; and (5) assessment of performance versus objectives.

PI have been used in a wide range of organisations and in different areas, both in private and public sectors, such as: industry, transportation systems, construction, banking industry, education, health, public administration, financial regulation, water supply and wastewater drainage services, among others. According to Neely et al. (1995), PI can be defined as metrics used to quantify the efficiency and/or the effectiveness of an activity. They are a tool that can help to understand, manage and improve what organisations do (PBM SIG, 1995), by means of analytical techniques which are important for eliminating subjectivity in the interpretation of system performance (Vilanova et al., 2015). PI are not only meant to describe some existing or desired condition, but rather to inform whether a strategy of prescribed actions achieve their intended effect – i.e., they meet the desired outcomes (Fekete & Stakhiv, 2014).

PI are composed of a number expressed in specific units, and a confidence grade which indicates the quality of the data represented by the indicator. In general, PI are expressed as ratios between variables; these may be commensurate (e.g., %) or non-commensurate (e.g., €/m3) (Sjovold et al., 2008; Alegre & Coelho, 2012). In the case of non-commensurate ratios, the denominator should represent one dimension of the system (e.g., total water pipe length; annual costs), that allows the comparison through time or between systems of different sizes (ISO, 2007a, 2007b, 2007c).

The selection of PI should take into account their relevance, analytical conditions (technical and scientific), measurability (data availability and costs), data quality and comparability (Hamilton, 1996; Vilanova et al., 2015). PI should be clearly defined, easy and economical to measure and verify, understandable, and relevant to a specific water utility. The overall framework of the performance assessment should be simple, well defined, comprehensive (covering all components of a water utility), and comparable with similar utilities at regional, national and international levels (Haider et al., 2014).

Data quality is an important aspect that should be considered in a PI system. The use of poor quality data may result in poor outcomes and, consequently, lead to poor decision-making (e.g., wrong decisions) (Masayna et al., 2007). Data quality may be defined as the extent to which information remains reliable and consistent across the organisation (PBM SIG, 2001).

The confidence grade of a PI may be assessed in terms of accuracy and reliability. Accuracy is related to the measurement errors in the acquisition of input data and reliability expresses the uncertainties on how reliable a source of data may be (ISO, 2007a, 2007b, 2007c). Since it is not easy to obtain such information, the IWA manuals propose a system of broad bands, which are a variation of the Office of Water Services of England and Wales (OfWAT) (Matos et al., 2003; Alegre et al., 2016). These IWA PI systems include four bands of data accuracy and three of data reliability (Table 1).

Table 1.

IWA criteria to classify data quality (adapted from Alegre et al., 2016).

Recommended accuracy bands 
0–5% Better than or equal to +/− 5% 
5–20% Worse than ±5%, but better than or equal to +/− 20% 
20–50% Worse than ±20%, but better than or equal to +/− 50% 
>50% Worse than ±50% 
Recommend data source reliability bands 
★ ★ ★ Highly reliable data source 
★ ★ Fairly reliable data source: worse than ★ ★ ★, but better than ★ 
★ Unreliable data source 
Recommended accuracy bands 
0–5% Better than or equal to +/− 5% 
5–20% Worse than ±5%, but better than or equal to +/− 20% 
20–50% Worse than ±20%, but better than or equal to +/− 50% 
>50% Worse than ±50% 
Recommend data source reliability bands 
★ ★ ★ Highly reliable data source 
★ ★ Fairly reliable data source: worse than ★ ★ ★, but better than ★ 
★ Unreliable data source 

To classify performance, it is necessary to compare PI values against reference values. This may be undertaken by defining performance functions for each PI, which establish a relation between the PI values and a scale of classification (Cardoso, 2007). In such functions, limits for good, satisfactory or poor performance can be defined, to name one example. Those limits may be established based on legislation requirements, literature references, historical data, or other water utilities' data.

Along with the use of PI, it is also essential to provide context information and explanatory factors (Matos et al., 2003; Alegre et al., 2016). Context information provides information on the inherent characteristics of a utility and accounts for differences among systems. This information may be of two types: information describing context and external factors that are not under the control of the utility (e.g., demographics, topography, climate); and characteristics that can only be influenced by management decisions in the long term (e.g., age of the infrastructures). It is particularly helpful when comparing indicators from different systems. An explanatory factor is any element of the system of PI that can be used to explain PI values (i.e., the level of performance at the analysis stage). This includes PI, variables, context information and other data elements not playing an active role before analysis stage. Explanatory factors may be presented as qualitative information (aggregated information, e.g., poor rural area, demographics) or as quantitative information (e.g., rainfall).

PI may be used to calculate performance indices or performance levels, which can be useful in the graphical representation of a set of PI (Alegre & Coelho, 2012). Indices and levels are other categories of performance metrics that may be used in the context of performance assessment (Sjovold et al., 2008; Alegre & Coelho, 2012). Performance indices are standardised and commensurable metrics and may result from the combination of more disaggregated performance metrics (e.g., weighted average of performance indicators) or from analysis tools (e.g., simulation models, statistical tools, cost efficiency methods). They can be used to assess future scenarios (e.g., using simulation results or statistical analysis). In opposition to PI, they contain a judgement (e.g., 0 – no function; 1 – minimum acceptable; 2 – good; 3 – excellent). Performance levels are presented in discrete categories since they are performance metrics of a qualitative nature. Typically, they are used when it is not suitable to apply quantitative metrics.

When applying PI it is fundamental to understand that performance assessment based on PI is always a partial and simplified view of the reality. If performance assessment is used out of context, it may lead to wrong interpretations and implemented measures. The trend in time of a PI can be more important and give a sustainable idea about the improvement, the stabilisation or the degradation of the performance, than the value itself from each indicator (Baptista & Alegre, 2009). Measuring should not be an objective in itself, but just the start of a process to achieve better performance (Cabrera et al., 2011).

Review of performance indicators for storm water systems

Performance assessment of storm water systems is still at an early stage. There are few works specifically developed for this purpose. In this regard, this review gathers examples of performance metrics that were developed by different authors and organisations, whose focus was to assess storm water systems' performance. The examples are organised in four groups of PI and other metrics applicable to: (1) conventional systems (combined and separate pipe systems); (2) sustainable urban drainage systems (SUDS); (3) both conventional systems and SUDS; and (4) wastewater systems. PI for SUDS are included since they represent a growing or complementary option within urban storm water management. PI for wastewater systems are also included due to their potential to be adapted for conventional storm water systems, given the similarities between the systems.

Works are presented that mention the use of performance metrics as a means to assess the systems' performance, despite the reviewed works using different terminologies to refer to performance metrics. Some refer to performance indicators, some just use indicators and others refer to performance measures. The performance metrics definition differs among those selected works and, in many cases, it does not follow the definition presented in ISO 24500 standards and Alegre et al. (2016). Given the wide range of applications, the review is focused on performance metrics, namely, PI, that are only related to systems performance (e.g., hydraulic, environmental, ecological, social and economic categories). It is beyond the scope of this review to assess other aspects of PI related to the management activity of water utilities (e.g., management of human resources, financial performance, level of response to users' complaints).

Performance indices and indicators for conventional systems

Having observed a lack of research on PI for storm water pipeline systems among US utilities, Bhimanadhuni (2015) developed a performance index based on a list of parameters that affect separate storm water pipes to aid utilities in their repair, rehabilitation and replacement decisions. This index was obtained by applying the weighted factor method. This method consists of evaluating all relevant attributes of the problem and allocating a score to each attribute based on a rating scheme. In this case, the attributes were the potential failure modules that may affect the performance of a storm water pipe: capacity, blockage, load, surface wear and structure. For each module, the essential performance parameters that affect it are indicated in Figure 1. Each performance parameter was rated on a 1–5 scale, where 1 corresponds to ‘excellent’ and 5 to ‘very poor’. The performance parameter scores were then combined into an overall performance index, based on a 1–5 scale, where 5 meant immediate attention required and 1 meant minor defects.

Fig. 1.

The hierarchal structure of a storm water pipe performance (adapted from Bhimanadhuni, 2015).

Fig. 1.

The hierarchal structure of a storm water pipe performance (adapted from Bhimanadhuni, 2015).

The proposed index focused on several important aspects of pipe performance, namely, operational and structural performance, that are less considered in most works. Nevertheless, the use of the performance index as it was developed may present disadvantages. On one hand, it relied on parameters that were composed of single variables, not enabling the comparison over time and not giving a reference in terms of systems dimension, which would subsequently impede the comparison between systems from different contexts. On the other hand, some parameters were characteristics of the pipe that were not expected to vary over time, as was the case with pipe length, diameter, material and shape, for example. They could in turn be used as explanatory factors and not as part of a performance index.

The effect of climate change on conventional storm water systems is one relevant motivation to assess their performance assessment. Several studies can be found on this topic, such as the work of Berggren (2008). Based on a literature review, Berggren (2008) investigated existing indicators that could be used to describe and compare the impacts of climate change on urban drainage systems in each rain event. The indicators were classified in terms of description of system performance (A), capacity exceeding (B), and description of consequences because of capacity exceeding (C) (Table 2). The author stated that it would be necessary to find more indicators and evaluate them in case studies, with respect to their sensitivity and to determine how well they could describe impacts.

Table 2.

Examples of indicators found in the literature by Berggren (2008).

Type of system Indicator Classification References 
Combined system CSO volume (m3Niemczynowicz (1989); Semanedi-Davies et al. (2005)  
CSO frequency (-) – 
Pumping station overflow (m3Semanedi-Davies et al. (2005)  
Inflow to sewer system (m3Niemczynowicz (1989)  
Inflow to WWTP (m3A (B) Semanedi-Davies et al. (2005)  
Number of properties affected (-) Ashley et al. (2005)  
Economic loss due to damage (€) Ashley et al. (2005)  
Separated system Total flow volume (m3Semanedi-Davies et al. (2005)  
Total runoff volume (m3Waters et al. (2003)  
Time to peak discharge (time) B (A) Waters et al. (2003)  
Volume of peak discharge (m3B (A) Waters et al. (2003)  
Number of pipes surcharged (-) Waters et al. (2003)  
Frequency of flood (-) – 
Duration of flood (h) – 
Pipe flow ratio (-) – 
Type of system Indicator Classification References 
Combined system CSO volume (m3Niemczynowicz (1989); Semanedi-Davies et al. (2005)  
CSO frequency (-) – 
Pumping station overflow (m3Semanedi-Davies et al. (2005)  
Inflow to sewer system (m3Niemczynowicz (1989)  
Inflow to WWTP (m3A (B) Semanedi-Davies et al. (2005)  
Number of properties affected (-) Ashley et al. (2005)  
Economic loss due to damage (€) Ashley et al. (2005)  
Separated system Total flow volume (m3Semanedi-Davies et al. (2005)  
Total runoff volume (m3Waters et al. (2003)  
Time to peak discharge (time) B (A) Waters et al. (2003)  
Volume of peak discharge (m3B (A) Waters et al. (2003)  
Number of pipes surcharged (-) Waters et al. (2003)  
Frequency of flood (-) – 
Duration of flood (h) – 
Pipe flow ratio (-) – 

CSO: combined sewer overflow; WWTP: wastewater treatment plant.

The information presented by Berggren (2008) was mainly focused on hydrological and hydraulic aspects of storm water pipe systems' performance and considered both system components and catchment scales. Most of the proposed single variables (e.g., volumes), inaccurately referred to as indicators, were unsuitable to be used as PI according to the ISO 24500 standards and Alegre et al. (2016). It would hinder the possibility to assess and compare different components of a storm water pipe system or different systems over time.

Nie et al. (2009) carried out simulations to study the possible consequences of three different climate scenarios (present, predicted and artificial climate scenarios) in sewer systems. The simulations were run for annual-, seasonal-, and daily-based events. The software applied in the simulations was the MOUSE programme developed by the DHI (Danish Hydraulic Institute). The selected case study was a catchment located in Fredrikstad, Norway, served by separate and combined systems. Six indicators were selected to describe the consequences, as follows: number of flooding manholes (-); total spilling water (1,000 m3); number of surcharging sewer nodes (-); total length of surcharging sewers (km); number of buildings at risk of flooding (-); total CSO volume (1,000 m3).

The above-mentioned variables, unjustifiably referred to as indicators, proposed by Nie et al. (2009) were also more related to the hydraulic performance of storm water pipe systems, besides accounting for the consequences at a social level. Those single variables were selected considering a specific case study area. For their adaptation in performance assessment programmes, it would be required to transform them to fulfil the PI requirements, following the international standards, i.e., integrate variables in each PI that represent systems' dimensions, enabling the comparison between systems of different sizes and over time.

For a combined sewer system, Kleidorfer et al. (2014) performed a sensitivity analysis to compare the impact of increased rainfall intensities (climate change impact) and the pavement of urban areas (urbanisation and land-use change). The software used was SWMM 5.0 and the case study was a catchment located in Innsbruck, Austria. The system performance was assessed using four indicators: ponded volume (flooding) leaving the system (Vflood in m3); combined sewer overflow (CSO) (VCSO in m3); discharge of total suspended solids load (LTSS in ton); and discharge of ammonium nitrogen load (LNH4-N in ton). The choice of the indicators was justified by the fact that they can be used to express system-wide performance, in contrast to values which must be calculated for each pipe, as, for example, flow peaks.

One advantage of the work of Kleidorfer et al. (2014) was the inclusion of an environmental performance assessment, which is less considered regarding storm water pipe systems. Despite this, single variables were also selected to be inadequately used as indicators, as described for the previous reviewed works.

Performance indicators for sustainable urban drainage systems

Regarding the monitoring of performance of SUDS, based on a literature review, Cherqui et al. (2013) proposed several PI related to hydraulic performance, hydrologic performance, treatment performance, economical aspects, lifespan and long-term effectiveness, and social aspects (Table 3). The authors emphasised that performance assessment of such devices should not be limited to pollution and hydrology. They stated that assessment of the delivered service should also be considered, having suggested how to carry out a survey and methods for data acquisition.

Table 3.

Proposed performance indicators for practitioners of SUDS (adapted from Cherqui et al., 2013).

Performance domains Performance indicators 
Hydraulic performance (At the site and catchment scales) Flow attenuation at the outlet 
Volume reduction at the outlet 
Lag-time 
Overflow frequency indicator 
Drainage duration frequency 
Hydrologic performance At the site scale Reduction in mean annual runoff volume back to natural volume 
Runoff frequency 
Similarity between the pre-developed volume of base flow and the volume of storm water released as filtered flows 
Reduction of days in which filtered flow exceeds ‘pre-developed base flow’ 
At the catchment scale Site-scale water fluxes (volume of inflow, outflow, infiltrated volume and evapotranspired or extracted volume through harvesting) 
Site-scale hydrologic indicators (frequency of runoff, flow duration curve) 
Optional information on the catchment-scale outcomes in terms of relevant flow indicators 
Treatment performance Pollutant concentration attenuation (event mean concentration) 
Event-based pollutant removal (mass) 
Pollution retention performance 
Depth of polluted soil 
Contamination indicator (soil) 
Economic aspects Preliminary costs 
Construction costs 
Operational costs 
Savings/return on investment 
Lifespan and long-term effectiveness Long-term functionalities 
Monitoring and maintenance check-list 
Social aspects Number of complaints 
Kind of odours smelled 
Percentage of satisfaction 
Number and gravity of personnel accidents 
Level of security for the staff or the public 
Performance domains Performance indicators 
Hydraulic performance (At the site and catchment scales) Flow attenuation at the outlet 
Volume reduction at the outlet 
Lag-time 
Overflow frequency indicator 
Drainage duration frequency 
Hydrologic performance At the site scale Reduction in mean annual runoff volume back to natural volume 
Runoff frequency 
Similarity between the pre-developed volume of base flow and the volume of storm water released as filtered flows 
Reduction of days in which filtered flow exceeds ‘pre-developed base flow’ 
At the catchment scale Site-scale water fluxes (volume of inflow, outflow, infiltrated volume and evapotranspired or extracted volume through harvesting) 
Site-scale hydrologic indicators (frequency of runoff, flow duration curve) 
Optional information on the catchment-scale outcomes in terms of relevant flow indicators 
Treatment performance Pollutant concentration attenuation (event mean concentration) 
Event-based pollutant removal (mass) 
Pollution retention performance 
Depth of polluted soil 
Contamination indicator (soil) 
Economic aspects Preliminary costs 
Construction costs 
Operational costs 
Savings/return on investment 
Lifespan and long-term effectiveness Long-term functionalities 
Monitoring and maintenance check-list 
Social aspects Number of complaints 
Kind of odours smelled 
Percentage of satisfaction 
Number and gravity of personnel accidents 
Level of security for the staff or the public 

The proposed set of PI may constitute a sound basis to develop a performance assessment system for different types of SUDS, since it covered different performance categories and scales of application. However, there are single variables (e.g., costs, number of complaints) named as PI, that need to be redesigned to fulfil the international standards requirements.

Another example of a monitoring programme based on performance assessment is the work of Jefferies (2004). The report gathered the results of a five-year monitoring programme and assessment of SUDS performance in Scotland. Two categories of SUDS were monitored: source control and site/regional control systems. Regarding performance assessment, more relevance was given to the study of hydrological and water quality parameters. The selected hydrological parameters were: initial runoff loss (mm), lag time (min), runoff production (%), reduction of peak flow (%), benefit factor (%) and events retained (%) (Jefferies, 2004). The selected water quality parameters were: total suspended solids (mg/L), biochemical oxygen demand (mg/L), copper (μg/L), zinc (μg/L), nickel (μg/L), and hydrocarbons (mg/L).

Most of the parameters selected would be suitable to be part of a PI system to assess SUDS performance. Nevertheless, the parameters' initial runoff loss and lag time need to be adapted to integrate the area dimension, for example, to enable comparison over time and between systems.

Also noteworthy is the fact that reduction of peak flow was considered a hydrological parameter by Jefferies (2004), while in the work of Cherqui et al. (2013), a similar PI (flow attenuation at the outlet) was considered a hydraulic one. There were different perspectives on the classification of PI by authors.

Other authors developed PI concerning only the monitoring of infiltration systems. Considering the uncertainty associated with the long-term sustainability of these systems, Moura et al. (2010) developed a decision support system, based on a multi-criteria method, to help evaluate the performance of an existing infiltration system at different stages of its lifespan. The case study was carried out in an urban industrial area, in Lyon (France). The evaluated system was composed of two compartments: one retention basin and one infiltration basin. A set of PI was built integrating technical, economic, environmental, social and sanitary aspects. Some examples of PI were as follows: flooding frequency (-); global hydraulic performance measuring the potential for clogging ([m/s] – source control systems; (-) – mixed areas); groundwater monitoring (dissolved oxygen concentration (mg/L) and specific conductance (μS/cm)); pre-treatment efficiency with regard to trapped total suspended solids (-); depth where pollution becomes low or nil (m); highly polluted points (%); number of dysfunctions observed in situ (-); percentage of mass of sediments cleaned out of systems that can be reused (%); difference between maintenance cost in one year and the average maintenance cost in previous years (€); and number of complaints from the local residents concerning the system (-).

The work of Moura et al. (2010) highlights the proposed PI for groundwater quality, that in many cases receive less attention under performance assessment programmes. PI related to soil quality and the fate of cleaned solids are interesting aspects to take into consideration. Again, many PI were incorrectly developed in terms of variables and they also need to be redesigned to reflect infiltration systems of variable sizes and at different time stages.

Dechesne et al. (2004) proposed a framework for evaluating the hydraulic and pollution retention performance of infiltration basins, in the long term. The indicators were tested on five infiltration basins in a suburban area in Lyon (France). The proposal included context indicators (clogging and contamination indicators), which evaluate the system state at a given time, and PI, that quantify the achievement of an expected system performance. The PI were developed to assess the following aspects: drainage duration (%), overflow frequency (-), predictive life period (years), particle filtration (%) and cumulative pollution trapping (%).

This work proposed PI only focused on the hydraulic and pollution retention performance of infiltration systems. The PI of pollution retention was only concerned with solids and the overflow frequency PI does not allow its trend over time to be known. It gives relevance to the need to provide explanatory information through clogging and contamination indicators, which were not included in most of the reviewed works, even though they were erroneously defined as context indicators.

Performance indicators for both conventional systems and SUDS

Matzinger et al. (2014) developed PI to support the definition of a storm water management strategy. The authors recommended, as a first step in establishing a storm water management strategy, the assessment of local problems and related objectives. Next, based on the defined indicators, it was supposed to pre-select measures depending on their ability to achieve the objectives. The PI were grouped into eight different categories (building physics and services, landscape quality, urban climate, biodiversity, groundwater, surface water, direct costs and resource use) (Table 4), which were related to several goals dependent on the local situation.

Table 4.

Goals and quantitative performance indicators for storm water measures (adapted from Matzinger et al., 2014).

Effect category Goals of storm water management Performance indicators 
Building physics and services Benefits from storm water use for inhabitants of buildings Rainwater recycling and reuse for service water [%] 
Energy saving from cooling (or heating) [kWh m−2
Landscape quality Structural richness Variance in micro relief [m] 
Variance in plant height [m] 
Dry plant volume [kg m−2
Proportion of open water surfaces [%] 
Usability Scale from 0 to 3: the higher, the better 
Urban climate Increase in thermal comfort Evaporation rate [%] 
Green volume per connected impervious area [m3 m−2
Surface albedo [-] 
Biodiversity Species biodiversity α- and β-diversity (floristic and faunistic) [number of species] 
Habitat diversity Structural diversity of vegetation [number of different elements] 
Habitat connectivity Average distance between measures [m] and species dispersal among measures [%] 
Rare species occurrence Proportion of rare species [%] 
Groundwater Local groundwater level target Change in groundwater recharge rate [mm] 
Avoid deterioration of groundwater quality Change in electric conductivity, chloride, sulphate, zinc and biocide concentrations [%] 
Surface water Combined sewer systems  
Reduction of CSO impacts (hydraulic, oxygen and ammonia stress) Reduction of one-year peak runoff rate [%] (compared to situation without the measure) 
Separate sewer systems  
Reduction of hydraulic stress Reduction of one-year peak runoff rate [%] (compared to situation without the measure) 
Reduction of eutrophication Reduction in P and N loading [%] (depending on total annual flow reduction and/or cleaning effect) 
Reduction of sediment contamination Reduction in TSS loading [%] (depending on total annual flow reduction and/or cleaning effect) 
Flooding (both separate and combined sewer systems)  
Support flood prevention Reduction of 30-year peak runoff rate [%] (compared to situation without the measure) 
Direct costs Minimise costs Present value of annual costs per connected impervious area [€ year−1 m−2
Resource use Minimise indirect environmental impacts Cumulative energy demand of non-renewable fuels [MJ m−2
Abiotic resource depletion of minerals [kg Fe-eq m−2
Global warming potential per connected impervious area [kg CO2- eq m−2
Effect category Goals of storm water management Performance indicators 
Building physics and services Benefits from storm water use for inhabitants of buildings Rainwater recycling and reuse for service water [%] 
Energy saving from cooling (or heating) [kWh m−2
Landscape quality Structural richness Variance in micro relief [m] 
Variance in plant height [m] 
Dry plant volume [kg m−2
Proportion of open water surfaces [%] 
Usability Scale from 0 to 3: the higher, the better 
Urban climate Increase in thermal comfort Evaporation rate [%] 
Green volume per connected impervious area [m3 m−2
Surface albedo [-] 
Biodiversity Species biodiversity α- and β-diversity (floristic and faunistic) [number of species] 
Habitat diversity Structural diversity of vegetation [number of different elements] 
Habitat connectivity Average distance between measures [m] and species dispersal among measures [%] 
Rare species occurrence Proportion of rare species [%] 
Groundwater Local groundwater level target Change in groundwater recharge rate [mm] 
Avoid deterioration of groundwater quality Change in electric conductivity, chloride, sulphate, zinc and biocide concentrations [%] 
Surface water Combined sewer systems  
Reduction of CSO impacts (hydraulic, oxygen and ammonia stress) Reduction of one-year peak runoff rate [%] (compared to situation without the measure) 
Separate sewer systems  
Reduction of hydraulic stress Reduction of one-year peak runoff rate [%] (compared to situation without the measure) 
Reduction of eutrophication Reduction in P and N loading [%] (depending on total annual flow reduction and/or cleaning effect) 
Reduction of sediment contamination Reduction in TSS loading [%] (depending on total annual flow reduction and/or cleaning effect) 
Flooding (both separate and combined sewer systems)  
Support flood prevention Reduction of 30-year peak runoff rate [%] (compared to situation without the measure) 
Direct costs Minimise costs Present value of annual costs per connected impervious area [€ year−1 m−2
Resource use Minimise indirect environmental impacts Cumulative energy demand of non-renewable fuels [MJ m−2
Abiotic resource depletion of minerals [kg Fe-eq m−2
Global warming potential per connected impervious area [kg CO2- eq m−2

The proposed PI by Matzinger et al. (2014) are transversal to different types of storm water systems and include performance domains that were not considered in other reviewed works, such as in the cases of landscape quality, biodiversity and urban climate. In this way, it helps widen the scope of performance assessment, highlighting the interactions and other functions that storm water systems may have, apart from the hydraulic and pollution concerns. Relating objectives and PI is another positive aspect of this work since this is a relevant step in performance assessment implementation, in accordance with the international recommendations.

Sundberg et al. (2004) proposed a framework for the assessment of sustainable storm water systems to be used in planning and in comparisons between different options. The indicators were formulated based on the Swedish context. The developed indicators were organised according to six criteria: existence, effectiveness, freedom of action, security, adaptability and co-existence (Table 5). The authors proposed indicators regarding the internal storm water system and the interaction between the storm water system and its different related systems. The framework relied on the perspective that sustainability could not be measured on one quantitative scale and that it was not sustainable if it did not consider a system's surroundings.

Table 5.

Indicators for sustainable storm water systems to be used for comparative assessments (adapted from Sundberg et al., 2004).

Basic system criteria Core system Related systems
 
Storm water system Urban water system Society Eco-system 
Existence Renewal rate/Degradation rate Amounts of storm water to the urban water system Number of pathogens discharged from the storm water system; Amount of flooding in urban areas; Changes in ground water levels Discharge of persistent organic pollutants and heavy metals to soil and water 
Effectiveness Investment costs; Energy/exergy use Costs of storm water systems/cost of total system Cost per capita – 
Freedom of action Amounts of source separated storm water Ratio of separate sewers/combined sewers; Contribution to ground water recharge; Potential for water reuse activities Contribution to a flexible future development Percentage of unpolluted storm water released 
Security Unscheduled maintenance Hydraulic overload in WWTP, number of days/year Number of flooded basements; Estimated damage of 10-year flood; Risk of accidents Peak toxicity in receiving waters over 1 year 
Adaptability Sensitivity to changed population density; Flexibility in management structures Contribution to recovery of sludge Ability to manage irregular precipitation; Ratio impervious area/total area Change of flows in natural waters 
Co-existence – Number and size of combined sewer overflows Occurrence of odour, bugs and other nuisances Contribution to biodiversity; Amounts of suspended solids, BOD, P, N, discharged to water. 
Basic system criteria Core system Related systems
 
Storm water system Urban water system Society Eco-system 
Existence Renewal rate/Degradation rate Amounts of storm water to the urban water system Number of pathogens discharged from the storm water system; Amount of flooding in urban areas; Changes in ground water levels Discharge of persistent organic pollutants and heavy metals to soil and water 
Effectiveness Investment costs; Energy/exergy use Costs of storm water systems/cost of total system Cost per capita – 
Freedom of action Amounts of source separated storm water Ratio of separate sewers/combined sewers; Contribution to ground water recharge; Potential for water reuse activities Contribution to a flexible future development Percentage of unpolluted storm water released 
Security Unscheduled maintenance Hydraulic overload in WWTP, number of days/year Number of flooded basements; Estimated damage of 10-year flood; Risk of accidents Peak toxicity in receiving waters over 1 year 
Adaptability Sensitivity to changed population density; Flexibility in management structures Contribution to recovery of sludge Ability to manage irregular precipitation; Ratio impervious area/total area Change of flows in natural waters 
Co-existence – Number and size of combined sewer overflows Occurrence of odour, bugs and other nuisances Contribution to biodiversity; Amounts of suspended solids, BOD, P, N, discharged to water. 

BOD: biochemical oxygen demand; P: phosphorus; N: nitrogen.

As in the previous work of Matzinger et al. (2014), a framework was proposed for assessment that gave a new perspective on what might be relevant and considered assessing storm water systems and their interactions with other systems. Different categories were included, and indicators were meant to assess performance at a catchment scale. Most of the information, inaccurately referred to as indicators, also needed to be redesigned, to integrate variables that would give the opportunity to assess and compare systems of different sizes and contexts and to know their evolution over time.

Performance assessment of storm water systems can also be found as part of the management plans of some cities or regions. One example is the Asset Management Plan 2015–2045 of Auckland city, in New Zealand (Auckland Council, 2015). In this plan, all aspects of storm water management across the Auckland region were included. The established priority areas were: asset operation/renewals; supporting and servicing Auckland's growth strategy demonstrating innovation and best practice; flooding control; and environmental improvement. To ensure the satisfaction of the community's needs, levels of service associated with performance measures and target values for different time periods were defined. The measurement and the reporting of service performance is carried out on an annual basis. Next, some of the performance measures related to the system performance assessment are presented:

  • the number of flooding events and the associated number of habitable floors affected per 1,000 properties (-);

  • the proportion of habitable floors that are predicted to flood in a ten-year event (%);

  • the proportion of habitable floors that are predicted to flood in a 100-year event (%);

  • number of blockages in the storm water network per 100 km (No./100 km);

  • Auckland Council storm water compliance with resource consents for discharge from its storm water system, measured by the number of: (a) abatement notices; (b) infringement notices; (c) enforcement orders; (d) successful prosecutions received in relation to consents;

  • the ratio of the length of watercourse consented to be physically improved versus physically degraded in each year (-).

Another example from New Zealand is the Wellington region. The organisation Wellington Water, which is a council-controlled organisation, is responsible for the management of three water networks and infrastructure (drinking water, wastewater and storm water). In the Statement of Intent 2016–2019 (Wellington Water, 2015), three levels of customer outcomes were presented (safe and healthy water, respectful of the environment, resilient networks support the economy) supported by 12 service goals. Each outcome and service goal had a set of PI that were used to assess the services' delivery to customers. The performance review is also undertaken on an annual basis. The PI related to the storm water system were as follows:

  • the number of non-consented overflows from the treatment plants (-);

  • the number of consented overflows from the treatment plants (-);

  • compliance with resource consents for discharge from its storm water system (full compliance is no notices/convictions): (a) abatement notices; (b) infringement notices; (c) enforcement orders; (d) convictions;

  • number of pipeline blockages per km of pipeline (No./100 km);

  • number of habitable floors affected per 1,000 storm water connections (No./1,000 connections).

From Australia, the case of Mitchell Shire Council is highlighted. The council developed an Infrastructure Asset Management (IAM) Plan, covering the urban storm water drainage system (Mitchell Shire Council, 2012). The plan included the definition of levels of service, which support the organisation's strategic goals and are based on customer expectations and statutory requirements. The levels of service were split into two categories: community and technical levels of service. Associated with each category there were key PI, levels of service, performance measures, targets, current performance actions to meet performance target and resources required. In Table 6, some of the developed key PI, levels of service and performance measures, are presented.

Table 6.

Some of the key PI, levels of service and performance measures, defined in the IAM Plan of Mitchell Shire Council (adapted from Mitchell Shire Council, 2012).

Key PI Level of service Performance measure 
Community levels of service 
 Function The drainage system meets user requirements for removal of storm water in accordance with design standards (a) Total number of flooding incidents arising from blockages; (b) Number of incidents affecting an individual property 
 Accessibility All weather access to properties free from blockage by storm water flooding Duration and frequency of access being impassable 
 Health and safety System is safe and hazard free Absence of significant health and safety hazards 
 Environmental standards Quality of discharge waters Number of contaminated discharges likely to have a negative impact on waterways 
Technical levels of service 
 System capacity Protection of property from surcharges caused by drains that have capacity below current design standards Frequency of drain surcharge flooding in: (a) Arterial roads; (b) Local streets, private yards, active recreation areas 
 Cost-effectiveness Provide and manage the drainage system in cost-effective manner Unit maintenance cost for key components 
Key PI Level of service Performance measure 
Community levels of service 
 Function The drainage system meets user requirements for removal of storm water in accordance with design standards (a) Total number of flooding incidents arising from blockages; (b) Number of incidents affecting an individual property 
 Accessibility All weather access to properties free from blockage by storm water flooding Duration and frequency of access being impassable 
 Health and safety System is safe and hazard free Absence of significant health and safety hazards 
 Environmental standards Quality of discharge waters Number of contaminated discharges likely to have a negative impact on waterways 
Technical levels of service 
 System capacity Protection of property from surcharges caused by drains that have capacity below current design standards Frequency of drain surcharge flooding in: (a) Arterial roads; (b) Local streets, private yards, active recreation areas 
 Cost-effectiveness Provide and manage the drainage system in cost-effective manner Unit maintenance cost for key components 

The examples of proposed PI in Auckland, Wellington region and Mitchell Shire Council plans were mainly focused on the flooding frequency and on its impacts in terms of public health and safety. Some of the PI were related to the impacts of storm water discharge into the receiving waters, and there was an example related to the maintenance cost of systems' components. Most of the proposed PI were meant to assess performance at a watershed scale and were more related to pipe systems. As in other reviewed works, many variables were inadequately proposed as PI, and they need to be redesigned to facilitate the comparison between systems' components and different systems, besides their evolution over time.

Other examples of performance indicators in the wastewater sector

As mentioned before, conventional storm water systems are still widely present in urban areas. As such, it is also relevant to collect PI that were developed for wastewater systems by institutional organisations and academic works (Table 7), and that may be adapted to assess storm water pipe systems' performance, due to the similar characteristics between the two systems.

Table 7.

PI developed for wastewater systems with potential application to storm water piped systems.

Author Objectives Performance indicators References 
Institutional organisation 
International Water Association (IWA) The developed PI system aims to provide objective and comprehensive management tools for utilities and other stakeholders involved in any aspect of wastewater provision Intermittent overflow discharge related to rainfall (%/year); Surcharging in gravity sewers in wet weather (%); High sewer surcharging (%); Inflow/Infiltration/Exfiltration (%); Sewer blockages (No./100 km sewer/year); Flooding from combined sewers (No./100 km sewer/year); Surface flooding (No./100 km sewer/year); Sewer collapses (No./100 km sewer/year); CSO control (%); Investments for new assets and reinforcement of existing assets (%); Investments for asset replacement and renovation (%) Matos et al. (2003)  
International Benchmarking Network for Water and Sanitation Utilities (IBNET) Participating in IBNET enables water and wastewater utilities from 135 countries to analyse their strengths and weaknesses in relation to those of peer organisations and track their performance over time Pipe breaks (No. breaks/km/year); Sewer system blockages (No. blockages/km/year) Danilenko et al. (2014)  
European Benchmarking Co-operation (EBC) Help European water and wastewater utilities from 20 countries to improve their services by applying an international benchmarking programme Sewer and connection blockages (No./100 km sewer); Flooding from combined sewers (No./100 km sewer); Sewer rehabilitation (%/year) EBC (2016)  
Water and Waste Services Regulatory Authority (ERSAR) Regulation of the quality of service provided by Portuguese operators by assessing the service provided to end-users and comparing operators to each other through the application of an indicators system Flooding occurrences (No./100 km sewer/year or No./1,000 connections/year); Occurrences of sewer structural collapses (No./100 km/year) LNEC & ERSAR (2013)  
Office of Water Services (OfWAT) Comparison of companies in terms of their performance in the water and sewerage sectors The total number of serious pollution incidents emanating from a discharge or escape of a contaminant from a sewerage company asset (No./10,000 km sewer/year) Ofwat (2013)  
American Water Works Association (AWWA) Part of its work is dedicated to benchmarking, based on PI, allowing utilities to track their own performance and to compare their results to peers to identify areas that can be improved Regulatory compliance – wastewater (%); Sewer overflow (No. events/100 miles of pipe); Collection system integrity (No. failures/100 miles of pipe) AWWA (2015)  
Bureau of Meteorology – Australia Comparison of the performance of 79 utilities and councils and 5 bulk water authorities that provide urban water services across Australia, in conjunction with State and Territory governments and the Water Services Association of Australia Sewer overflows reported to the environmental regulator (No./100 km of sewer main) Bureau of Meteorology (2018)  
Academic work 
Cardoso, M. A. Development of a methodology for a standardised, systematic, objective and flexible technical performance assessment of urban drainage systems Water level (m); Flow velocity (m/s); Infiltration (%); Inflow (%); Overflow volume (%); Overflow frequency (no./time); Discharge concentration of: BOD5, COD, TSS, TN, TP, faecal and total coliforms (mg/L); Exfiltration (%) Cardoso (2007)  
De Toffol, S. Determination of suitable indicators to assess sewer system performance, based on the simulation of synthetic or real catchments CSO efficiency (%); CSO volume (m3/year); Maximum overflow (m3/time); CSO events (No./year); Mean discharged pollutants load (kg/year); Critical oxygen deficit (mg/L) Toffol (2006)  
Author Objectives Performance indicators References 
Institutional organisation 
International Water Association (IWA) The developed PI system aims to provide objective and comprehensive management tools for utilities and other stakeholders involved in any aspect of wastewater provision Intermittent overflow discharge related to rainfall (%/year); Surcharging in gravity sewers in wet weather (%); High sewer surcharging (%); Inflow/Infiltration/Exfiltration (%); Sewer blockages (No./100 km sewer/year); Flooding from combined sewers (No./100 km sewer/year); Surface flooding (No./100 km sewer/year); Sewer collapses (No./100 km sewer/year); CSO control (%); Investments for new assets and reinforcement of existing assets (%); Investments for asset replacement and renovation (%) Matos et al. (2003)  
International Benchmarking Network for Water and Sanitation Utilities (IBNET) Participating in IBNET enables water and wastewater utilities from 135 countries to analyse their strengths and weaknesses in relation to those of peer organisations and track their performance over time Pipe breaks (No. breaks/km/year); Sewer system blockages (No. blockages/km/year) Danilenko et al. (2014)  
European Benchmarking Co-operation (EBC) Help European water and wastewater utilities from 20 countries to improve their services by applying an international benchmarking programme Sewer and connection blockages (No./100 km sewer); Flooding from combined sewers (No./100 km sewer); Sewer rehabilitation (%/year) EBC (2016)  
Water and Waste Services Regulatory Authority (ERSAR) Regulation of the quality of service provided by Portuguese operators by assessing the service provided to end-users and comparing operators to each other through the application of an indicators system Flooding occurrences (No./100 km sewer/year or No./1,000 connections/year); Occurrences of sewer structural collapses (No./100 km/year) LNEC & ERSAR (2013)  
Office of Water Services (OfWAT) Comparison of companies in terms of their performance in the water and sewerage sectors The total number of serious pollution incidents emanating from a discharge or escape of a contaminant from a sewerage company asset (No./10,000 km sewer/year) Ofwat (2013)  
American Water Works Association (AWWA) Part of its work is dedicated to benchmarking, based on PI, allowing utilities to track their own performance and to compare their results to peers to identify areas that can be improved Regulatory compliance – wastewater (%); Sewer overflow (No. events/100 miles of pipe); Collection system integrity (No. failures/100 miles of pipe) AWWA (2015)  
Bureau of Meteorology – Australia Comparison of the performance of 79 utilities and councils and 5 bulk water authorities that provide urban water services across Australia, in conjunction with State and Territory governments and the Water Services Association of Australia Sewer overflows reported to the environmental regulator (No./100 km of sewer main) Bureau of Meteorology (2018)  
Academic work 
Cardoso, M. A. Development of a methodology for a standardised, systematic, objective and flexible technical performance assessment of urban drainage systems Water level (m); Flow velocity (m/s); Infiltration (%); Inflow (%); Overflow volume (%); Overflow frequency (no./time); Discharge concentration of: BOD5, COD, TSS, TN, TP, faecal and total coliforms (mg/L); Exfiltration (%) Cardoso (2007)  
De Toffol, S. Determination of suitable indicators to assess sewer system performance, based on the simulation of synthetic or real catchments CSO efficiency (%); CSO volume (m3/year); Maximum overflow (m3/time); CSO events (No./year); Mean discharged pollutants load (kg/year); Critical oxygen deficit (mg/L) Toffol (2006)  

CSO: combined sewer overflow; BOD5: 5-day biochemical oxygen demand; COD: chemical oxygen demand; TSS: total suspended solids; TN: total nitrogen; TP: total phosphorus.

Discussion

By reviewing several research studies, the present article intends to significantly contribute to the state of the art in the performance assessment of storm water systems. The review of existing methodologies helps in defining the main areas of concern in the field, serving as a starting point to develop a framework to build a storm water performance assessment system. The review focused on categories that enable the assessment of systems performance, namely, hydraulic, hydrologic, environmental, economic and social.

Next, we identified strengths and weaknesses of the reviewed works that should be considered and improved. Also presented is a reflection on the possible reasons why performance assessment of storm water systems has not become a common practice yet.

The given examples of PI covered different types of storm water systems, from conventional to non-conventional systems. Those PI were related to several performance domains, and were an opportunity to compare what aspects would be more relevant to assess according to different authors.

In the storm water pipe systems' group, most of the PI were proposed to assess hydraulic performance, which is commonly in line with the main objectives and concerns of this type of system, such as flood control. The infrastructural category had less expression, although the infrastructural condition may have a great impact on the overall systems' performance. Only one work proposed a performance index in which was included variables related to the infrastructural domain.

Regarding the SUDS group, there was more diversity of performance categories, with the environmental performance being the most considered category. This may be because the main objectives toward SUDS are more environment-related ones, namely, in terms of pollution retention capacity. Hydraulic and hydrological performance were also recurrent categories for which PI were developed, followed by social and economic performance.

The group of both storm water pipe systems and SUDS included works that proposed performance assessment frameworks based on PI accounting for the ecological, climate and landscape impacts, and the interactions between storm water systems and other urban systems. This enhances the fact that storm water systems may deliver other benefits in urban areas, besides hydraulic and environmental ones.

The PI proposed in the management plans of cities and regions in New Zealand and Australia showed that the main priorities were flood control and public health and safety protection, as well as some environmental concerns. Nevertheless, since assessing storm water systems' performance is, to some extent, uncommon in many countries, it is worth mentioning this effort carried out in New Zealand and Australia. It reflects that storm water systems are considered an important and vital infrastructure in the urban water sector.

The review of wastewater PI was also included in the present paper due to similarities between systems. It was recognised that it could be advantageous to take into consideration the adaptation of PI designed for wastewater systems that integrate the storm water component. Many of those examples are well accepted and have been widely tested, such as the IWA indicators.

Overall, most of the proposed PI derived from technical performance categories, while the economic and social categories were less considered. This enhances the need to develop PI frameworks that enable the assessment of systems' performance integrating different performance dimensions – technical, economic and social.

Regarding the scale of application, different approaches were carried out. Some authors only focused on PI to assess systems at a local scale, such as in the case of SUDS devices; others conducted the assessment at a catchment level. Few integrated both scales of application, despite the fact that the latter approach may bring more benefits. Usually it is not enough to assess whether individual systems are meeting their performance requirements. It may also be essential to determine how different systems interact at a catchment scale. This would facilitate the detection of problems, the definition of priority areas needing intervention, and the potential for enhancing the synergy between systems and improving overall performance more effectively.

Another relevant aspect of the reviewed works was the existence of different interpretations of what is a PI, which was then reflected in the proposed methodologies. Some authors considered as PI what in fact are dimensional variables, such as volumes (m3) or masses (tonnes). In some cases, the reason may be that the work was focused on assessing the system performance per rain event; while in others, the performance assessment was applied to individual systems or single case studies. Using these PI compromises further comparison with other systems, which may not be the initial authors' intention. This emphasises the importance of following a well-defined and univocal terminology and knowing how to employ it properly.

It is also interesting to verify in most of the reviewed works that little emphasis was placed on the need to provide contextual information and/or explanatory factors and to classify data quality. Without this complementary information, it is difficult to understand the obtained results and which factors may explain them, and to make fair comparisons among different systems. Not knowing how accurate and reliable data are may compromise the entire assessment process. For example, decisions that are made based on such data may be revealed to be inadequate.

At this point, it is also important to reflect on why there has been such poor progress in carrying out performance assessment of storm water systems, despite some relevant development in the research field. Some of the reasons that may justify this are presented next.

Assessing if a system is meeting its performance requirements (e.g., hydraulic, infrastructural, environmental, economic and social) is a highly demanding process. It has several requirements, such as collection and disaggregation of information, implementation of new technology for information collection, organisational restructuring, among other aspects (Pinto et al., 2017). The costs associated with data collection, storage and retrieval are the most expensive aspects of a performance assessment programme, with the need for additional staff and resources to be directed at the programme (PBM SIG, 2001). This may constitute a barrier to the implementation of systems' performance assessment, especially in the context of funding constraints in the storm water sector.

Storm water management tends to have a low priority in terms of funding allocation within municipalities, unless a major storm or regulatory action demands intervention (EPA, 2008). Insufficient funding has implications on the long-term management strategies for storm water systems, such as the maintenance of existing storm water infrastructure and the transition to new sustainable drainage solutions. Funding is inexorably interrelated with governance and institutional arrangements that manage the water sector (Rees et al., 2008). Storm water systems have not had a paramount role in water governance, mainly because of the low perception of these systems as a vital component of the public water service and the lack of a regulatory framework and proper funding mechanisms. Consequently, this situation hinders the implementation of performance assessment, bringing about present and future costs, as well as difficulties to manage storm water systems in a sustainable manner and to cope with risks associated with urbanisation sprawl and climate change. The use of performance assessment systems would help attain faster decision-making processes, empower decision-makers, justify decisions taken and priorities established, at the utilities level, and it would be a key tool for the supervision of the quality of service provided by utilities, at the regulation level (Pinto et al., 2017), contributing to the performance improvement in the storm water sector.

New funding mechanisms are then necessary to help the sector to cope with the numerous challenges it faces. In this sense, some countries (Australia, Canada, Ecuador, France, Germany, Poland, South Africa and United States) have implemented storm water fees in many municipalities (Tasca et al., 2017), to help fund and prioritise storm water infrastructure and management strategies. The United States is one of the most evolved countries in terms of storm water fee collection. Across the country, there are over 1,600 storm water utilities (Campbell et al., 2017), whose function is to manage the fees to recover the costs of storm water infrastructure regulatory compliance, planning, maintenance, capital improvements, and repair and replacement (EPA, 2008). According to a survey aimed at utility leaders (Black & Veatch, 2017), the top three storm water challenges stated were: (i) availability of adequate funding, (ii) enhancing public awareness and support for storm water management, and (iii) management of the expanding regulatory requirements. Most utilities are still in a phase in which their concerns are more financial, not being as mature as water and wastewater ones. The information available to the public is still mainly related to the utilities' capacity to collect storm water fees. Despite no reference to performance assessment as a strategic management tool in the sector, in the medium term, having funding sources and utilities dedicated to storm water systems may serve as a precursor for the development of performance assessment programmes.

Another reason for the poor use of performance assessment may be related to the fact that, although the reviewed PI methodologies covered different performance categories and systems, many of them were of narrow application, having not been sufficiently tested, which does not encourage their adaptation and use. Therefore, there is a need to develop a storm water performance assessment framework that may constitute a reference in the field, filling the identified gaps in the reviewed works and being applied by water utilities and municipalities.

Current and future work

Performance assessment based on PI provides a tool for organisations from any field to know their activity, the extent to which their objectives are being achieved, what factors are influencing their performance and what can be done to improve it, supporting the decision-making process. This practice has been successfully undertaken in water supply and wastewater systems, while it has been incipient in the case of storm water systems. Given the challenges that these latter systems face, the importance of integrating performance assessment systems into their management programmes is recognised.

The reviewed works proposed performance assessment methodologies, based on performance metrics, namely PI, that presented some gaps which hinder its further adaptation by water utilities and municipalities. In this regard, the need has been identified to develop a performance assessment framework of reference for storm water systems, that follows best practice recommendations.

Thus, the development of a performance assessment framework that shall be objective, standardised and flexible in its application is proposed. The framework shall also be objective-oriented, the attainment of which will be verified through assessment criteria. Performance then needs to be quantified through the application of performance metrics, such as PI, and classified based on reference values. The definition of performance metrics follows the recommendations presented in ISO 24500 standards (ISO, 2007a, 2007b, 2007c) and in the IWA Manuals of Best Practices, related to performance indicators for water and wastewater services (Matos et al., 2003; Alegre et al., 2016).

The ongoing performance assessment framework considers different activity contexts, integrating different types of storm water systems (conventional, SUDS and the combination of both), their functional categories (e.g., hydraulic, environmental, ecological, infrastructural, economic and social) and different scales of application. The reviewed works in the present article constitute a starting point for the framework development.

The defined characteristics for the performance assessment framework are meant to fill the identified gaps in the field and to facilitate its application by water utilities and municipalities, as is verified for water and wastewater systems. It would contribute to changing the current paradigm and to sensitise the sector to implement performance assessment systems, in order to support continuous improvement and enable them to be better prepared to face their present and future challenges.

Acknowledgements

The authors would like to acknowledge the financial support from Fundação para a Ciência e a Tecnologia (FCT) for the PhD grant PD/BD/114461/2016, under the doctoral programme H2Doc (‘Environmental Hydraulics and Hydrology’).

References

References
Adamsson
J.
, (
1997
).
The Swedish Experience and Viewpoints
.
In: Annals of IWSA Workshop on Performance Indicators for Transmission and Distribution Systems
,
Lisbon
,
Portugal
.
Alegre
H.
, (
1999
).
Performance indicators for water supply systems
. In:
Drought Management Planning in Water Supply Systems – Proceedings From the UIMP International Course Held in Valencia, December 1997
.
Cabrera
E.
&
García-Serra
J.
(eds).
Kluwer Academic Publishers
,
Dordrecht, The Netherlands
, pp.
148
178
.
Alegre
H.
&
Cabrera
E.
, (
2011
).
Alegre
H.
&
Coelho
S. T.
, (
2012
).
Infrastructure asset management of urban water systems
. In:
Water Supply System Analysis – Selected Topics
.
Ostfeld
A.
(ed.).
InTech
, pp.
49
74
.
Alegre
H.
,
Hirner
W.
,
Baptista
J. M.
&
Parena
R.
, (
2000
).
Performance Indicators for Water Supply Services – Manual of Best Practices
,
1st edn
.
IWA Publishing
,
London
,
UK
.
Alegre
H.
,
Cabrera
E.
&
Merkel
W.
, (
2009
).
Performance assessment of urban utilities: the case of water supply, wastewater and solid waste
.
Journal of Water Supply: Research and Technology – AQUA
58
(
5
),
305
315
.
http://doi.org/10.2166/aqua.2009.041
.
Alegre
H.
,
Baptista
J. M.
,
Cabrera
E.
,
Cubillo
F.
,
Duarte
P.
,
Hirner
W.
,
Merkel
W.
&
Parena
R.
, (
2016
).
Performance Indicators for Water Supply Services – Manual of Best Practices
,
3rd edn
.
IWA Publishing
,
London
,
UK
.
Ashley
R. M.
,
Balmfort
D. J.
,
Saul
A. J.
&
Blanskby
J. D.
, (
2005
).
Flooding in the future – predicting climate change, risks and responses in urban areas
.
Water Science and Technology
52
(
5
),
265
273
.
Auckland Council
(
2015
).
Stormwater Asset Management Plan 2015-2045
.
Auckland
,
New Zealand
.
AWWA
(
2015
).
Benchmarking Performance Indicators for Water and Wastewater Utilities: 2013 Survey Data and Analyses Report
.
American Water Works Association
,
Denver, CO
,
USA
.
AWWARF
(
1996
).
Performance Benchmarking for Water Utilities
.
American Water Works Research Foundation
,
Denver, CO
,
USA
.
Baptista
J. M.
,
Alegre
H.
, (
2009
).
Target definition and assessment of performance
. In:
Strategic Asset Management of Water Supply and Wastewater Infrastructures
.
Alegre
H.
&
Almeida
M. D. C.
(eds).
IWA Publishing
,
London
,
UK
, pp.
151
168
.
Berggren
K.
, (
2008
).
Indicators for urban drainage system – assessment of climate change impacts
. In:
Proceedings of the 11th International Conference on Urban Drainage
,
2008
,
Edinburgh, Scotland, UK
.
Bhimanadhuni
S.
, (
2015
).
Development of A Performance Index for Stormwater Pipeline Infrastructure
.
Master thesis
,
Virginia Polytechnic Institute and State University
,
Blacksburg, VA
,
USA
.
Black & Veatch
(
2017
).
2016 Stormwater utility survey. https://pages.bv.com/rs/916-IZV-611/images/2016-Stormwater-Utility-Survey.pdf (accessed 14 May 2018).
Bureau of Meteorology
(
2018
).
National Performance Report 2016-2017: Urban Water Utilities, Part A
.
Bureau of Meteorology
,
Melbourne
,
Australia
.
Butler
D.
&
Davies
J. W.
, (
2011
).
Urban Drainage
,
3rd edn
.
Spon Press
,
Abingdon
,
UK
.
Cabrera
E.
Jr
&
Pardo
M. Á.
, (
2008
).
Performance Assessment of Urban Infrastructure Services – Drinking Water, Wastewater and Solid Waste
.
IWA Publishing
,
London
,
UK
.
Cabrera
E.
Jr.
,
Dane
P.
,
Haskins
S.
&
Theuretzbacher-Fritz
H.
, (
2011
).
Benchmarking Water Services: Guiding Water Utilities to Excellence
.
IWA Publishing
,
London
,
UK
.
Campbell
C.
,
Dymond
R.
,
Key
K.
&
Dritschel
A
, . (
2017
).
Western Kentucky University Stormwater Utiltiy Survey 2017
.
Bowling Green
,
KY, USA
. .
Cardoso
M. A.
, (
2007
).
Avaliação do Desempenho de Sistemas de Drenagem Urbana. (Performance Assessment of Urban Drainage Systems)
.
PhD thesis
,
Technical University of Lisbon
,
Lisbon
,
Portugal
.
Charlesworth
S. M.
, (
2010
).
A review of the adaptation and mitigation of global climate change using sustainable drainage in cities
.
Journal of Water and Climate Change
1
(
3
),
165
180
.
http://doi.org/10.2166/wcc.2010.035
.
Cherqui
F.
,
Granger
D.
,
Métadier
M.
,
Fletcher
T.
,
Barraud
S.
,
Lalanne
P.
&
Litrico
X.
, (
2013
).
Indicators related to BMP performances: operational monitoring propositions
. In:
8th Conference NOVATECH
,
2013
,
Lyon, France
.
Danilenko
A.
,
van den Berg
C.
,
Macheve
B.
&
Moffitt
L. J.
, (
2014
).
The IBNET Water Supply and Sanitation Blue Book 2014: The International Benchmarking Network for Water and Sanitation Utilities Databook
.
The World Bank
,
Washington, DC
,
USA
. .
Dechesne
M.
,
Barraud
S.
&
Bardin
J. P.
, (
2004
).
Indicators for hydraulic and pollution retention assessment of stormwater infiltration basins
.
Journal of Environmental Management
71
(
4
),
371
380
.
http://doi.org/10.1016/j.jenvman.2004.04.005
.
EBC
(
2016
).
Learning From International Best Practices: 2016 Water and Wastewater Benchmark
.
Public Report of European Benchmarking Co-operation
.
EPA
(
2008
).
Funding stormwater programs. United States Environmental Protection Agency and the EPA Region III states of Pennsylvania, Maryland, Delaware, District of Columbia, Virginia and West Virginia. EPA 833-F-07-012.
Fekete
B. M.
,
Stakhiv
E. Z.
, (
2014
).
Performance indicators in the water resources management sector
. In:
The Global Water System in the Anthropocene
.
Bhaduri
A.
,
Bogardi
J.
,
Leentvaar
J.
&
Marx
S.
(eds).
Springer Water. Springer
,
Cham
,
Switzerland
.
http://doi.org/10.1007/978-3-319-07548-8_2.
Fletcher
T. D.
,
Shuster
W.
,
Hunt
W. F.
,
Ashley
R.
,
Butler
D.
,
Arthur
S.
,
Trowsdale
S.
,
Barraud
S.
,
Semadeni-Davies
A.
,
Bertrand-Krajewski
J.
,
Mikkelsen
P. S.
,
Rivard
G.
,
Uhl
M.
,
Dagenais
D.
&
Viklander
M.
, (
2015
).
SUDS, LID, BMPs, WSUD and more – the evolution and application of terminology surrounding urban drainage
.
Urban Water Journal
12
(
7
),
525
542
.
http://doi.org/10.1080/1573062X.2014.916314
.
Graham
A.
, (
2016
).
Sustainable drainage systems: delivering multiple benefits for people and wildlife
. In:
Sustainable Surface Water Management – A Handbook for SuDS
.
Charlesworth
S. M.
&
Booth
C. A.
(eds).
John Wiley & Sons, Ltd
,
Chichester
,
UK
, pp.
91
104
.
http://doi.org/10.1002/9781118897690.ch7
.
Haider
H.
,
Sadiq
R.
&
Tesfamariam
S.
, (
2014
).
Performance indicators for small- and medium-sized water supply systems: a review
.
Environmental Reviews
22
,
1
40
.
http://doi.org/10.1139/er-2013-0013
.
Hamilton
K.
, (
1996
).
Policy-driven indicators for sustainable development
. In:
Mediterranean Blue Plan. Environmental Performance Indicators Workshop
.
The World Bank
,
Damascus
,
Syria
.
ISO 24510
(
2007a
).
Activities Relating to Drinking Water and Wastewater Services – Guidelines for the Assessment and for the Improvement of the Service to Users
.
International Organization for Standardization
,
Geneva
,
Switzerland
.
ISO 24511
(
2007b
).
Activities Relating to Drinking Water and Wastewater Services – Guidelines for the Management of Wastewater Utilities and for the Assessment of Drinking Water Services
.
International Organization for Standardization
,
Geneva
,
Switzerland
.
ISO 24512
(
2007c
).
Service Activities Relating to Drinking Water and Wastewater – Guidelines for the Management of Drinking Water Utilities and for the Assessment of Drinking Water Services
.
International Organization for Standardization
,
Geneva
,
Switzerland
.
Jefferies
C.
, (
2004
).
SUDS in Scotland – The Monitoring Programme of Scottish Universities SUDS Monitoring Group
.
SNIFFER Final Report SR (02)51. http://www.sniffer.org.uk/files/6613/4183/8009/SR0251.pdf (accessed 26 January 2017).
Kleidorfer
M.
,
Mikovits
C.
,
Jasper-Tönnies
A.
,
Huttenlau
M.
,
Einfalt
T.
&
Rauch
W.
, (
2014
).
Impact of a changing environment on drainage system performance
.
Procedia Engineering
70
(
2009
),
943
950
.
http://doi.org/10.1016/j.proeng.2014.02.105
.
Lau
J.
,
Butler
D.
&
Schuetze
M.
, (
2002
).
Is combined sewer overflow spill frequency/volume a good indicator of receiving water quality impact?
Urban Water
4
,
181
189
.
http://doi.org/10.1016/S1462-0758(02)00013-4
.
LNEC & ERSAR
(
2013
).
Guide for assessing the quality of water and waste services provided to users – 2nd generation of the assessment system
.
2nd edition, National Civil Engineering Laboratory (LNEC)
,
Water and Waste Services Regulatory Authority (ERSAR)
,
Lisbon
,
Portugal
.
Marques
R. C.
&
De Witte
K.
, (
2010
).
Towards a benchmarking paradigm in European water utilities
.
Public Money Management
30
,
42
48
.
http://doi.org/10.1080/09540960903492364
.
Masayna
V.
,
Koronios
A.
,
Gao
J.
&
Gendron
M.
, (
2007
).
Data Quality and KPIs : A Link To Be Established
. In:
Proceedings of the 2nd World Congress on Engineering Asset Management (EAM) and the 4th International Conference on Condition Monitoring
,
Harrogate, UK
, pp.
1376
1386
.
Matos
R.
,
Cardoso
A.
,
Ashley
R.
,
Duarte
P.
,
Molinari
A.
&
Schulz
A.
, (
2003
).
Performance Indicators for Wastewater Services – Manual of Best Practices
.
IWA Publishing
,
London
,
UK
.
Matzinger
A.
,
Schmidt
M.
,
Riechel
M.
,
Hein
A.
,
Bräcker
J.
,
Strehl
C.
,
Nickel
D.
,
Libbe
J.
,
Sieker
H.
,
Pallasch
M.
,
Köhler
M.
,
Kaiser
D.
,
Brückmann
S.
,
Möller
C.
,
Büter
B.
,
Gross
G.
,
Günther
R.
,
Säumel
I.
,
Taute
T.
,
Schwarzmüller
H.
,
Bartel
H.
,
Heise
S.
,
Remy
C.
,
Sonnenberg
H.
,
Schmitt
T.
,
Heinzmann
B.
,
Joswig
K.
,
Rehfeld-Klein
M.
,
Reichmann
B.
&
Rouault
P.
, (
2014
).
Quantifying the Effects of Urban Stormwater Management – Towards a Novel Approach for Integrated Planning
. In:
Proceedings of the 13th International Conference on Urban Drainage
,
Sarawak, Malaysia
.
Mcintosh
A. C.
&
Yñoguez
C.
, (
1997
).
Second Water Utilities Data Book – Asian and Pacific Region
.
Asian Development Bank's Regional Technical Assistance no. 5694
,
Manila
,
Philippines
.
Mitchell Shire Council
(
2012
).
Infrastructure Asset Management Plan – Part F – Urban Stormwater Drainage System
.
Mitchell Shire Council
,
Broadford, Victoria
,
Australia
.
Moura
P.
,
Barraud
S.
,
Baptista
M.
&
Malard
F.
, (
2010
).
Multicriteria decision-aid method to evaluate the performance of stormwater infiltration systems over the time
. In:
7th Conference NOVATECH
,
2010
,
Lyon, France
.
MWA
(
1996
).
Performance Indicators for Water Supply
.
Malaysian Water Association
,
Kuala Lumpur
,
Malaysia
.
Nafi
A.
,
Tcheng
J.
&
Beau
P.
, (
2015
).
Comprehensive methodology for overall performance assessment of water utilities
.
Water Resources Management
29
(
15
),
5429
5450
.
http://doi.org/10.1007/s11269-015-1127-3
.
Neely
A.
,
Gregory
M.
&
Platts
K.
, (
1995
).
Performance measurement system design: a literature review and research agenda
.
International Journal of Operations & Production Management
15
(
4
),
80
116
.
http://dx.doi.org/10.1108/01443579510083622
.
Nie
L.
,
Lindholm
O.
,
Lindholm
G.
&
Syversen
E.
, (
2009
).
Impacts of climate change on urban drainage systems – a case study in Fredrikstad, Norway
.
Urban Water Journal
6
(
4
),
323
332
.
http://doi.org/10.1080/15730620802600924
.
Niemczynowicz
J.
, (
1989
).
Impact of the greenhouse effect on sewerage systems – Lund case study
.
Hydrological Sciences Journal
34
(
6
),
651
666
.
http://doi.org/10.1080/02626668909491373
.
Ofwat
(
1996a
).
1995-1996 Report on Levels-of-Service for the Water Industry in England and Wales
.
Office of Water Services
,
Birmingham
,
UK
.
Ofwat
(
1996b
).
1995-1996 Report on the Cost of Water Delivered and Sewage Collected
.
Office of Water Services
,
Birmingham
,
UK
.
Ofwat
(
2013
).
Key Performance Indicators – Guidance
.
Office of Water Services
,
Birmingham
,
UK
.
PBM SIG
(
1995
).
How to Measure Performance: A Handbook of Techniques and Tools
.
Training Resources and Data Exchange (TRADE) Performance-based Management Special Interest Group, Oak Ridge Associated Universities
,
USA
.
PBM SIG
(
2001
).
The performance-based management handbook, vol.4, Collecting data to assess performance
.
Training Resources and Data Exchange (TRADE), Performance-based Management Special Interest Group, Oak Ridge Institute for Science and Education
,
USA
.
Pinto
F. S.
,
Simões
P.
&
Marques
R. C.
, (
2017
).
Raising the bar: the role of governance in performance assessments
.
Utilities Policy
49
,
38
47
.
http://doi.org/10.1016/j.jup.2017.09.001
.
Rees
J.
,
Penny
J.
&
Hall
A.
, (
2008
).
Water Financing and Governance
.
Global Water Partnership
,
Stockholm
,
Sweden
.
Revitt
D. M.
,
Scholes
L.
&
Ellis
J. B.
, (
2008
).
Assessment criteria for sustainable urban drainage systems
. In:
DayWater: an Adaptive Decision Support System for Urban Stormwater Management
.
Thevenot
D. R.
(ed.).
IWA Publishing
,
London
,
UK
, pp.
65
78
.
Rohrhofer
K.
,
Leroch
K.
,
Cate
F. M.
, (
2008
).
Performance indicators as a control mechanism for the effective use of donor funds
. In:
Performance Assessment of Urban Infrastructure Services
.
Cabrera
E.
&
Pardo
M. A.
(eds).
IWA Publishing
,
London
,
UK
, pp.
95
103
.
Semanedi-Davies
A.
,
Hernebring
C.
,
Svensson
G.
&
Gustafsson
L. -G.
, (
2005
).
The Impacts of Climate Change and Urbanisation on Urban Drainage in Helsingborg
.
Final Report, Department of Water Resources Engineering, Lund University
,
Lund
,
Sweden
.
Sjovold
F.
,
Conroy
P.
&
Algaard
E.
, (
2008
).
Performance Assessment of Urban Infrastructure Services.State-of-the-art in the Drinking Water, Wastewater and Solid Waste Sectors
.
Final scientific report of COST Action C18, COST
.
Sundberg
C.
,
Svensson
G.
&
Söderberg
H.
, (
2004
).
Re-framing the assessment of sustainable stormwater systems
.
Clean Technologies and Environmental Policy
6
(
2
),
120
127
.
http://doi.org/10.1007/s10098-003-0227-6
.
Tasca
F.
,
Assunção
L.
&
Finotti
A.
, (
2017
).
International experiences in stormwater fee
.
Water Science and Technology
2017
(
1
),
287
299
.
http://doi.org/10.2166/wst.2018.112
.
Toffol
S.
, (
2006
).
Sewer System Performance Assessment – An Indicators Based Methodology
.
PhD thesis
,
University of Innsbruck
,
Innsbruck
,
Austria
.
van den Berg
C.
&
Danilenko
A.
, (
2011
).
The IBNET Water Supply and Sanitation Performance Blue Book 2011: The International Benchmarking Network for Water and Sanitation Utilities Databook
.
The World Bank
,
Washington, DC
,
USA
.
Vieira
P.
, (
2009
).
Avaliação de Desempenho de Estações de Tratamento de água Para Consumo Humano (Performance Assessment of Drinking Water Treatment Plants)
.
PhD thesis
,
University of Algarve
,
Faro
,
Portugal
.
Vilanova
M. R. N.
,
Filho
P. M.
&
Balestieri
J. A. P.
, (
2015
).
Performance measurement and indicators for water supply management: review and international cases
.
Renewable and Sustainable Energy Reviews
43
,
1
12
.
http://doi.org/10.1016/j.rser.2014.11.043
.
Waters
D.
,
Watt
W. E.
,
Marsalek
J.
&
Anderson
B. C.
, (
2003
).
Adaptation of a storm drainage system to accommodate increased rainfall resulting from climate change
.
Journal of Environmental Planning and Management
46
(
5
),
755
770
.
http://doi.org/10.1080/0964056032000138472
.
Wellington Water
(
2015
).
Statement of Intent 2016-19
.
Wellington
,
New Zealand
.
Winz
I.
,
Brierley
G.
&
Trowsdale
S.
, (
2011
).
Dominant perspectives and the shape of urban stormwater futures
.
Urban Water Journal
8
(
6
),
337
349
.
http://doi.org/10.1080/1573062X.2011.617828
.
Woods-Ballard
B.
,
Kellagher
R.
,
Martin
P.
,
Jefferies
C.
,
Bray
C.
&
Shaffer
P.
, (
2007
).
The SUDS Manual
.
CIRIA Report No. C697
,
London
,
UK
.
Woods-Ballard
B.
,
Wilson
S.
,
Udale-Clarke
H.
,
Illman
S.
,
Scott
T.
,
Ashley
R.
&
Kellagher
R.
, (
2015
).
The SUDS Manual
.
CIRIA Report no. C753
,
London
,
UK
.
Yeppes
G.
&
Dianderas
A.
, (
1996
).
Water and Wastewater Utilities Indicators
.
Water and Sanitation Division, The World Bank
,
Washington, DC
,
USA
.
Zhang
D.
,
Gersberg
R. M.
,
Ng
W. J.
&
Tan
S. K.
, (
2017
).
Conventional and decentralized urban stormwater management: a comparison through case studies of Singapore and Berlin, Germany
.
Urban Water Journal
14
(
2
),
113
124
.
http://doi.org/10.1080/1573062X.2015.1076488
.