Abstract

Efficiency is a key topic nowadays in the water sector as customers' expectations are continuously increasing, legislation becomes more demanding, utilities become more exposed to public opinion and expectations are increasing towards a high-quality service at an affordable cost and both combined with high risk avoidance. The implementation of efficiency projects, such as Non-Revenue Water (NRW) reduction projects are fundamental both from an economic and an environmental perspective. However, decisions on the implementation of these projects are often reduced to short-term economic criteria or to a cost-benefit analysis at best, not considering an adequate timeframe that would potentially allow the accommodation of the adopted NRW reduction measures (capital and operational expenditures). Infrastructure asset management requires a strategic view on water systems based on the long-term balance of performance, cost and risk aiming at the adequate management of utilities' physical, human, technological and intangible assets. Water managers should take these principles into consideration when deciding whether to implement efficiency projects, since the long-term effect of current water systems' inefficiencies can translate into significant capital expenditures in the future. This paper presents an evaluation of tangible and intangible gains that result from NRW reduction projects and why intangible gains should be part of decision-making processes on whether to implement them or not.

INTRODUCTION

Water utilities' responsibility to provide high quality standards on a sustainable basis requires a significant effort in the adequate use of resources and on the management of its assets to ensure safe drinking-water quality and to meet customers' expectations. Water utilities are increasingly challenged to improve their efficiency; however, decisions on the implementation of efficiency projects like Non-Revenue Water (NRW) reduction projects, are often reduced to short-term economic criteria or to a cost-benefit analysis, at best.

Narrowing the decision on implementing efficiency projects to operational costs or capital expenditure needs without properly evaluating the direct and indirect gains generated from this type of project is a biased analysis that overlooks the improvement of the utility's intangible assets.

Intangible assets such as knowledge, skills, relationships, processes, brands or culture are vital strategic resources. Governments have started highlighting the importance of intangibles as drivers for economic growth and encouraging firms to pay higher attention to their intangible assets. Investment in intangible aspects of business is essential for companies to provide added-value products and services (Mehta & Madhani 2008).

Urban water infrastructure delivers a range of services to communities that have societal value and are paid for through water bills and/or taxes. Changes in the structure, type, function and management of this infrastructure influence both the cost of service provision and the value generated (Marlow et al. 2011). Investment in infrastructure should ideally be assessed in this context, which requires the analysis of cost and benefits at various spatial and temporal scales, including strategic and tactical levels. If these are not included, there is a tendency to adopt short-term management strategies that could result in lower service levels in the future, which increase the overall level of cost imposed on a community (Burn et al. 2007).

An example is the investment required when implementing NRW reduction projects, understood as the implementation of intentional actions to decrease the volumes of real losses, apparent losses and unbilled authorized consumption. Typically, these projects involve investment regarding the purchase of flow meters to increase metering coverage, valves to implement district metered areas (DMA), leak detection equipment for location of invisible bursts, network rehabilitation to improve the infrastructure's condition, and customer meters to decrease metering inaccuracies in customer billing. While the investment and the operational costs associated with these projects are generally well estimated, the benefits derived from these projects are not.

The benefits of NRW reduction and control are many and interlinked. NRW represents an opportunity cost not only for the concerned service provider, but also for cities, the environment and the broader economy. As stated by World Bank (2016), NRW directly improves several aspects of water utilities (Figure 1 – black arrows) as part of an ongoing cycle where these aspects contribute to the improvement of others (Figure 1).

Figure 1

Example of inter-related benefits of NRW management (Adapted from World Bank 2016).

Figure 1

Example of inter-related benefits of NRW management (Adapted from World Bank 2016).

The value generated from efficiency projects can be divided into direct gains, such as reduction of real losses, and indirect gains, such as the improvement of water system's operational control. Additionally, the benefits can also be divided into tangible and intangible gains.

On one hand, tangible gains typically fall into one of the following categories: cost reduction, revenue optimization, risk reduction, regulatory compliance and the ability to enter new markets and to develop new products (Sherman 2015).

Intangible gains, on the other hand, correspond to non-monetary assets that cannot be seen, touched or physically measured (Mehta & Madhani 2008) being typically grouped into three broad categories: rights (e.g. leases, agreements, licenses), relationships (e.g. trained workforce, customer relationships) and intellectual property (e.g. patents, trademarks, marketing strategies).

A proper identification of the value generated by NRW reduction projects can support water managers to make a reasoned decision on the implementation of these projects based on quantifiable or qualitative criteria and learning from the experience of other utilities.

AGS, a company owned by Marubeni and responsible for the management of 13 water utilities in Portugal and Brazil under concession agreements and public-private partnerships and for the service provision of engineering services to water utilities in Europe, South America and Asia, must deal with a significant amount of assets, including physical, human (Feliciano et al. 2016), technological and intangible assets, to get the most value from them.

Although the valuation of intangible assets has become a widespread topic of interest in the new economy (Tsai et al. 2016), very few project development studies, at inter-organizational project level, adopt a comprehensive view of value considering long-term and intangible gains (Yan & Wagner 2017). When evaluating projects at an urban water utility's level, the consideration of such gains seems to be even less common.

To our knowledge, the identification, quantification and/or classification of intangible gains resulting from NRW reduction projects has not been formally done. As such, the aim of the present work was to analyse the true benefits and gains resulting from such projects and why these should be part of decision-making processes on whether to implement the projects or not. A proper identification of the value generated by NRW reduction projects can support water managers in making a reasoned decision on the implementation of these projects based on quantifiable or qualitative criteria and to learn from the experience of other utilities.

METHODS

Two water utilities currently implementing NRW reduction projects under AGS' coordination and technical support based on infrastructure asset management methodologies were selected to determine the direct and indirect gains of the projects. To acknowledge the tangible and intangible gains generated by the projects, two alternatives were considered: a status-quo alternative, corresponding to not implementing the project (t0) and project implementation after three years (t3).

Characterization and evaluation of the operational performance

The water utilities were firstly characterized and evaluated in terms of operational performance at status quo and after three years of project implementation. Selected metrics (Table 1) included the assessment of NRW (%), real losses per input volume (%), apparent losses per input volume (%), mains bursts [no./(100 km.year)] and service connections bursts [no./(1000 sc.year)] (Alegre et al. 2016). Additionally, the Infrastructure Value Index (IVI) was also considered at utility level as it provides an indication of the maturity level of a specific water network, being defined as the ratio between the current value of an infrastructure and its replacement cost (Alegre 2007).

Table 1

Performance and cost metrics selected for utility characterization

DesignationUnitsFormulationSourceReference values
Non-revenue water by volume  Alegre et al. (2016)   [0;20]
] 20;30]
]30;100] 
Apparent losses per input volume   [0;5]
]5;15]
]15;100] 
Real losses per input volume  Adapted from Alegre et al. (2016)   [0;15]
]15;25]
]25;100] 
Mains bursts No./(100 km.year)  Alegre et al. (2016)   [0;30]
]30;60]
]60;+∞[ 
Service connections bursts No./(1000 sc.year)   [0;15]
]15;30]
]30;+∞[ 
Infrastructure Value Index – 
t: Reference time;
N: Total number of assets;
rci,t: Replacement cost of asset i at time t;
ruli,t: Residual useful life of asset i at time t;
euli: Expected useful life of asset i. 
Alegre (2007)   [0.4;0.6]
]0.6;1.0]
[0.0;0.4[ 
DesignationUnitsFormulationSourceReference values
Non-revenue water by volume  Alegre et al. (2016)   [0;20]
] 20;30]
]30;100] 
Apparent losses per input volume   [0;5]
]5;15]
]15;100] 
Real losses per input volume  Adapted from Alegre et al. (2016)   [0;15]
]15;25]
]25;100] 
Mains bursts No./(100 km.year)  Alegre et al. (2016)   [0;30]
]30;60]
]60;+∞[ 
Service connections bursts No./(1000 sc.year)   [0;15]
]15;30]
]30;+∞[ 
Infrastructure Value Index – 
t: Reference time;
N: Total number of assets;
rci,t: Replacement cost of asset i at time t;
ruli,t: Residual useful life of asset i at time t;
euli: Expected useful life of asset i. 
Alegre (2007)   [0.4;0.6]
]0.6;1.0]
[0.0;0.4[ 

Good performance Fair performance Poor performance

The qualitative assessment of each measure, in terms of ‘good’, ‘fair’ or ‘poor’ performance was accomplished by comparing the results with reference values from the Portuguese water Regulator assessment system (ERSAR 2019).

Evaluating gains from efficiency projects

A set of direct and indirect gains were identified for these water utilities, covering areas such as organization, human resources, data and information, operational control, quality of service, relationship with stakeholders, environmental, economic and infrastructure sustainability (Table 2).

Table 2

Gains generated from NRW reduction projects

GainDescriptionType
DirectIndirectTangibleIntangible
Reduction in water losses volumes Decrease in water volumes entering the system   
Increase in revenue water volumes Volume recovered due to the decrease in customer metering errors   
Reduction in transmission and distribution costs Decrease in energy costs related to water pumping and in chemical costs related to chlorination   
Reduction of penalties due to service interruptions Decrease in the number of fines imposed by the Regulator due to supply interruptions   
Postponement of investment to increase systems' capacity Suppress or postpone investment in facilities to deal with system's water loss volumes   
Reduction of operation and maintenance costs Decrease in costs related with bursts repair and equipment failures   
Improvement of the company's image Improvement of the company's image perceived by customers, Regulator entity and other stakeholders   
Increase in organizational efficiency Improvement of internal communication and operational workflows   
Increase in technical knowledge Increase of teams' responsiveness and autonomy   
Improvement of data management Implementation of new tools and practices for data management (collection, integration with other systems, performance assessment)   
Increase in resilience to climate change Water system capability to handle with extreme events such as droughts or storms   
Reduction of illegal connections Decrease of illegal connections due to detection, enforcement and customer awareness   
GainDescriptionType
DirectIndirectTangibleIntangible
Reduction in water losses volumes Decrease in water volumes entering the system   
Increase in revenue water volumes Volume recovered due to the decrease in customer metering errors   
Reduction in transmission and distribution costs Decrease in energy costs related to water pumping and in chemical costs related to chlorination   
Reduction of penalties due to service interruptions Decrease in the number of fines imposed by the Regulator due to supply interruptions   
Postponement of investment to increase systems' capacity Suppress or postpone investment in facilities to deal with system's water loss volumes   
Reduction of operation and maintenance costs Decrease in costs related with bursts repair and equipment failures   
Improvement of the company's image Improvement of the company's image perceived by customers, Regulator entity and other stakeholders   
Increase in organizational efficiency Improvement of internal communication and operational workflows   
Increase in technical knowledge Increase of teams' responsiveness and autonomy   
Improvement of data management Implementation of new tools and practices for data management (collection, integration with other systems, performance assessment)   
Increase in resilience to climate change Water system capability to handle with extreme events such as droughts or storms   
Reduction of illegal connections Decrease of illegal connections due to detection, enforcement and customer awareness   

To make reasoned decisions on the adoption of a NRW project, one should compare two alternatives: the implementation of the project and a status-quo alternative. In both cases, it is important to include all the relevant criteria for comparison purposes, even though some of them are difficult to quantify (intangible gains).

According to Delgado-Galván et al. (2010), when dealing with intangibles, judgments are rarely consistent unless they are forced in some artificial manner. The problem derives from the fact that comparisons will only work with well-defined scales of measurement. Nevertheless, direct comparisons are necessary to establish measurements for intangible properties that have no scales of measurement. To evaluate intangible gains generated from NRW reduction projects, a qualitative scale for the identified gains is proposed in Table 3. For each intangible gain, support data was identified to substantiate the qualitative evaluation and diminish the inherent subjectivity.

Table 3

Qualitative scale for the evaluation of intangible gains

GainEvaluation dataLevel
12345
Very LowLowMediumHighVery High
Improvement of company's image 
  • - Water Regulator ranking;

  • - Quality of service perceived by customers;

  • - Complaints.

 
Low reputation on a National level (public opinion) Low reputation on a National level (in the water sector) Moderate reputation with some problems on a local level Good reputation with some customers complaints Very good reputation 
Increase in organizational efficiency 
  • - Communication frequency with other departments;

  • - Data requests;

  • - Inter-departmental meetings.

 
Company departments working with no internal communication Low level of communication between similar departments (technical or financial and commercial) Company with technical and commercial departments with separate goals Company with technical and commercial departments with common goals in specific areas and topics Cross-company decision making processes 
Increase in technical knowledge 
  • - Response time to anomalous situations;

  • - Requests from external entities;

  • - Time spent in R&D activities;

  • - Participation in technical events.

 
Qualified technicians with no specific knowledge Qualified technicians with some specific knowledge and low control of the system Qualified technicians with some specific knowledge and moderate control of the system Qualified technicians with moderate knowledge and control of the system Qualified technicians with high knowledge level (system, sector and company) 
Improvement of data management 
  • - Information systems for monitoring & control;

  • - Information systems integration;

  • - Variables collected.

 
Company with no support tools and low quality data Incomplete information systems (GIS, SCADA, CRM) Complete information systems (GIS, SCADA, CRM, maintenance, work orders, water quality, others) Integration between systems and control procedures on data quality Company with tools and data that support the systems’ knowledge and real-time control 
Increase in resilience to climate change 
  • - Concerns with environmental aspects;

  • - Service interruption time due to extreme events.

 
No concern related to environmental aspects Few concerns with environmental aspects (only when extreme events occur) Few environmental concerns reflected in water saving plans Environmental concerns with structured tactical plans to reduce water losses Company with strategic plans regarding water scarcity, NRW and water safety 
GainEvaluation dataLevel
12345
Very LowLowMediumHighVery High
Improvement of company's image 
  • - Water Regulator ranking;

  • - Quality of service perceived by customers;

  • - Complaints.

 
Low reputation on a National level (public opinion) Low reputation on a National level (in the water sector) Moderate reputation with some problems on a local level Good reputation with some customers complaints Very good reputation 
Increase in organizational efficiency 
  • - Communication frequency with other departments;

  • - Data requests;

  • - Inter-departmental meetings.

 
Company departments working with no internal communication Low level of communication between similar departments (technical or financial and commercial) Company with technical and commercial departments with separate goals Company with technical and commercial departments with common goals in specific areas and topics Cross-company decision making processes 
Increase in technical knowledge 
  • - Response time to anomalous situations;

  • - Requests from external entities;

  • - Time spent in R&D activities;

  • - Participation in technical events.

 
Qualified technicians with no specific knowledge Qualified technicians with some specific knowledge and low control of the system Qualified technicians with some specific knowledge and moderate control of the system Qualified technicians with moderate knowledge and control of the system Qualified technicians with high knowledge level (system, sector and company) 
Improvement of data management 
  • - Information systems for monitoring & control;

  • - Information systems integration;

  • - Variables collected.

 
Company with no support tools and low quality data Incomplete information systems (GIS, SCADA, CRM) Complete information systems (GIS, SCADA, CRM, maintenance, work orders, water quality, others) Integration between systems and control procedures on data quality Company with tools and data that support the systems’ knowledge and real-time control 
Increase in resilience to climate change 
  • - Concerns with environmental aspects;

  • - Service interruption time due to extreme events.

 
No concern related to environmental aspects Few concerns with environmental aspects (only when extreme events occur) Few environmental concerns reflected in water saving plans Environmental concerns with structured tactical plans to reduce water losses Company with strategic plans regarding water scarcity, NRW and water safety 

To allow a reasoned analysis of the alternatives considered, metrics and qualitative levels were converted to a normalized scale ranging from zero to three through performance functions (Cardoso et al. 2004). These functions convert variables into performance indices where values between zero and one correspond to poor performance, values between one and two correspond to fair performance and values between two and three correspond to good performance (Figure 2).

Figure 2

Conversion of quantitative metrics and qualitative levels to indices through performance functions.

Figure 2

Conversion of quantitative metrics and qualitative levels to indices through performance functions.

Additionally, an analysis of the economic benefits derived from the projects was carried out to complement the approach. For this purpose, investment costs (network rehabilitation, customer meters, flow meters and equipment) and accumulated results (tangible direct and indirect gains) were determined for an adequate timeframe that would allow to incorporate the results of the adopted NRW reduction measures (10-year period).

The economic benefits for the intangible gains were determined considering the number of working hours saved in resolving customer complaints and managing large amounts of data and increased revenue from externalizing the increased know-how acquired in the project.

ESTIMATING GAINS FROM NRW REDUCTION PROJECTS: A CASE-STUDY

Table 4 presents the main characteristics of the water utilities chosen for this study. Utility A is located in an arid area with water scarcity problems and has a somewhat aged infrastructure with a high level of burst frequency. Utility B is located in a Mediterranean climate area with abundant water availability, and presents a balanced infrastructure, a high level of real losses level and lower water production costs when compared to Utility A.

Table 4

Characterization of water utilities A and B

UtilityMains length (km)Service connections (no.)Customers (no.)Distribution network materials
1,117 112,415 161,896 Asbestos cement (35%), PVC (32%), HDPE (29%), others (4%) 
2,336 218,822 245,750 PVC (48%), asbestos cement (29%), HDPE (18%), others (5%) 
UtilityMains length (km)Service connections (no.)Customers (no.)Distribution network materials
1,117 112,415 161,896 Asbestos cement (35%), PVC (32%), HDPE (29%), others (4%) 
2,336 218,822 245,750 PVC (48%), asbestos cement (29%), HDPE (18%), others (5%) 

Figure 3 presents the utilities' assessment in terms of economic, environmental and infrastructure sustainability before (t0) and after three years (t3) of the start of the project based on selected metrics from the literature.

Figure 3

Utilities assessment regarding NRW, IVI and burst frequency at year 0 (t0) and year 3 (t3) of the project (red–poor performance, yellow–fair performance, green–good performance).

Figure 3

Utilities assessment regarding NRW, IVI and burst frequency at year 0 (t0) and year 3 (t3) of the project (red–poor performance, yellow–fair performance, green–good performance).

Analysing the results, it is possible to observe that NRW levels have decreased in both utilities (from 36% to 27% in Utility A and from 43% to 36% in Utility B), corresponding to a combined reduction of about 10.5 million m3 in the first three years of the project. NRW reduction resulted from the decrease of input volumes (direct gain), achieved from active leakage detection activities supported by network sectorization, metering and real-time flow monitoring software, and from the increase in revenue volumes (direct gain) through customer meter replacement and detection of unauthorized consumption. The observed increase in the level of apparent losses resulted from a more accurate estimation of metering errors when compared to the projects' initial stage. In terms of burst frequency, the number of natural bursts increased in Utility A, evidencing infrastructure problems, mainly due to aging and poor installation conditions of younger pipes, while in Utility B natural bursts have decreased, evidencing a good infrastructure condition that sustains reduction of real losses.

For comparison purposes, the tangible gains identified in Table 2 were quantified through established metrics while intangible gains were evaluated considering the qualitative matrix presented in Table 3. These were converted to a scale from 0 to 3; that is, from poor performance to good performance (Figure 4) using the respective performance function (Figure 2). When assessing the gains of NRW projects, typically only NRW-related metrics are considered. However, when including other types of gains, tangible and intangible, the impact of the project and the different improvement areas it enables (e.g. organizational efficiency, data management) become clear, even if intangible gains are estimated in a qualitative manner.

Figure 4

Project gains comparison for alternatives, status-quo (t0) and project implementation (t3).

Figure 4

Project gains comparison for alternatives, status-quo (t0) and project implementation (t3).

Figure 5

(a) Investment and NRW reduction; (b) project results for a 10-year period in Utility A.

Figure 5

(a) Investment and NRW reduction; (b) project results for a 10-year period in Utility A.

Figure 6

(a) Investment and NRW reduction; (b) project results for a 10-year period in Utility B.

Figure 6

(a) Investment and NRW reduction; (b) project results for a 10-year period in Utility B.

To have an integrated analysis of this type of project it is also important to analyse the economic dimension.

To evaluate the costs and the benefits of the NRW reduction projects, executed and planned investment (at current prices) and NRW reduction volumes were determined for both utilities. Additionally, project results (R) for a 10-year period were estimated for the two utilities, considering direct gains (DG) and direct and indirect gains (IG) combined (Figures 5 and 6). All tangible gains (direct and indirect) identified in Table 2 were considered in the analysis, according to Equation (1). 
formula
(1)
  • m: number of direct gains

  • n: number of indirect gains

When analysing accumulated results, one can observe that the project's break-even is obtained around six years for Utility A and Utility B, evidencing that when short-term analysis of NRW projects are adopted they can turn out to be disadvantageous from an economic perspective by not allowing enough time for the economic gains to overcome costs and amortizations associated with the project.

The incorporation of indirect gains such as the decrease in Regulator's fines or the postponing of infrastructure capacity investment (differential area highlighted in grey in Figures 5(b) and 6(b)) will result in earlier break evens for both utilities, as expected.

Although Utility A presents higher water production cost when compared to Utility B, the latter will achieve positive results earlier than Utility A because NRW reduction volumes are greater in Utility B. Additionally, the incorporation of indirect gains is estimated to have higher impact in Utility B as the NRW reduction allowed them to postpone investment to increase systems' capacity in some areas of the distribution system that were at the limit of storage capacity.

In this case-study, the generation of intangible gains (IntG) was clear in terms of the improvement of utilities' image, systems' resilience, technical know-how and data management capabilities. Financial savings by reducing manpower required to resolve customer complaints and operational problems (which, in turn, allowed avoidance of planned investments) and revenue increase by externalization of the acquired know-how were included in the financial analysis according to Equation (2). By incorporating intangible gains in the analysis, water managers will have a more powerful tool to justify systems' rehabilitation and upgrade (Figure 7). 
formula
(2)
  • m: number of direct gains

  • n: number of indirect gains

  • p: number of intangible gains

Figure 7

Estimated results for a 10-year period in Utility B including intangible gains (green line). The full colour version of this figure is available in the online version of this paper, at http://dx.doi.org/10.2166/ws.2020.079.

Figure 7

Estimated results for a 10-year period in Utility B including intangible gains (green line). The full colour version of this figure is available in the online version of this paper, at http://dx.doi.org/10.2166/ws.2020.079.

FINAL REMARKS

The case-study herein presented demonstrated that cost-benefit analyses of NRW reduction projects should be developed in a timeframe that allows the effect of the adopted NRW reduction measures (investment and operational expenditures) to be accommodated and that indirect gains should be incorporated in the decision-making process of implementing efficiency projects. Also, intangible gains should be considered in the decision-making process. Even if they are not easily quantifiable in monetary terms, a qualitative evaluation of intangibles will provide water managers with additional arguments to justify systems' improvement and efficiency. Thus, an identification of all possible short, medium and long-term benefits should be done and qualified or quantified when possible in order to obtain a clear picture of the impact that efficiency projects will have in the water utility. The methodology presented is an approach to the identification and incorporation of overlooked gains from NRW reduction projects in the economic analysis of the benefits derived from them. Further work is required to diminish the subjectivity inherent to intangible gains estimation.

Infrastructure asset management requires a strategic view of water systems based on the long-term balance of performance, cost and risk, aiming at the adequate management of utilities' physical, human, technological and intangible assets. Water managers should change decision-making processes from purely economic criteria to include environmental and social criteria as well, considering not only tangible gains but also the intangible ones since the long-term effect of current water systems' inefficiencies can translate into significant capital expenditures, environmental and social consequences in the future.

REFERENCES

REFERENCES
Alegre
H.
2007
Drinking Water and Wastewater Infrastructure Asset Management
.
Program developed under certification process for the functions of scientific research coordination in LNEC
,
Lisbon
,
Portugal
.
ISBN: 9789724921341. (in Portuguese)
.
Alegre
H.
,
Baptista
J. M.
,
Cabrera
E.
Jr.
,
Cubillo
F.
,
Duarte
P.
,
Hirner
W.
,
Merkel
W.
&
Parena
R.
2016
Performance Indicators for Water Supply Services
, 3rd edn.
IWA Publishing
,
London, UK
.
Burn
S.
,
Marlow
D.
,
Moglia
M.
&
Buckland
P.
2007
Asset management for urban infrastructure
.
Water Asset Management International
3
(
2
),
12
18
.
IWA Publishing, London, UK
.
Cardoso
M. A.
,
Coelho
S.
,
Matos
R.
&
Alegre
H.
2004
Performance assessment of water supply and wastewater systems
.
Urban Water Journal.
1
(
1
).
doi: 10.1080/15730620410001732053
.
Delgado-Galván
X.
,
Pérez-Garcia
R.
,
Izquierdo
J.
&
Mora-Rodríguez
J.
2010
An analytic hierarchy process for assessing externalities in water leakage management
.
Mathematical and Computer Modelling.
52
(
7–8
),
1194
1202
.
doi.org/10.1016/j.mcm.2010.03.014
.
ERSAR – Portuguese Water and Solid Waste Services Regulator
2019
Water, Wastewater and Solid Waste Services Assessment Guide
.
3rd generation of the assessment system. Technical guide no. 22
,
Lisbon
, p.
352
.
(in Portuguese)
.
Feliciano
J.
,
Almeida
R.
,
Santos
A.
,
Ramalho
P.
,
Ganhão
A.
,
Covas
D.
&
Alegre
H.
2016
Assessing human resources renovation needs in water utilities
.
Water Practice & Technology.
11
(
4
),
728
735
.
doi.org/10.2166/wpt.2016.078
.
Marlow
D.
,
Pearson
L.
,
MacDonald
D. H.
,
Whitten
S.
&
Burn
S.
2011
A framework for considering externalities in urban water asset management
.
Water Science & Technology.
64
(
11
),
2199
2206
.
doi: 10.2166/wst.2011.789
.
Mehta
A.
&
Madhani
P.
2008
Intangible assets – an introduction
.
The Accounting World
8
(
9
),
11
19
.
Sherman
R.
2015
Justifying BI: Building the business and technical case
. In:
Business Intelligence Guidebook
.
ISBN 978-0-12-411461-6
, p.
550
.
doi:10.1016/C2012-0-06937-2
.
Tsai
C.-F.
,
Lu
Y.-H.
,
Hung
Y.-C.
&
Yen
D. C.
2016
Intangible assets evaluation: the machine learning perspective
.
Neurocomputing
175
,
110
120
.
doi:10.1016/j.neucom.2015.10.041
.
World Bank
2016
Using Performance-Based Contracts to Reduce Non-Revenue Water
.
International Bank for Reconstruction and Development
.
Report No. 130050. June
, p.
22
.
Yan
T.
&
Wagner
M.
2017
Do what and with whom? value creation and appropriation in inter-organizational new product development projects
.
International Journal of Production Economics.
191
,
1
14
.
https://doi.org/10.1016/j.ijpe.2017.05.010
.