Abstract

Analyses of complex water management decision-making problems, involving tradeoffs amongst multiple criteria, are often undertaken using multi-criteria decision analysis (MCDA) techniques. Various forms of uncertainty may arise in the application of MCDA methods, including imprecision, inaccuracy or ill determination of data. The ELECTRE family methods deal with imperfect knowledge of data by incorporating ‘pseudo-criteria’, with discrimination thresholds, to interpret the outranking relation as a fuzzy relation. However, the task of selecting thresholds for each criterion can be difficult and ambiguous for decision-makers. In this paper, we propose a confidence-interval-based approach which aims to reduce the subjective input required by decision-makers. The proposed approach involves defining the uncertainty in the input values using confidence intervals and expressing thresholds as a function of the interval estimates. The usefulness of the approach is illustrated by applying it to evaluate the water supply and sewerage services in Spain. Results show that the confidence interval approach may be interesting in some cases (e.g. when dealing with statistical data from surveys or measuring equipment), but should never replace the preferences or judgments of the actors involved in the decision process.

INTRODUCTION

Decision-making in water management is inherently complex. Water decisions often involve large numbers of alternatives, competing objectives, and participation of multiple stakeholders with conflicting interests (Hyde et al. 2005). Consequently, a formal framework for water resources decision-making is required. Multi-criteria decision analysis (MCDA) provides a structured approach for analyzing decision problems with multiple objectives and criteria (Mutikanga et al. 2011). MCDA can assist decision-makers in identifying critical issues, assigning relative priorities to those issues, selecting the best compromise solutions, and enhancing communication in the evaluation of decision problems (Flug et al. 2000).

Numerous MCDA methods have been developed over the years, and are commonly classified in three classes: full aggregation approach, outranking approach, and goal, aspiration or preference-level approach (Ishizaka & Nemery 2013). The ELimination and Choice Expressing the REality (ELECTRE) methods developed by Roy (1991) belong to the group of outranking approaches and are one of the best known and widely applied methods, especially in Europe (Wang & Triantaphyllou 2006). This is evident from their broad use in wide-ranging decision-making situations, from natural resources and environmental management to structural engineering, logistics and supply chain management, and public planning and policy decisions (Govindan & Jepsen 2016). In water management, the specific application areas include ranking water allocation strategies (Bella et al. 1996; Zardari et al. 2010), assessing projects for river basin planning and development (Duckstein et al. 1982; Raj 1995), selecting alternative strategies for managing irrigation systems (Raju et al. 2000; Pedras & Pereira 2009), choosing operation rules for reservoir systems (Ko et al. 1994; Malekmohammadi et al. 2011), prioritizing pipe rehabilitation projects in water and sewer networks (Carriço et al. 2012; Tscheikner-Gratl et al. 2017), comparing watershed management schemes (Tecle et al. 1988; Ceccato et al. 2011) or identifying priority water users or regions for future inversions (Roy et al. 1992; Morais & Almeida 2006). However, despite their extensive application, the drawbacks of ELECTRE methods are still discussed by researchers (Figueira & Roy 2009; Figueira et al. 2013), mainly what their theoretical limitations are and whether they aid the decision-making process.

In addition, as in every other MCDA method, uncertainty is ubiquitous in the ELECTRE decision-making process. According to French (1995), different forms of uncertainty may arise in decision analysis from imprecision, ambiguity or lack of clarity. One form is the uncertainty about the selection of criteria that adequately represent the objectives of the decision problem. Another is the uncertainty surrounding the assignment of criteria weights. There is also uncertainty related to the numerical accuracy of input data. Data uncertainty (i.e. the degree to which data are inaccurate, imprecise or unknown) can be due to many factors, such as inherent variability (from the natural processes that continually affect water resources), measurement errors (caused by equipment or random sampling effects) and boundary conditions (from external factors that cannot be accounted for explicitly) (Klauer et al. 2006). However, as stated by Xu & Tung (2008), MCDA methods are often applied without much consideration given to the uncertainty in the input data and their propagation into the problem solution. As can be expected, data uncertainty may have an important influence on the ranking of alternatives (Eastman et al. 1993), which thus casts significant doubt on the decision analysis results.

Dealing with inaccurate, imprecise, uncertain or ill-determined data is one of the foremost strong features of the ELECTRE family methods (Figueira & Roy 2005). Instead of ‘true criteria’, ELECTRE methods include ‘pseudo-criteria’, with discrimination thresholds, to account for the imperfect knowledge of the data (Figueira et al. 2013). However, fixing the discrimination thresholds for each criterion can be a difficult and ambiguous task for decision-makers, and remains a problematic issue (Govindan & Jepsen 2016). A number of researchers have addressed the need for more comprehensive approaches for selecting appropriate threshold values. Rogers & Bruen (1998) described a methodology for choosing realistic threshold values for use in environmental appraisal systems. The method took into account the effect on human beings of the difference between criterion scores. Hokkanen & Salminen (1997) provided another approach for selecting thresholds in the context of solid waste management systems. It associated thresholds with the possible error range in criteria, which was inferred with the help of regression analyses. On the other hand, Banias et al. (2010) overcame the subjectivity issue by connecting the thresholds to the performance values range (i.e. the difference between the maximum and minimum values), divided by the number of alternatives. The idea behind this was to emphasize the discrimination power of the method: the more alternatives there were, the more necessary it was to have finer thresholds to discriminate among them. This approach, which echoed others in the literature (Haralambopoulos & Polatidis 2003; Polatidis & Morales 2016), provided a simple way for determining the thresholds, but ignored the uncertainty underlying the data. More work needs to be done in order to assist decision-makers in choosing thresholds in a rational and defendable manner.

In this paper, we introduce an extension of the ELECTRE III method to address the issue of fixing discrimination thresholds. We propose a ‘confidence interval-based’ approach, where uncertainty in the input data is defined using confidence intervals and thresholds are expressed as a function of the interval estimates. Our objectives are to: (i) introduce a new approach for threshold determination, which provides a means of reducing the degree of subjectivity; and (ii) test the proposed approach by applying it to a priority ranking of water supply and sewerage services in Spain.

METHODS

ELECTRE III

The ELECTRE III method is based upon developing a preference relation, called the ‘outranking relation’, among alternatives evaluated on several criteria. The outranking relation is defined as a binary relation, S, between two alternatives, a1 and a2, such that a1Sa2 if there are enough arguments to declare that alternative a1 is ‘at least as good as’ alternative a2 (Bouyssou 1996). To build the outranking relation, a series of pairwise comparisons of the alternatives is done using the concordance–discordance principle. It represents, in a sense, the reasons for and against an outranking situation (Roy 1996): a1 outranks a2 if a majority of criteria support this assertion (concordance condition) and if the opposition of the other criteria is not ‘too strong’ (non-discordance condition). The method, in the second phase of outranking relation exploitation, derives two pre-orders: downward, Z1, and upward, Z2. Both pre-orders Z1 and Z2 are constructed through descending and ascending distillation procedures, respectively (for details of these procedures, see Roy 1996). A final pre-order of alternatives is suggested at the intersection of Z1 and Z2. Figure 1 illustrates a summary of the method.

Figure 1

General structure of ELECTRE III method.

Figure 1

General structure of ELECTRE III method.

The construction of the concordance and discordance indexes requires the definition of three discrimination thresholds for each criterion:

  • The indifference threshold, qi, beneath which the decision-maker is indifferent to two alternatives.

  • The preference threshold, pi, above which the decision-maker shows a clear preference for one alternative over the other.

  • The veto threshold, vi, above which the decision-maker negates any possible outranking relationship indicated by the other criteria.

Choosing realistic values for each threshold involves a high degree of subjectivity. In order to facilitate this task for decision-makers, we propose an approach that allows for less subjective input through defining thresholds as a function of the confidence intervals of the alternatives' performances. Hence, we address two concerns that may affect the validity of the rankings: (i) the subjectivity in choosing threshold values and (ii) the imprecision in performance values due to measurement error. The idea behind the approach is explained in Figure 2.

Figure 2

Confidence-interval approach.

Figure 2

Confidence-interval approach.

This way, our approach will provide a different set of q–p–v thresholds for each pair of alternatives and criteria. The equations for the proposed approach are as follows:  
formula
(1)
 
formula
(2)
 
formula
(3)
where Vi(aj) is the performance value of alternative aj for criterion i, and Vi(aj)U and Vi(aj)L the upper and lower limits of its confidence interval.

Case study

We selected a real case study to test the proposed approach. It consisted in a priority ranking of water supply and sewerage services in Spain. The objective was to prioritize the different regions of Spain according to their need for better water supply and sewerage services. This prioritization could be used to support current or future political actions regarding water management in Spain.

The alternatives in the decision problem were the 17 Autonomous Communities of Spain (Andalucía, Aragón, Asturias, Baleares, Canarias, Cantabria, Castilla y León, Castilla-La Mancha, Catalunya, Comunitat Valenciana, Extremadura, Galicia, Madrid, Murcia, Navarra, País Vasco and Rioja). The 11 criteria used to rank the regions consisted of water supply, wastewater, economic and structural factors. A description of each criterion is contained in Table 1.

Table 1

Criteria used in the case study

CriteriaDefinitionUnitsDirection
C1: Volume of drinkable water available Water treated in drinking water treatment plants. Litres/inhabitant/day 
C2: Volume of water supplied to the public network Water entering the distribution network from drinking water treatment plants or service deposits. Includes both registered and non-registered water. Litres/inhabitant/day 
C3: Percentage of water losses Water not registered or distributed to the users. It includes both physical losses (i.e. water leaks, breakages and faults in the distribution network and outlets) and apparent losses (i.e. undercounting, fraud and other non-physical losses). Percentage over total volume − 
C4: Volume of treated wastewater Wastewater treated in treatment plants. All types of treatment are considered (primary, secondary or biological, and tertiary treatments; and soft technologies and septic tanks). m3/inhabitant/day 
C5: Volume of reused wastewater Wastewater reused, including all types of uses (agriculture, industry, watering gardens, leisure sports areas, cleaning of streets and sewage, etc.). m3/inhabitant/day 
C6: Unit cost of water supply Cost charged to users for the full amount of water supplied on the network. It includes both the rates and tariffs paid for water supply. Euros/m3 − 
C7: Unit cost of sewage Cost charged to users for the full amount of wastewater collected and treated. It includes both the municipal sewerage fees and taxes of an ecological nature collected for third parties. Euros/m3 − 
C8: Length of the water supply network Total length of the distribution network. It excludes transmission lines and service pipes. kilometre/inhabitant 
C9: Length of the sewerage network Total length of the sewerage network. It excludes service connections. kilometre/inhabitant 
C10: Volume of water leaked Water leaked due to water pipe breaks in the distribution network. It excludes leaks from active leakage control. m3/kilometre/year − 
C11: Number of storm water tanks Storm water retention tanks included in the sewer system. n° 
CriteriaDefinitionUnitsDirection
C1: Volume of drinkable water available Water treated in drinking water treatment plants. Litres/inhabitant/day 
C2: Volume of water supplied to the public network Water entering the distribution network from drinking water treatment plants or service deposits. Includes both registered and non-registered water. Litres/inhabitant/day 
C3: Percentage of water losses Water not registered or distributed to the users. It includes both physical losses (i.e. water leaks, breakages and faults in the distribution network and outlets) and apparent losses (i.e. undercounting, fraud and other non-physical losses). Percentage over total volume − 
C4: Volume of treated wastewater Wastewater treated in treatment plants. All types of treatment are considered (primary, secondary or biological, and tertiary treatments; and soft technologies and septic tanks). m3/inhabitant/day 
C5: Volume of reused wastewater Wastewater reused, including all types of uses (agriculture, industry, watering gardens, leisure sports areas, cleaning of streets and sewage, etc.). m3/inhabitant/day 
C6: Unit cost of water supply Cost charged to users for the full amount of water supplied on the network. It includes both the rates and tariffs paid for water supply. Euros/m3 − 
C7: Unit cost of sewage Cost charged to users for the full amount of wastewater collected and treated. It includes both the municipal sewerage fees and taxes of an ecological nature collected for third parties. Euros/m3 − 
C8: Length of the water supply network Total length of the distribution network. It excludes transmission lines and service pipes. kilometre/inhabitant 
C9: Length of the sewerage network Total length of the sewerage network. It excludes service connections. kilometre/inhabitant 
C10: Volume of water leaked Water leaked due to water pipe breaks in the distribution network. It excludes leaks from active leakage control. m3/kilometre/year − 
C11: Number of storm water tanks Storm water retention tanks included in the sewer system. n° 

aDirection of the criterion refers to whether it needs to be maximized (+) or minimized (−).

Data on the regions were obtained from the ‘Survey on Water Supply and Sewerage’ done by the Spanish National Institute of Statistics. The survey is framed within the National Statistic Plan 2013–2016 (INE 2014), and aims to provide access to reliable and regular data regarding water management in Spain. The survey consists in a questionnaire on the collection, purchase, sale, supply and distribution of water, as well as collection and treatment of wastewater, by companies or institutions in the same Autonomous Community. The sample for the survey is extracted based on geographical coverage: it covers all municipalities with a population of more than 15,000 inhabitants, which is nearly two-thirds of the Spanish population. The sampling error is estimated to be 5%.

The data for year 2014 are shown in Table 2. These data constituted the performance values for ELECTRE III (note: we considered that all criteria had the same importance, and thus the same weight coefficients). The application of the mathematical model was undertaken with the use of R software (v3.3.1).

Table 2

Criteria performance values for the Autonomous Communities, with their confidence interval

CriteriaC1C2C3C4C5C6C7C8C9C10C11
Regions 
 A1: Andalucía 282 ± 14 253 ± 13 19.6 ± 0.98 0.239 ± 0.012 0.019 ± 0.001 1.06 ± 0.05 0.75 ± 0.04 5.5 ± 0.28 3.8 ± 0.19 3,281 ± 164 8 ± 0.4 
 A2: Aragón 332 ± 17 281 ± 14 19.9 ± 1.00 0.416 ± 0.021 0.003 ± 0.000 0.69 ± 0.03 0.76 ± 0.04 3.9 ± 0.20 3.3 ± 0.17 5,170 ± 259 12 ± 0.6 
 A3: Asturias 428 ± 21 297 ± 15 17.4 ± 0.87 0.524 ± 0.026 0.036 ± 0.002 0.6 ± 0.03 0.72 ± 0.04 8.1 ± 0.41 4.9 ± 0.25 2,317 ± 116 0 ± 0.0 
 A4: Baleares 284 ± 14 272 ± 14 16.7 ± 0.84 0.299 ± 0.015 0.136 ± 0.007 1.08 ± 0.05 1.11 ± 0.06 3.7 ± 0.19 3.3 ± 0.17 4,527 ± 226 0 ± 0.0 
 A5: Canarias 327 ± 16 264 ± 13 20.3 ± 1.02 0.181 ± 0.009 0.036 ± 0.002 1.72 ± 0.09 0.37 ± 0.02 7.4 ± 0.37 2.6 ± 0.13 2,650 ± 133 2 ± 0.1 
 A6: Cantabria 373 ± 19 347 ± 17 25.1 ± 1.26 0.455 ± 0.023 0.009 ± 0.000 1 ± 0.05 0.75 ± 0.04 6.7 ± 0.34 4.2 ± 0.21 4,761 ± 238 62 ± 3.1 
 A7: Castilla y León 418 ± 21 329 ± 16 16.5 ± 0.83 0.431 ± 0.022 0.004 ± 0.000 0.54 ± 0.03 0.41 ± 0.02 6.6 ± 0.33 4.3 ± 0.22 3,000 ± 150 21 ± 1.1 
 A8: Castilla-La Mancha 318 ± 16 265 ± 13 19 ± 0.95 0.255 ± 0.013 0.007 ± 0.000 0.82 ± 0.04 0.46 ± 0.02 6.7 ± 0.34 3.9 ± 0.20 2,738 ± 137 6 ± 0.3 
 A9: Catalunya 263 ± 13 219 ± 11 11.2 ± 0.56 0.233 ± 0.012 0.009 ± 0.000 1.41 ± 0.07 1.34 ± 0.07 5.4 ± 0.27 1.9 ± 0.10 1,669 ± 83 16 ± 0.8 
 A10: Comunitat Valenciana 279 ± 14 271 ± 14 15.8 ± 0.79 0.232 ± 0.012 0.138 ± 0.007 1.21 ± 0.06 0.86 ± 0.04 7.6 ± 0.38 2.9 ± 0.15 2,043 ± 102 9 ± 0.5 
 A11: Extremadura 310 ± 16 262 ± 13 24 ± 1.20 0.406 ± 0.020 0 ± 0.000 1 ± 0.05 0.52 ± 0.03 6.4 ± 0.32 3 ± 0.15 3,594 ± 180 2 ± 0.1 
 A12: Galicia 304 ± 15 243 ± 12 16.4 ± 0.82 0.33 ± 0.017 0 ± 0.000 0.67 ± 0.03 0.44 ± 0.02 5.8 ± 0.29 4.9 ± 0.25 2,504 ± 125 54 ± 2.7 
 A13: Madrid 220 ± 11 217 ± 11 4.6 ± 0.23 0.264 ± 0.013 0.006 ± 0.000 1.31 ± 0.07 0.77 ± 0.04 2.8 ± 0.14 2.2 ± 0.11 1,295 ± 65 63 ± 3.2 
 A14: Murcia 235 ± 12 235 ± 12 13.5 ± 0.68 0.249 ± 0.012 0.125 ± 0.006 1.84 ± 0.09 0.89 ± 0.04 7.5 ± 0.38 4.1 ± 0.21 1,535 ± 77 10 ± 0.5 
 A15: Navarra 307 ± 15 261 ± 13 17.6 ± 0.88 0.34 ± 0.017 0 ± 0.000 0.74 ± 0.04 0.67 ± 0.03 4.8 ± 0.24 5.2 ± 0.26 3,470 ± 174 21 ± 1.1 
 A16: País Vasco 265 ± 13 234 ± 12 8.9 ± 0.45 0.539 ± 0.027 0.008 ± 0.000 0.84 ± 0.04 0.91 ± 0.05 5.6 ± 0.28 2.1 ± 0.11 1,350 ± 68 28 ± 1.4 
 A17: Rioja 308 ± 15.4 299 ± 15.0 14 ± 0.700 0.471 ± 0.024 0 ± 0.000 0.55 ± 0.028 0.6 ± 0.030 3.4 ± 0.17 3 ± 0.15 4,539 ± 227 0 ± 0.0 
CriteriaC1C2C3C4C5C6C7C8C9C10C11
Regions 
 A1: Andalucía 282 ± 14 253 ± 13 19.6 ± 0.98 0.239 ± 0.012 0.019 ± 0.001 1.06 ± 0.05 0.75 ± 0.04 5.5 ± 0.28 3.8 ± 0.19 3,281 ± 164 8 ± 0.4 
 A2: Aragón 332 ± 17 281 ± 14 19.9 ± 1.00 0.416 ± 0.021 0.003 ± 0.000 0.69 ± 0.03 0.76 ± 0.04 3.9 ± 0.20 3.3 ± 0.17 5,170 ± 259 12 ± 0.6 
 A3: Asturias 428 ± 21 297 ± 15 17.4 ± 0.87 0.524 ± 0.026 0.036 ± 0.002 0.6 ± 0.03 0.72 ± 0.04 8.1 ± 0.41 4.9 ± 0.25 2,317 ± 116 0 ± 0.0 
 A4: Baleares 284 ± 14 272 ± 14 16.7 ± 0.84 0.299 ± 0.015 0.136 ± 0.007 1.08 ± 0.05 1.11 ± 0.06 3.7 ± 0.19 3.3 ± 0.17 4,527 ± 226 0 ± 0.0 
 A5: Canarias 327 ± 16 264 ± 13 20.3 ± 1.02 0.181 ± 0.009 0.036 ± 0.002 1.72 ± 0.09 0.37 ± 0.02 7.4 ± 0.37 2.6 ± 0.13 2,650 ± 133 2 ± 0.1 
 A6: Cantabria 373 ± 19 347 ± 17 25.1 ± 1.26 0.455 ± 0.023 0.009 ± 0.000 1 ± 0.05 0.75 ± 0.04 6.7 ± 0.34 4.2 ± 0.21 4,761 ± 238 62 ± 3.1 
 A7: Castilla y León 418 ± 21 329 ± 16 16.5 ± 0.83 0.431 ± 0.022 0.004 ± 0.000 0.54 ± 0.03 0.41 ± 0.02 6.6 ± 0.33 4.3 ± 0.22 3,000 ± 150 21 ± 1.1 
 A8: Castilla-La Mancha 318 ± 16 265 ± 13 19 ± 0.95 0.255 ± 0.013 0.007 ± 0.000 0.82 ± 0.04 0.46 ± 0.02 6.7 ± 0.34 3.9 ± 0.20 2,738 ± 137 6 ± 0.3 
 A9: Catalunya 263 ± 13 219 ± 11 11.2 ± 0.56 0.233 ± 0.012 0.009 ± 0.000 1.41 ± 0.07 1.34 ± 0.07 5.4 ± 0.27 1.9 ± 0.10 1,669 ± 83 16 ± 0.8 
 A10: Comunitat Valenciana 279 ± 14 271 ± 14 15.8 ± 0.79 0.232 ± 0.012 0.138 ± 0.007 1.21 ± 0.06 0.86 ± 0.04 7.6 ± 0.38 2.9 ± 0.15 2,043 ± 102 9 ± 0.5 
 A11: Extremadura 310 ± 16 262 ± 13 24 ± 1.20 0.406 ± 0.020 0 ± 0.000 1 ± 0.05 0.52 ± 0.03 6.4 ± 0.32 3 ± 0.15 3,594 ± 180 2 ± 0.1 
 A12: Galicia 304 ± 15 243 ± 12 16.4 ± 0.82 0.33 ± 0.017 0 ± 0.000 0.67 ± 0.03 0.44 ± 0.02 5.8 ± 0.29 4.9 ± 0.25 2,504 ± 125 54 ± 2.7 
 A13: Madrid 220 ± 11 217 ± 11 4.6 ± 0.23 0.264 ± 0.013 0.006 ± 0.000 1.31 ± 0.07 0.77 ± 0.04 2.8 ± 0.14 2.2 ± 0.11 1,295 ± 65 63 ± 3.2 
 A14: Murcia 235 ± 12 235 ± 12 13.5 ± 0.68 0.249 ± 0.012 0.125 ± 0.006 1.84 ± 0.09 0.89 ± 0.04 7.5 ± 0.38 4.1 ± 0.21 1,535 ± 77 10 ± 0.5 
 A15: Navarra 307 ± 15 261 ± 13 17.6 ± 0.88 0.34 ± 0.017 0 ± 0.000 0.74 ± 0.04 0.67 ± 0.03 4.8 ± 0.24 5.2 ± 0.26 3,470 ± 174 21 ± 1.1 
 A16: País Vasco 265 ± 13 234 ± 12 8.9 ± 0.45 0.539 ± 0.027 0.008 ± 0.000 0.84 ± 0.04 0.91 ± 0.05 5.6 ± 0.28 2.1 ± 0.11 1,350 ± 68 28 ± 1.4 
 A17: Rioja 308 ± 15.4 299 ± 15.0 14 ± 0.700 0.471 ± 0.024 0 ± 0.000 0.55 ± 0.028 0.6 ± 0.030 3.4 ± 0.17 3 ± 0.15 4,539 ± 227 0 ± 0.0 

RESULTS AND DISCUSSION

Discrimination thresholds

The discrimination thresholds are introduced to allow the correct interpretation of the differences between the alternatives' performances. One way for giving numerical values to such thresholds would be by coming back to their definition and analyzing the main sources of imprecision and uncertainty (Roy 1991). Thus, in this context of water supply and sewerage services, we can value the thresholds as follows:

  • C1: volume of drinking water available per inhabitant and day. The variation in volume between regions was 208 l/inhab/d. In light of this variation, a difference of 100 l/inhab/d was not considered convincing evidence, while a difference of 200 l/inhab/d or more was taken to imply strict preference.

  • C2: volume of drinking water supplied to the network per inhabitant and day. The variation in volume was 130 l/inhab/d. We assumed that indifference remained up to 50 l/inhab/d and strict preference started from 100 l/inhab/d.

  • C3: percentage of water losses. In Spain, the mean values for losses were 16.5%. We thus considered that a difference of 15% was not an indication for preference, while a difference of 25% showed strict preference.

  • C4: volume of treated wastewater per inhabitant per day. The variation in volume was 0.358 m3/inhab/d, so we selected 0.15 and 0.25 m3/inhab/d as an indication for indifference and strict preference, respectively.

  • C5: volume of wastewater reused per inhabitant per day. The variation in volume was 0.138 m3/inhab/d. We assumed that differences below 0.05 m3/inhab/d were not evidence for preference, while differences above 0.15 m3/inhab/d showed strict preference.

  • C6: unit cost of water supply. The mean value for the cost of water was 1.005 EUR/m3. We considered that indifference remained under 1 EUR/m3 and strict preference began from 2 EUR/m3.

  • C7: unit cost of sewage. In this case, the mean value for the cost was 0.725 EUR/m3, so we fixed the indifference and preference levels as 0.75 and 1.5 EUR/m3, respectively.

  • C8: length of the water network per inhabitant. The length of the water network ranged from 2.8 km/inhab in Madrid to 8.1 km/inhab in Asturias. A difference of 2.5 km/inhab was not seen as convincing evidence, while a difference of 5 km/inhab was seen to imply strict preference.

  • C9: length of the sewerage network per inhabitant. The length of the water network ranged from 1.9 km/inhab in Catalunya to 4.9 km/inhab in Asturias and Galicia. We considered that differences in length below 1.5 km/inhab were not significant, but differences above 3 km/inhab were a sign of strict preference.

  • C10: volume of water leaked per kilometre and year. The variation in volume was 3,875 m3/km/y, so we chose 2,000 and 3,500 m3/km/y as levels of indifference and strict preference, respectively.

  • C11: number of storm tanks. The number of storm tanks ranged from 0 in various regions (Asturias, Baleares and Rioja) to 63 in Madrid. We decided that differences in the number below 20 were not indicative of preference, while differences above 40 were a sign of strict preference.

The veto values for all 11 criteria were determined in reference to the value of the preference threshold. As Roy et al. (1986) pointed out, unless there are good reasons for adopting another choice, the ratio v/p can be fixed as a constant for each criterion. We selected a ratio of 2, as shown in Table 3.

Table 3

Thresholds for criteria (obtained based on our subjective input)

ThresholdsC1C2C3C4C5C6C7C8C9C10C11
Indifference (q100 50 15 0.15 0.05 0.75 2.5 1.5 2,000 20 
Preference (p200 100 25 0.25 0.15 1.5 3,500 40 
Veto (v400 200 50 0.5 0.3 10 7,000 80 
ThresholdsC1C2C3C4C5C6C7C8C9C10C11
Indifference (q100 50 15 0.15 0.05 0.75 2.5 1.5 2,000 20 
Preference (p200 100 25 0.25 0.15 1.5 3,500 40 
Veto (v400 200 50 0.5 0.3 10 7,000 80 

As seen, fixing the thresholds involved a significant subjective input by us. Although we did not pick threshold values in an arbitrary manner but by examining the data, a certain amount of arbitrariness was inevitable. Roy et al. (1986) emphasized the need for a sensitivity analysis, using extreme values of q–p–v, to verify that this subjective input did not significantly affect the final ranking of alternatives.

In the approach we propose, we attempt to reduce the degree of subjectivity when choosing the thresholds by expressing them in terms of the confidence intervals of the performance values (see Equations (1)–(3)). This approach can be interesting in some cases, when working with statistical data. Let us remember that the indifference threshold describes the largest difference between the performance values so that the decision-maker is indifferent between two alternatives, while the preference threshold describes the largest difference that makes him prefer one over the other. Consequently, it is reasonable to say that two alternatives could be considered indifferent if their confidence intervals overlap; otherwise, one would be preferred over the other. The veto threshold, on the other hand, is not associated with the sources of imprecision and uncertainty, but with a base principle of the outranking relation: the discordance concept. However, as explained by Roy et al. (1986), the size of the veto threshold is generally fixed in terms of the preference thresholds (i.e. v/p ratio). That is why we computed the veto thresholds as twice the preference values.

We would like to emphasize that this approach is not designed to ‘estimate’ the value of the discrimination thresholds. These thresholds are not experimental values to be estimated, but rather values used to model the decision-maker's preferences. Our confidence interval approach only aims to assist decision-makers in selecting numerical values for thresholds in specific cases, but should never replace the preferences of actors in the decision process.

Ranking of regions

After the determination of the discrimination thresholds (either with our subjective input or using the confidence interval approach), the mathematical model for the ranking is resolved. Two complete pre-orders are first constructed, through descending and ascending distillation procedures. The descending distillation ranks the alternatives from the best available to the worst, while the ascending does it in the reverse manner. Figure 3 presents both pre-orders in graphs where the axes are the positions of the Autonomous Communities.

Figure 3

Ascending and descending distillation results for ELECTRE III A (thresholds from Table 3) and ELECTRE III B (thresholds from Equations (1)–(3)).

Figure 3

Ascending and descending distillation results for ELECTRE III A (thresholds from Table 3) and ELECTRE III B (thresholds from Equations (1)–(3)).

Distillations with the first set of thresholds (those fixed with our subjective input) show Catalunya (A9) as the region most in need of better water supply and sanitation services, followed by Aragón, Extremadura, Madrid and Navarra (A2, A11, A13 and A15). This can be interpreted as a result of the bad performances of Catalunya in the majority of evaluation criteria (C1, C2, C3, C4, C6, C7 and C9). The outcome from distillations with the confidence interval approach is, however, different. Whereas Aragón, Extremadura and Navarra remained at the bottom of the ranking, Catalunya and Madrid occupied a higher rank. This is a consequence of the uncertainty in the performance values. As seen in Figure 4, although Catalunya (A9) occupied the bottom ranks in almost all criteria, the confidence intervals for its performance values overlapped with other regions. Our approach considers two alternatives to be indifferent if their confidence intervals overlap. That is why it resulted in a different ranking of regions.

Figure 4

Performance values Vi(aj) with confidence intervals Vi(aj)L-Vi(aj)U for each criterion j (Note: regions are ordered according to their performance values).

Figure 4

Performance values Vi(aj) with confidence intervals Vi(aj)L-Vi(aj)U for each criterion j (Note: regions are ordered according to their performance values).

It is important to draw attention to the fact that both rankings are equally relevant and valid. It would be wrong to say that one ranking is good or bad only by referring to a mathematical model. As Roy (2005) stated when explaining the purpose of MCDA, these models should not be viewed as being conceived to discover a pre-existing truth. It is not possible to know which is the ‘right’ ranking and which is not, because the ‘right’ ranking does not exist. Decision-aiding based on MCDA models is only meant to guide the decision-making process.

In the same way, discrimination thresholds are not ‘real values’ that exist somewhere. They are merely numbers designed to reflect a system of preferences. Consequently, there should always be room for a substantial degree of subjectivity/flexibility in their determination (Roy et al. 1986). Our confidence interval approach may be interesting in some cases (e.g. when dealing with statistical data), but only to guide the decision-maker in this inevitably arbitrary process. Robustness analyses will still be needed to assess the extent of the influence of this arbitrariness on the final results, as well as to better define the choice of numerical values in view of this effect.

CONCLUSIONS

ELECTRE outranking methods are one of the best known and widely applied in the context of decision aid. The output of ELECTRE depends critically on the input information, hence the data input should ideally be precise. Yet, in reality, the available data are often uncertain. Discrimination thresholds (indifference, preference and veto) were incorporated in ELECTRE methods to take into account the imperfect knowledge of the data. Fixing these thresholds for each criterion can be, however, a difficult and ambiguous task for analysts and decision-makers, as it involves a substantial element of subjectivity.

We propose an approach that allows for less subjective input in the determination of thresholds. This is achieved by characterizing the uncertainty in the performance values by defining the confidence intervals of the available data, and expressing the discrimination thresholds as a function of these interval estimates. Ranking of alternatives is therefore provided to the decision-maker without his subjective input. The illustration of the proposed approach using the water and sewerage case study demonstrates how uncertainty in the data can be used to define the discrimination thresholds. It also highlights the significant difference in rankings when thresholds were set with and without our subjective input.

However, the confidence interval approach should not be viewed as ‘better’ than basing the thresholds on our judgments. Thresholds are not experimental values that need to be estimated, but rather values that we use to model our, or the decision-maker's, preferences. The only aim of the proposed approach is to guide him in some cases, with specific data: statistical data.

REFERENCES

REFERENCES
Banias
G.
,
Achillas
C.
,
Vlachokostas
C.
,
Moussiopoulos
N.
&
Tarsenis
S.
2010
Assessing multiple criteria for the optimal location of a construction and demolition waste management facility
.
Building and Environment
45
(
10
),
2317
2326
.
Bella
A.
,
Duckstein
L.
&
Szidarovszky
F.
1996
A multicriterion analysis of the water allocation conflict in the Upper Rio Grande basin
.
Applied Mathematics and Computation
77
(
2–3
),
245
265
.
Bouyssou
D.
1996
Outranking relations: do they have special properties?
Journal of Multi-Criteria Decision Analysis
5
(
2
),
99
111
.
Carriço
N.
,
Covas
D. I. C.
,
Almeida
M. C.
,
Leitão
J. P.
&
Alegre
H.
2012
Prioritization of rehabilitation interventions for urban water assets using multiple criteria decision-aid methods
.
Water Science and Technology
66
(
5
),
1007
1014
.
Ceccato
L.
,
Giannini
V.
&
Giupponi
C.
2011
Participatory assessment of adaptation strategies to flood risk in the Upper Brahmaputra and Danube river basins
.
Environmental Science & Policy
14
(
8
),
1163
1174
.
Duckstein
L.
,
Gershon
M.
&
McAniff
R.
1982
Model selection in multiobjective decision making for river basin planning
.
Advances in Water Resources
5
(
3
),
178
184
.
Eastman
J. R.
,
Kyem
P. A. K.
,
Toledano
J.
&
Jin
W.
1993
GIS and Decision Making
.
United Nations Institute for Training and Research (UNITAR)
,
Geneva, Switzerland
.
Figueira
J.
,
Roy
B.
2005
ELECTRE methods
. In:
Multiple Criteria Decision Analysis: State of the Art Surveys
(
Figueira
J.
,
Greco
S.
&
Ehrogott
M.
, eds),
Springer New York
,
New York, NY, USA
, pp.
134
162
.
Figueira
J. R.
,
Greco
S.
,
Roy
B.
&
Słowiński
R.
2013
An overview of ELECTRE methods and their recent extensions
.
Journal of Multi-Criteria Decision Analysis
20
(
1–2
),
61
85
.
Flug
M.
,
Seitz
H. L. H.
&
Scott
J. F.
2000
Multicriteria decision analysis applied to Glen Canyon Dam
.
Journal of Water Resources Planning and Management
126
(
5
),
270
276
.
French
S.
1995
Uncertainty and imprecision: modelling and analysis
.
The Journal of the Operational Research Society
46
(
1
),
70
79
.
Govindan
K.
&
Jepsen
M. B.
2016
ELECTRE: A comprehensive literature review on methodologies and applications
.
European Journal of Operational Research
250
(
1
),
1
29
.
Haralambopoulos
D. A.
&
Polatidis
H.
2003
Renewable energy projects: structuring a multi-criteria group decision-making framework
.
Renewable Energy
28
(
6
),
961
973
.
Hokkanen
J.
&
Salminen
P.
1997
Choosing a solid waste management system using multicriteria decision analysis
.
European Journal of Operational Research
98
(
1
),
19
36
.
INE
2014
National Statistics Institute: Survey on Water Supply and Sewerage
.
(accessed 15 May 2018)
.
Ishizaka
A.
&
Nemery
P.
2013
Multi-criteria Decision Analysis: Methods and Software
.
John Wiley & Sons, Ltd
,
Chichester, UK
.
Klauer
B.
,
Drechsler
M.
&
Messner
F.
2006
Multicriteria analysis under uncertainty with IANUS – method and empirical results
.
Environment and Planning C: Government and Policy
24
(
2
),
235
256
.
Ko
S. K.
,
Fontane
D. G.
&
Margeta
J.
1994
Multiple reservoir system operational planning using multi-criterion decision analysis
.
European Journal of Operational Research
76
(
3
),
428
439
.
Malekmohammadi
B.
,
Zahraie
B.
&
Kerachian
R.
2011
Ranking solutions of multi-objective reservoir operation optimization models using multi-criteria decision analysis
.
Expert Systems with Applications
38
(
6
),
7851
7863
.
Morais
D. C.
&
Almeida
A. T.
2006
Water supply system decision making using multicriteria analysis
.
Water SA
32
(
2
),
229
235
.
Mutikanga
H. E.
,
Sharma
S. K.
&
Vairavamoorthy
K.
2011
Multi-criteria decision analysis: a strategic planning tool for water loss management
.
Water Resources Management
25
(
14
),
3947
3969
.
Pedras
C. M. G.
&
Pereira
L. S.
2009
Multicriteria analysis for design of microirrigation systems. Application and sensitivity analysis
.
Agricultural Water Management
96
(
4
),
702
710
.
Polatidis
H.
&
Morales
J. B.
2016
Increasing the applicability of wind power projects via a multi-criteria approach: methodology and case study
.
International Journal of Sustainable Energy
35
(
10
),
1014
1029
.
Raj
P. A.
1995
Multicriteria methods in river basin planning – a case study
.
Water Science and Technology
31
(
8
),
261
272
.
Raju
K. S.
,
Duckstein
L.
&
Arondel
C.
2000
Multicriterion analysis for sustainable water resources planning: a case study in Spain
.
Water Resources Management
14
(
6
),
435
456
.
Roy
B.
2005
Paradigms and challenges
. In:
Multiple Criteria Decision Analysis: State of the Art Surveys
(J. Figueira, S. Greco & M. Ehrgott, eds)
,
Springer
,
New York, NY, USA
,
pp. 3–24
.
Roy
B.
1996
Multicriteria Methodology for Decision Aiding
.
Kluwer, Dordecht
,
The Netherlands
.
Roy
B.
,
Présent
M.
&
Silhol
D.
1986
A programming method for determining which Paris metro stations should be renovated
.
European Journal of Operational Research
24
(
2
),
318
334
.
Roy
B.
,
Slowinski
R.
&
Treichel
W.
1992
Multicriteria programming of water supply systems for rural areas
.
Water Resources Bulletin
28
(
1
),
13
31
.
Tecle
A.
,
Fogel
M. M.
&
Duckstein
L.
1988
Multicriterion analysis of forest watershed management alternatives
.
Water Resources Bulletin
24
(
6
),
1169
1178
.
Xu
Y. P.
&
Tung
Y. K.
2008
Decision-making in water management under uncertainty
.
Water Resources Management
22
(
5
),
535
550
.
Zardari
N. H.
,
Cordery
I.
&
Sharma
A.
2010
An objective multiattribute analysis approach for allocation of scarce irrigation water resources
.
Journal of the American Water Resources Association
46
(
2
),
412
428
.