The current worldwide water resources issue is one of the crucial matters to overcome obstacles to sustainable development. This problem, formerly tackled in a sectored manner, is now pointing towards an analysis directed to treating the watershed as a management unit, with regards to all dimensions of knowledge and, especially, to the public participation in the decision-making processes. As an alternative to measure its performance, it has been sought out to develop indexes aimed to measure its sustainability, but there is still a lack of the use of composed efficient methodologies that also enable public participation in decision-making. This research presents a methodology comprising 15 indexes for the calculation of the Watershed Sustainability Index (WSI), followed by the application of the PROMETHEE multi-criteria analysis method and the COPELAND multi-decision-maker method. The methodology was applied to evaluate the performance of subwatersheds of the Piranhas-Açu watershed, located in the Brazilian northeast semi-arid region. The performance ordering, obtained through the application of the methods, emphasizes that subwatersheds' performances are uneven. It can be noticed that the subwatersheds' performances are still far from ideal in relation to water resources management, even in the ones that displayed satisfactory index levels.

Nowadays, the issue of sustainability has expanded in discussions around the world and when it comes to the area of water resources this discussion is growing towards finding its best use performance through various analyses (Gleick, 1998; Loucks, 2000; Postel, 2000; Tundisi, 2003; Vieira, 2003; Mays, 2007). According to Gleick (1998), water is not only essential to sustain life, but it also plays a fundamental role in the support of ecosystems, in economic development, in community well-being and in cultural values. A starting point to seek an understanding of this relationship is upon the ‘own’ definition of water sustainability, and, in this sense, some authors have developed definitions that contemplate this connection (Gleick et al., 1995; Loucks, 2000; Mays, 2007). To Mays (2007), water sustainability is the ‘ability to provide and manage water in terms of quantity and quality in order to meet current human and environmental needs, while not impeding the next generations to do the same’. Gleick et al. (1995) conceptualize it as ‘the use of water for support and the ability of the human society to strengthen and flourish for an indefinite future without harming the integrity of the water cycle or of the ecologic system that depends on it’. In turn, Veira (2003) defines water sustainability as ‘the continuous and consistent fulfilment of society's demands through a guaranteed water supply, in quantity and quality’. Loucks (2000) defines water sustainability as ‘the designation and planning of water resources to fully contribute to society's goals now and in the future while maintaining their ecological, environmental and hydrological integrity’. It seems that Loucks' definition is the more adequate when thinking in terms of indexes since it contemplates the necessity of planning and management based on the ‘goals’ of today's society as well as aiming towards its future. His definition approximates current methodologies that focus on measuring sustainability from a perspective that contemplates the participation of society in planning. This brings the idea of the identification of a set of indexes to perform the required analysis of water resources sustainability and, in this sense, it is a more adequate definition than those who wish to focus on water resources management (Chaves & Alipaz, 2007).

In terms of planning, in recent times there has been progress in the development of indexes that specifically cover water resources related performance analysis (Brown & Matlock, 2011), including the Basic Human Needs Index, the Imported Water and Cereal Availability Index, Indexes of Local Relative Use and Reuse of Water, the Watershed Sustainability Index (WSI), the Water Stress Supply Index and the Physical and Economic Water Shortage Index. Such indexes seek to perform water resources analysis in order to contribute to watershed planning. Amid such indexes, the WSI has the advantage of encompassing water resources sustainability (Loucks, 2000). The application of the WSI, however, has had some limitations when it comes to public participation (Juwana et al., 2012) and, therefore, it lacks methodologies that complement it.

To compare water resources performance analysis, Carvalho & Curi (2013) opted for using the PROMETHEE II (Preference Ranking Organization Method for Enrichment Evaluation) method, since it has presented advantages over other methodologies (Brans et al., 1986; Pomerol & Barba-Romero, 2012; Velasquez & Hester, 2013). The PROMETHEE II method can be applied to incorporate preferences of different decision-makers, especially in scenarios where there are several decision-makers, such as the Brazilian case of Watershed Committees. A method aimed at aggregating the ranking selections of decision-makers is the Copeland method, which has shown unique advantages over the Edge and Condorcet methods (Pomerol & Barba-Romero, 2012).

The aim of the present study is to propose the application of the WSI (Chaves & Alipaz, 2007) in the Piranhas-Açu watershed, located in the Brazilian semi-arid region. The indexes of the WSI are used in a multicriteria analysis using the PROMETHEE methodology, applied together with the multi-decision-maker ranking aggregation methodology (COPELAND), with a view to contemplating public participation in water resources analysis process, leading the WSI to a breakthrough in methodological terms. This sub-basin, as presented by Amorim et al. (2016), is among the most problematic in the Brazilian semi-arid region, considered by some to be one among the most problematic in the world (Cirilo, 2008), and, therefore, is an opportune scenario for the analysis intended here.

Indexes aimed at water resources

Brown & Matlock (2011) stated that over the last few years there has been advancement in the elaboration of indexes aimed at measuring aspects related to water resources performance analysis and, in a recent review, presented some of the most internationally well-known indexes related to water1. In order to do so, researchers elaborated characterization of the various indexes highlighting categories in which such indexes are classified. A first category of indexes is aimed at basic human needs. In this category the authors have placed the Falkenmark Index, the Basic Human Needs Index, and Water Resources Availability and Cereal Import Index. In the second category are the indexes aimed at vulnerability of water resources where the Index of Local Relative Water Use and Reuse, the Watershed Sustainability Index (WSI), the Water Supply Stress Index and the Physical and Economical Water Scarcity Index were placed. A third category, in turn, consists of indexes aimed at incorporating environmental water requirements. In this category we highlight the Population Growth Impacts Water Resource Availability Index and the Index of Assessing Water Resource Supplies Using the Water Stress Index. Finally, the authors establish their own Water Footprinting category, highlighting the various aspects that are part of the index, including the analysis of social and environmental impacts.

In this sense, in recent times, the United Nations2 (UN) has promoted the creation of an index focused on the sustainability analysis of water resources from the watershed, known as the Watershed Sustainability Index (WSI), which has covered a wide field of application and from that, international boundaries have been embraced so that the application of the index has contributed to broader analyses around the world. In terms of Brazil, the WSI has recently been applied in the southeast and midwest regions, however, the major problems related to water resources are situated in the northeast, where the world's most problematic semi-arid region (Cirilo, 2008) is located, and where there still has been no progress in its application. Despite the many advantages of the WSI3, Juwana et al. (2012) in a recent study comparing the most well-known water sustainability indexes in the world, highlighted some limitations of the WSI regarding public participation, especially in the composition of weights.

Thinking about public participation has been the hallmark of current water resource management systems, marked by management models that aim to contemplate the complexity through a participative process in construction. According to Cirilo (2008), Brazil, following the worldwide adopted logic, currently has a decentralized management system in which public participation, exerted through the Watershed Committee, has a singular participation in the efficacy of the process.

Multi-criteria PROMETHEE methodology of support to decision-making

Seeking an effective decision-making process in watershed management, several methods have pointed to the use of multi-criteria decision support analysis and in the environmental area the methods of the PROMETHEE family (preference ranking organization method for enrichment evaluation) have stood out (Brans et al., 1986; Pomerol & Barba-Romero, 2012; Velasquez & Hester, 2013).

In a comparative study with many current methods, Balali et al. (2014) concluded that PROMETHEE II is the favourite method aimed at decision-making support due to: (a) the method is consistent and easy to understand; (b) it does not require too much interaction with the decision-makers; (c) the linearity and the additive hypotheses for the preference function are acceptable for the decision-makers. To the authors, there are unique characteristics of PROMETHEE that are not available on AHP or on other famous multi-criteria decision-making support methods such as ELECTRE and TOPSIS (Balali et al., 2014).

Velasquez & Hester (2013) presented a comprehensive analysis of various methodologies aimed at the decision-making, highlighting that, among the many methods, when it comes to applications that involve the environment, PROMETHEE has been the most chosen method and therefore the best suited for the application. Carvalho & Curi (2013) agreed with this point of view, stating that PROMETHEE is the most adequate method for studies aimed at water resources, arguing that: (a) it is a non-compensatory method; (b) it is a method that favours well-balanced alternatives, that is, seeks to establish a complete pre-order; (c) allows the use of various preference functions, best suited to questions that are objective and subjective. It is interesting to note here that this flexibility is mainly about the decision-maker's ability to establish preference functions. Considering possible application sites such as developing countries, the ease of use may be a defining factor to the use or not of a given method. Table 1 presents a comparison between various multi-criteria methods developed by Velasquez & Hester (2013).

Table 1.

Analysis of methods aimed at decision-making support (adapted from Velasquez & Hester, 2013).

MethodBenefitsApplication areas
Multi-attribute utility theory (MAUT) Takes into account uncertainty; can incorporate preferences Economy, finance, actuarial, water, management, energy management, agriculture 
Analytic hierarchy process (AHP) Easy to use; scalable; the hierarchical structure can easily be adjusted to fit many size issues; non-data intensive Performance type issues, resource management, corporate and strategic policy, public policy, political strategy, planning 
Case-based reasoning Non-data intensive; requires little maintenance; may improve over time; can adapt to changes in the environment Business, vehicle insurance, medicine and engineering design 
Data envelopment analysis (DEA) Able to handle multiple inputs and outputs; efficiency can be analyzed and quantified Economics, medicine, utilities, road safety, agriculture, retail and business issues 
Diffuse set theory Allows inaccurate input; takes into account insufficient training Engineering, economics, management, environment, sociology and medicine 
Simple multi-attribute rating technique (SMART) Simple; allows any type of weighting technique; less effort by decision-makers Environment, construction, transportation and logistics, military, manufacturing and assembly issues 
Goal programming (GP) Able to deal with large-scale problems; can produce endless alternatives Production planning, scheduling, health care, portfolio selection, distribution systems, energy planning, water reservoir management, wildlife management 
ELECTRE It takes into account uncertainty and inaccuracy Energy, economy, environment, water management and transportation issues 
PROMETHEE Easy to use; requires no assumption that the criteria are proportionate Environment, hydrology, water management, business and finance, chemistry, logistics and transportation, manufacturing and assembly, energy, agriculture 
Simple additive weighting (SAW) Ability to offset criteria; intuitive for manufacturers' decision; the calculation is simple, does not require complex computer programs Water resources management, business and financial management 
Technique for order preference by similarity to ideal solution It has a simple process; easy to use and program; the number of steps remains the same regardless of the number of attributes Supply chain and logistics management, engineering, manufacturing systems, business and marketing, environment, human resources and water resources management 
MethodBenefitsApplication areas
Multi-attribute utility theory (MAUT) Takes into account uncertainty; can incorporate preferences Economy, finance, actuarial, water, management, energy management, agriculture 
Analytic hierarchy process (AHP) Easy to use; scalable; the hierarchical structure can easily be adjusted to fit many size issues; non-data intensive Performance type issues, resource management, corporate and strategic policy, public policy, political strategy, planning 
Case-based reasoning Non-data intensive; requires little maintenance; may improve over time; can adapt to changes in the environment Business, vehicle insurance, medicine and engineering design 
Data envelopment analysis (DEA) Able to handle multiple inputs and outputs; efficiency can be analyzed and quantified Economics, medicine, utilities, road safety, agriculture, retail and business issues 
Diffuse set theory Allows inaccurate input; takes into account insufficient training Engineering, economics, management, environment, sociology and medicine 
Simple multi-attribute rating technique (SMART) Simple; allows any type of weighting technique; less effort by decision-makers Environment, construction, transportation and logistics, military, manufacturing and assembly issues 
Goal programming (GP) Able to deal with large-scale problems; can produce endless alternatives Production planning, scheduling, health care, portfolio selection, distribution systems, energy planning, water reservoir management, wildlife management 
ELECTRE It takes into account uncertainty and inaccuracy Energy, economy, environment, water management and transportation issues 
PROMETHEE Easy to use; requires no assumption that the criteria are proportionate Environment, hydrology, water management, business and finance, chemistry, logistics and transportation, manufacturing and assembly, energy, agriculture 
Simple additive weighting (SAW) Ability to offset criteria; intuitive for manufacturers' decision; the calculation is simple, does not require complex computer programs Water resources management, business and financial management 
Technique for order preference by similarity to ideal solution It has a simple process; easy to use and program; the number of steps remains the same regardless of the number of attributes Supply chain and logistics management, engineering, manufacturing systems, business and marketing, environment, human resources and water resources management 

A final factor to justify the use of PROMETHEE is that it can be complemented by other methodologies, especially those of rank aggregation of multi-participant decision-makers (Carvalho & Curi, 2013; De Araújo, 2016; Sugiartawan & Hartati, 2018) allowing for the attainment of a highly relevant final rank. In this sense, Carvalho & Curi (2013) presented a manner of using PROMETHEE jointly with the rank aggregation method of COPELAND, due to its good properties and generating good results. The decision-maker's participation takes place with the weighting of the multi-criteria that are considered in the analysis of the watershed performance.

Rank aggregation methods

The so-called rank aggregation methods are considered very intuitive and little demanding, both in terms of computational effort and in relation to the required information to be provided by the decision-maker. There are two forms of rank aggregation method: a priori aggregation method, which, in general, uses the average values of the criteria weights, and a posteriori aggregation methods, which combines the ranks attained for each of the decision-maker's weighting choices. The disadvantage of the a priori aggregation method is that the average of criteria weights may not represent any decision-makers' preferences. In the published literature, the three most referenced a posteriori rank aggregation methods are Borda, Condorcet and Copeland (Pomerol & Barba-Romero, 2012), and other versions based on them. The main advantages of ease-of-use and comprehension of these methods in environmental studies is reported by Laukkanen et al. (2004) and Kangas et al. (2006), and more recently by Nascimento (2016), Carvalho & Curi (2013) and De Araújo (2016).

Borda method

The Borda method, proposed by Jean Charles de Borda (1733–1799), uses an ordinal scale, offered to the decision-maker and the options are awarded a rank through a score. Then, it consists of attributing one point to the ‘most preferred’ option, two points to the ‘second preference’ and so on until the last option (candidate or competitor). In the end, these points are summed and the option with the lowest score is selected. This method, which in its essence is a sum of points as described, has the main advantage of simplicity. However, despite its simplicity and the broad applications of its variants, the Borda method does not respect one of the most important Arrow's axioms, that of the independence of irrelevant alternatives. Such facts can generate distortions, especially the extreme dependence of the results in reference to the chosen evaluation set, and the possibility of dishonest manipulations.

Condorcet method

According to Pomerol & Barba-Romero (2012), the Condorcet method, devised by Jean-Marie Antoine Nicolas de Caritat, Marquis of Condorcet (1743–1794), is considered a precursor of the current multi-criteria French school and works with relationships of surpassing. The alternatives are always compared two by two and a graph is constructed (Boaventura Neto, 2003) to express the relationship between them. This less simple method has the advantage of preventing distortion by making the relative position of two options independent of their relative positions to any other. However, it may lead to the so-called ‘Condorcet paradox’ or intransitivity situation. This happens when alternative A surpasses alternative B, which surpasses C, which in turn, surpasses alternative A. This situation can be exploited in certain problems when the goal is to group options. However, when it occurs, it makes it impossible to generate a ranking of the options. When intransitivity cycles do not appear, and a full pre-order is desired, the Condorcet method should be preferred over the Borda method. If the goal is to make a choice, even with intransitivities, the Condorcet method has one advantage: it requires interactive interventions with the decision-maker, avoiding the optimal paradigm. This paradigm is criticized by Clímaco (2003).

Copeland method

As presented by Pomerol & Barba-Romero (2012), the Copeland method is considered an improvement of the Condorcet method and uses the same adjacency matrix that represents the graph of the Condorcet method. From this matrix, the sum of wins minus the losses is calculated, in a voting by simple majority. The options are then ranked by the result of this sum. The Copeland method allies the advantage of always providing a full ranking (differently from Condorcet) with the fact of yield the same results as the Condorcet, when the latter does not display any intransitivity cycle. When these cycles are present, the Copeland method makes it possible to rank and keep the ranking of the options that do not belong to an intransitivity cycle. In spite of being more computationally demanding than Borda, when there is need to establish a pre-order, or sensu lato order, this method always yields an answer (in contrast to Condorcet method) and, although it does not eliminate, it greatly reduces the influence of irrelevant alternatives. Copeland's method can be considered an improvement between the opposite philosophies of Borda and Condorcet, bringing together, as much as possible, the advantages of both and, thus, is a more appropriate approach to the development of research, being already validated in studies in the Brazilian semi-arid (Carvalho & Curi, 2013; De Araújo, 2016; Nascimento, 2016), which is considered by some to be the most problematic semi-arid region in the world (Cirilo, 2008).

Issues in the Brazilian semi-arid and the Piranhas-Açu watershed

In an analysis aimed towards public policies, Cirilo (2008) affirms that the Brazilian semi-arid is considered one of the most problematic in the whole world, since this region displays issues that are more difficult to overcome than those of other semi-arid regions throughout the world. Among those issues, is the fact that the Brazilian semi-arid region has soils that, for the most part, are very shallow with almost outcropping rocks, which compromises the existence of aquifers, their recharge and water quality. It also presents high temperatures which lead to high evaporation rates. The presence of few perennial rivers and one of the highest population concentrations among world semi-arid regions exert unique and excessive pressures on the water resources. To Cirilo (2008), the Brazilian semi-arid region presented, until the 1990s, a history of bad public policies, or even non-existent, especially on the establishment of small reservoirs highly vulnerable to drought and well drilling in crystalline soils. Allied to these mistakes, the lack of water management was the key factor to maintaining the critical regional framework drought after drought. As a way of alleviating the suffering of underserved populations, the usual solutions are the use of water tankers, work fronts to provide them with some income or other purely palliative measures.

Broadening the discussion on water resources and political aspects, Amorim et al. (2016) points to a specific region within the Brazilian semi-arid that, in addition to the aforementioned issues presented by Cirilo (2008), has stood out as one of the most problematic, due to the conflict for the use of water resources as it comprises a subwatershed that includes part of two of the poorest Brazilian states. The Piranhas-Açu subwatershed, situated in an area where there is widespread dispute about the use of water, has generated concerning prognostics. According to Amorim et al. (2016), in this subwatershed the intensive use of water from the Curema-Açu system is causing conflicts between users, mainly for irrigation and shrimp culture purposes (in the stretch downstream to the Armando Ribeiro Gonçavels dam, located in the state of Rio Grande do Norte), and also between the governments of Paraíba and Rio Grande do Norte states. The intensity of this usage was reflected in the increase of water grant requests that exceeded the capacity of regularization of the water systems of the basin. This scenario has generated several impasses and studies have been made to understand how to better decide this issue. Analysing the basin, Amorim et al. (2016) highlights the following need:

‘It is concluded that there is a need to create participatory environments within the Committee. Among these should be the management committees in each reservoir of the watershed. It is also important to create a Technical Chamber for Conflict Resolution […] Considering the point of view of the managing body (the National Water Agency), with the water scarcity in the Watershed increasing since 2012, institutional measures began to be conducted by the Agency, such as the reactivation of the Operational Technical Group (mid-2013) and the issuance of resolutions in 2014 and 2015 to discipline water use in the Watershed.’

Indexes, sustainability and policy

Amidst problematic scenarios such as those of the Brazilian semi-arid region, sustainability analysis could provide subsidies for better development of political actions becoming an important alternative to contribute to this process. According to Hák et al. (2016), there is a necessity to formulate sustainable development indexes that encompass the political process, as the ultimate target of a sustainability index is many times linked to a political process issue. To Hák et al. (2016), in favour of an efficient operationalization process, as well as the success of the entire agenda of sustainable development objectives, the political process must consider scientific knowledge and evidence already in the early stages of the political cycle. Underdevelopment or underestimation of an index characteristic could lead to false assessments of the degree of achievement of targets and thereby erode the credibility of sustainable development objectives. Hence, the index structure for sustainable development objectives needs more conceptual and methodological work than merely the production of new social, economic and environmental statistics. In a similar manner, Caldas et al. (2018), in a study based on the Portuguese municipalities, analysed the relationship between sustainability and the European Union's investment programme. Such studies contributed to a broad economic analysis through applied models (Caldas et al., 2019a, 2019b). Carvalho & Curi (2013), in a different analysis, presented broad research involving various aspects related to water resources and from this developed a sustainability index aimed to better understand this problem. MacGillivray (2017) highlights the Index of Sustainable Economic Welfare (ISEW) compiled by various US and European research organizations, strongly resonating with growing concerns about the gap between economic prosperity and the ‘quality of life’ experienced by the citizens of the richest nations on earth. According to Lowe et al. (2015), indexes are being used at the national, state and local levels to compare the liveability of cities and regions. However, there are challenges in adopting such indexes. Planning researchers find it a challenge to create indexes that measure something that is publicly evaluated, while other researchers are concerned with the scarce systemic research on the relationships between policies, the built environment and well-being. Thus, the elaboration of a coherent index can become an important step to a sustainable future that is to come.

Piranhas-Açu watershed

The Piranhas-Açu river watershed (Figure 1) encompasses a 42,900 km2 territory, distributed between the states of Paraíba and Rio Grande do Norte. Approximately 1,552,000 people inhabit the region of the watershed. Farming is the main economic activity of the area, where the small subsistence agriculture of beans, intercropped corn and extensive livestock can be highlighted. The watershed is fully located in semi-arid territory, with average rainfall ranging from 400 to 800 mm yearly, and mainly concentrated between February and May (ANA, 2016). The Piranhas-Açu river watershed, located in the north-eastern semi-arid, is formed by one Federal Government-managed river and several tributary rivers from the states of Paraíba and Rio Grande do Norte.

Fig. 1.

Location of the Piranhas-Açu watershed (Source: author).

Fig. 1.

Location of the Piranhas-Açu watershed (Source: author).

Close modal

Methodological procedures

The applied methodology, in accordance with Figure 2, can be divided into four steps, with the first relating to the calculation of indicators of the watershed sustainability index and the other three related to the application of the multi-criteria and multi-decision-maker analyses.

Fig. 2.

Methodological sequence.

Fig. 2.

Methodological sequence.

Close modal

The following is a breakdown of each step of the proposed methodology sequence.

Watershed sustainability index

Aiming to provide the Hydrological Programme of UNESCO with an index that includes the analysis of water resources (at the watershed level), Chaves & Alipaz (2007) elaborated an index from a set of four different indicators: hydrology, environment, life and policy, forming the acronym HELP, and seeking to promote an adequate methodology for the application in various contexts around the world. Such index has been currently classified as one of the most well regarded globally (Juwana et al., 2012). Table 2 presents the indicators of the WSI, split into three categories: pressure, state and response. The advantage of using this model (PRESSURE–STATE–RESPONSE) is that it incorporates the cause and effect process, aiding society stakeholders and the decision-making politicians in visualizing the interconnections between parameters. The WSI is applied to the watershed by selecting a period of five (or four) years, enabling the follow-up of its development.

Table 2.

Indicators and parameters of the Watershed Sustainability Index (Chaves & Alipaz, 2007).

IndicatorsPressure parametersStateResponse
Hydrology Variation in per capita water availability in the last 5 years Per capita water availability Improvements in water use efficiency over the period 
BOD5 over the past 5 years Average BOD5 (long term) Improvements in sewage treatment over the period 
Environment EPI of the watershed Percentage of the watershed with natural vegetation Evolution in terms of watershed conservation (reserves, good management practices) 
Life Variation of the HDI of the watershed region over the past five years HDI weighted by municipal area Evolution in the HDI of the watershed region in the period 
Policy Variation in HDI in education (HDI-education) Legal and institutional capacity in terms of integrated watershed management Evolution of IWRM spending in the period 
IndicatorsPressure parametersStateResponse
Hydrology Variation in per capita water availability in the last 5 years Per capita water availability Improvements in water use efficiency over the period 
BOD5 over the past 5 years Average BOD5 (long term) Improvements in sewage treatment over the period 
Environment EPI of the watershed Percentage of the watershed with natural vegetation Evolution in terms of watershed conservation (reserves, good management practices) 
Life Variation of the HDI of the watershed region over the past five years HDI weighted by municipal area Evolution in the HDI of the watershed region in the period 
Policy Variation in HDI in education (HDI-education) Legal and institutional capacity in terms of integrated watershed management Evolution of IWRM spending in the period 

EPI, Environmental pressure indicator; WSI, Watershed Sustainability Index; DBO5, Biochemical oxygen demand variation; HDI, Human Development Index; IWRM, integrated water resources management.

In its classic form of application the WSI index is calculated from a simple average of the values referring to the set of indicators, through the following formula:
(1)
where:
  • H = hydrology indicators (6 indicators)

  • E = environment indicators (3 indicators)

  • L = life indicators (3 indicators)

  • P = policy indicators (3 indicators)

As presented by Juwana et al. (2012), this index is calculated in a manner in which there is no consultation with the representatives of the watershed, and this generates uncertainty as to the participation of decision-makers. With a view to filling this void in the current research, public participation was introduced in the weighting (importance) evaluation of each of the 15 indicators. The decision-makers were members of the Watershed Committee. To proceed with this analysis, the PROMETHEE method was applied to establish a calculation simulation with the weights of the decision-makers (a total of 27, since there were 27 members of the committee). Finally, with the possession of these 27 calculated results, the Copeland aggregation method was applied with the goal of establishing a final score and presenting a result.

PROMETHEE multi-criteria method

The PROMETHEE II method has the advantage of requiring very clear additional information, which can be easily obtained and managed either by the decision-maker or by the analyst, highlighting its characteristics related to objectivity and flexibility, and therefore being adequate in a relevant way to the WSI, marked by the necessity of an easy application. In terms of the method, for the purpose of the present application, two steps are necessary for each indicator (criterion): (a) definition of the preference function; (b) definition of indicator weights.

Definition of the preference function: Among the PROMETHEE II method preference functions, the following ones were selected: function type I (U-usual), type II (U-Shape), type III (V-Shape) and type V (Linear). The computational support was accomplished through the Visual PROMETHEE® 1.4 software. The definition of parameters was carried out through the application of descriptive statistics in accordance with Carvalho & Curi (2013).

Definition of indicator weights: For each of the 27 decision-makers that answered the questionnaire regarding the evaluation of their preferences (weights), a simulation was generated. The weights, following Carvalho & Curi (2013), were assigned according to the following sequence (Table 3).

Table 3.

Definition of indicator weights.

Answer to the questionnaireValue
Very important 
Important 0.75 
Medium 0.5 
Low 0.25 
None 
Answer to the questionnaireValue
Very important 
Important 0.75 
Medium 0.5 
Low 0.25 
None 

After the results obtained from PROMETHEE II, 27 matrices were elaborated, obtained from pairwise comparisons with each of the subwatersheds in order to find the following step (multi-decision-maker ranking).

Copeland's multi-decision-maker method

After simulating and obtaining the respective subwatersheds' performance according to each expert opinion, the next step was to determine the final ranking of the subwatersheds using the COPELAND method, considering the comparisons that were drawn after the application of PROMETHEE II. At this stage, pairwise comparison matrices were built between the watersheds and later the final matrix that presents the final ranking of the watershed. Finally, the last step consisted of the analysis of the results and final consideration of the study.

Methodological classification of the research

The adapted WSI is fed from two sources: secondary and primary data (through interviews) (Table 4). Regarding the primary data, they were collected following the same criteria adopted by Carvalho & Curi (2013) in their sampling that aimed to compose an index focused on the sustainability of water resources. The interviews took place with members of the Piancó-Piranhas-Açu Watershed Committee, through participation in their meetings. Regarding the secondary data (aimed at feeding the WSI model), a database (from water resources agencies) search was carried out to compose the hydrological indicator (hydrological series and water quality monitoring). In turn, the environmental indicator was determined from satellite image analysis (applying supervised classification on the ArcGis 10.5 software for land-use map composition) provided by the United States Geological Survey database (in the ‘earthexplorer’ section). Furthermore, the ‘life’ indicator was fed by information from the Human Development Index (HDI). Regarding the policy aspects, the composition of the adapted indicator used information collected from the application of questionnaires (primary data, as already mentioned).

Table 4.

Data sources for the composition of the WSI index.

IndicatorSource
Hydrology quantity Brazilian National Water Agency – ANAa/Geological Survey of Brazil – CPRM 
Hydrology quality Brazilian National Water Agency – ANAb 
Environment U.S. Department of the Interior U.S. Geological Survey USGSc 
Life Brazilian Institute of Geography and Statistics – IBGEd 
Policy Brazilian Institute of Geography and Statistics – IBGEe 
IndicatorSource
Hydrology quantity Brazilian National Water Agency – ANAa/Geological Survey of Brazil – CPRM 
Hydrology quality Brazilian National Water Agency – ANAb 
Environment U.S. Department of the Interior U.S. Geological Survey USGSc 
Life Brazilian Institute of Geography and Statistics – IBGEd 
Policy Brazilian Institute of Geography and Statistics – IBGEe 

aANA (2012) Brazilian National Water Agency [WWW Document]. Hidroweb. URL http://hidroweb.ana.gov.br.

bSame as above.

cSatellite images Landsat 5 collected from the U.S. Department of the Interior U.S. Geological Survey, URL: http://earthexplorer.usgs.gov

dBrazilian Institute of Geography and Statistics – IBGE (2015) [WWW Document] URL http://www.cidades.ibge.gov.br/xtras/home.php.

eSame as above.

Indicator weights from the evaluation of members of the watershed committee

Once the indicator values were presented and the questionnaire4 was highlighted at the 14th Meeting of the Watershed Committee, it is necessary to present the value of the indicator weights based on the perception of the committee members (Table 5).

Table 5.

Weights of indicatorsa of the WSI according to the perception of members of the Piranhas-Açu Watershed Committee.

Indicator 1
Indicator 2
Indicator 3
Indicator 4
F%F%F%F%
Very high 13 48.1 13 48.1 11 40.7 25.9 
High 33.3 18.5 25.9 33.3 
Medium 7.4 14.8 18.5 25.9 
Low 7.4 3.7 7.4 
None 3.7 7.4 
No opinion 14.8 7.4 7.4 
Total of decision-makers 27 100 27 100 27 100 27 100 
Indicator 5
Indicator 6
Indicator 7
Indicator 8
F%F%F%F%
Very high 10 37 15 55.6 12 44.4 16 59.3 
High 29.6 25.9 25.9 18.5 
Medium 22.2 7.4 11.1 11.1 
Low 3.7 3.7 7.4 3.7 
None 
No opinion 7.4 11.1 7.4 
Total of decision-makers 27 100 27 100 27 100 27 100 
Indicator 9
Indicator 10
Indicator 11
Indicator 12
F%F%F%F%
Very high 25.9 17 63 16 59.3 18 66.7 
High 11 40.7 22.2 22.2 22.2 
Medium 22.2 7.4 11.1 
Low 3.7 3.7 
None 3.7 
No opinion 7.4 7.4 3.7 7.4 
Total of decision-makers 27 100 27 100 27 100 27 100 
Indicator 13
Indicator 14
Indicator 15
F%F%F%
Very high 11 40.7 10 37 15 55.6   
High 12 44.4 11 40.7 29.6 
Medium 7.4 11.1 11.1 
Low 3.7 3.7 
None 
No opinion 3.7 7.4 3.7 
Total of decision-makers 27 100 27 100 27 100 
Indicator 1
Indicator 2
Indicator 3
Indicator 4
F%F%F%F%
Very high 13 48.1 13 48.1 11 40.7 25.9 
High 33.3 18.5 25.9 33.3 
Medium 7.4 14.8 18.5 25.9 
Low 7.4 3.7 7.4 
None 3.7 7.4 
No opinion 14.8 7.4 7.4 
Total of decision-makers 27 100 27 100 27 100 27 100 
Indicator 5
Indicator 6
Indicator 7
Indicator 8
F%F%F%F%
Very high 10 37 15 55.6 12 44.4 16 59.3 
High 29.6 25.9 25.9 18.5 
Medium 22.2 7.4 11.1 11.1 
Low 3.7 3.7 7.4 3.7 
None 
No opinion 7.4 11.1 7.4 
Total of decision-makers 27 100 27 100 27 100 27 100 
Indicator 9
Indicator 10
Indicator 11
Indicator 12
F%F%F%F%
Very high 25.9 17 63 16 59.3 18 66.7 
High 11 40.7 22.2 22.2 22.2 
Medium 22.2 7.4 11.1 
Low 3.7 3.7 
None 3.7 
No opinion 7.4 7.4 3.7 7.4 
Total of decision-makers 27 100 27 100 27 100 27 100 
Indicator 13
Indicator 14
Indicator 15
F%F%F%
Very high 11 40.7 10 37 15 55.6   
High 12 44.4 11 40.7 29.6 
Medium 7.4 11.1 11.1 
Low 3.7 3.7 
None 
No opinion 3.7 7.4 3.7 
Total of decision-makers 27 100 27 100 27 100 

aIndicator 1. Δ1, Change in per capita water availability of the watershed in the last 5 years (m3/inhabitant * year); Indicator 2. Δ2, Change in BOD5 (last 5 years); Indicator 3. E.P.I. (Rural and Urban); Indicator 4. Variation in H.D.I. income (last 5 years); Indicator 5. Variation in H.D.I. education (last 5 years); Indicator 6. Per capita water availability (m3/inhabitant * year); Indicator 7. Average BOD5 of the watershed (mg/l); Indicator 8.% of watershed with natural vegetation; Indicator 9. M.H.D.I. watershed (population weighted); Indicator 10. Institutional Capacity in Integrated Water Resources Management (legal and organizational); Indicator 11. Improvement in efficient water use (last 5 years); Indicator 12. Improvement in proper water/sewage treatment (last 5 years); Indicator 13. Evolution in conservation areas (protected areas and good management practices, BPM) in the last 5 years; Indicator 14. Evolution in M.H.D.I. (last five years); Indicator 15. Evolution in spending on Integrated Natural Resource Management (last 5 years).

PROMETHEE evaluation matrix

As mentioned above, for the PROMETHEE method, an evaluation matrix was built where the columns represent the criteria (C1, C2, etc.), and the lines represent the options (A1, A2, A3, etc.). For the purposes of application in the present case, the criteria are the indicators of the Watershed Sustainability Index [C1 = Δ1 − Variation in per capita water availability of the watershed in the last 5 years (m3/inhabitant * year); C2 = Δ2 − Variation in BOD5 (last 5 years), and so on] and the options are the Piranhas-Açu watershed subwatersheds (A1 = Seridó, A2 = Piancó, and so on). The evaluation matrix for the analysis of the watershed sustainability index indicators is presented next (Table 6).

Table 6.

Evaluation matrix.

Indicator of the Watershed Sustainability Index (WSI)Δ1, Change in per capita water availability of the watershed in the last 5 years (m3/inhabitant * year)Δ2, Change in BOD5 (last 5 years)E.P.I. (Rural and Urban)Variation in H.D.I. income (last 5 years)Variation in H.D.I. education (last 5 years)Per capita water availability (m3/inhabitant * year)Average BOD5 of the watershed (mg/l)% of watershed with natural vegetationM.H.D.I. watershed (population weighted)Institutional Capacity in Integrated Water Resources Management (legal and organizational)Improvement in efficient water use (last 5 years)Improvement in proper water/sewage treatment (last 5 years)Evolution in conservation areas (protected areas and good management practices -- BPM) in the last 5 yearsEvolution in M.H.D.I. (last five years)Evolution in spending on Integrated Natural Resource Management (last 5 years)
Criterion C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 
Preference function (type) 
Weight The weight of each criterion was defined by each Watershed Committee participanta 
Objective Mab Mic Mi Ma Ma Ma Mi Ma Ma Ma Ma Ma Ma Ma Ma 
Subwatershed Serido (A1) 11.9 − 41.0 13.3 5.6 20.4 1,409 11.7 47.0 0.64 Meand Mean bade − 20 9.7 − 20 
Pianco (A2) 3.5 − 64.7 − 6.3 8.6 31.8 4,569 1.9 68.4 0.57 Mean Mean bad − 20 15.5 − 20 
Peixe (A3) 2.9 5.4 − 23.5 6.0 28.8 1,904 1.4 48.8 0.62 Mean Mean bad − 20 13.6 − 20 
Pataxó (A4) 2.9 − 6.7 11.4 3.7 19.5 1,434 3.9 51.5 0.58 Mean Mean bad − 20 − 10.8 − 20 
Paraú (A5) − 2.1 − 4.0 − 17.7 5.3 25.3 7,330 6.23 43.9 0.59 Mean Mean bad − 20 12.4 − 20 
Média Piranhas Po (A6) 1.6 − 100 − 49.2 5.6 26.5 4,983 2.2 66.4 0.59 Mean Mean bad − 20 12.08 − 20 
Média Piranhas Pa. Po (A7) 8.9 2.5 − 25.8 4.1 24.5 3,369 1.5 67.7 0.6 Mean Mean bad − 20 10.74 − 20 
Média Piranhas Pa (A8) 4.1 365.4 50.3 5.0 29.6 4,216 2.1 74.0 0.57 Mean Mean bad − 20 13.42 − 20 
Espiranhas (A9) − 0.2 − 16.5 34.4 6.0 32.1 1,547 1.9 47.6 0.58 Mean Mean bad − 20 − 15.34 − 20 
Difusas Baixo P (A10) 2.7 0.1 18.8 3.8 18.3 1,832 3.0 42.0 0.62 Mean Mean bad − 20 8.58 − 20 
Alto piranhas (A11) 3.7 6.8 23.4 7.1 32.6 6,713 1.6 58.9 0.55 Mean Mean bad − 20 15.38 − 20 
Indicator of the Watershed Sustainability Index (WSI)Δ1, Change in per capita water availability of the watershed in the last 5 years (m3/inhabitant * year)Δ2, Change in BOD5 (last 5 years)E.P.I. (Rural and Urban)Variation in H.D.I. income (last 5 years)Variation in H.D.I. education (last 5 years)Per capita water availability (m3/inhabitant * year)Average BOD5 of the watershed (mg/l)% of watershed with natural vegetationM.H.D.I. watershed (population weighted)Institutional Capacity in Integrated Water Resources Management (legal and organizational)Improvement in efficient water use (last 5 years)Improvement in proper water/sewage treatment (last 5 years)Evolution in conservation areas (protected areas and good management practices -- BPM) in the last 5 yearsEvolution in M.H.D.I. (last five years)Evolution in spending on Integrated Natural Resource Management (last 5 years)
Criterion C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 
Preference function (type) 
Weight The weight of each criterion was defined by each Watershed Committee participanta 
Objective Mab Mic Mi Ma Ma Ma Mi Ma Ma Ma Ma Ma Ma Ma Ma 
Subwatershed Serido (A1) 11.9 − 41.0 13.3 5.6 20.4 1,409 11.7 47.0 0.64 Meand Mean bade − 20 9.7 − 20 
Pianco (A2) 3.5 − 64.7 − 6.3 8.6 31.8 4,569 1.9 68.4 0.57 Mean Mean bad − 20 15.5 − 20 
Peixe (A3) 2.9 5.4 − 23.5 6.0 28.8 1,904 1.4 48.8 0.62 Mean Mean bad − 20 13.6 − 20 
Pataxó (A4) 2.9 − 6.7 11.4 3.7 19.5 1,434 3.9 51.5 0.58 Mean Mean bad − 20 − 10.8 − 20 
Paraú (A5) − 2.1 − 4.0 − 17.7 5.3 25.3 7,330 6.23 43.9 0.59 Mean Mean bad − 20 12.4 − 20 
Média Piranhas Po (A6) 1.6 − 100 − 49.2 5.6 26.5 4,983 2.2 66.4 0.59 Mean Mean bad − 20 12.08 − 20 
Média Piranhas Pa. Po (A7) 8.9 2.5 − 25.8 4.1 24.5 3,369 1.5 67.7 0.6 Mean Mean bad − 20 10.74 − 20 
Média Piranhas Pa (A8) 4.1 365.4 50.3 5.0 29.6 4,216 2.1 74.0 0.57 Mean Mean bad − 20 13.42 − 20 
Espiranhas (A9) − 0.2 − 16.5 34.4 6.0 32.1 1,547 1.9 47.6 0.58 Mean Mean bad − 20 − 15.34 − 20 
Difusas Baixo P (A10) 2.7 0.1 18.8 3.8 18.3 1,832 3.0 42.0 0.62 Mean Mean bad − 20 8.58 − 20 
Alto piranhas (A11) 3.7 6.8 23.4 7.1 32.6 6,713 1.6 58.9 0.55 Mean Mean bad − 20 15.38 − 20 

aThe weights were established by interviewing the members of the Watershed Committee and in this sense each distinct weight forms a different scenario.

bMaximize.

cMinimize.

dAverage.

eBad.

The criteria weights for the above evaluation were the ones attained for each participant of the Piranhas-Açu River Basin Committee.

Results of the application of Copeland's multi decision-maker method

With the application of the PROMETHEE II method, it was possible to rank the options under the individual preferences of each member who answered the questionnaire (a total of 27 responses) of a group of decision-makers. Thus, it was sought to aggregate this information in a single rank, which represents the group's preference. Table 7 presents a matrix elaborated from the results of the PROMETHEE II application that served as the base for an analysis, aiming at the final aggregation of the options' rankings, in order to take into consideration all the different opinions regarding the weights of the criteria. For this analysis, the COPELAND ranking aggregation method was adopted. In terms of score, the Piancó subwatershed obtained the highest score (264 wins and 6 losses), hence being in a better situation than any other subwatershed of the Piranhas-Açu watershed. On the other hand, Pataxó got the lowest score (1 win and 269 losses) and therefore has the worst performance among all of Piranhas-Açu subwatersheds. Table 7 presents the relationship between wins and losses as a matrix.

Table 7.

Final results of COPELAND analysis.

SubwatershedSeridóPiancóPeixePataxóParaúMédia Piranhas PotiguarMédio P. Paraibano PotiguarMédio Piranhas ParaibanoEspinharasDifusas Baixo PiranhasAlto PiranhasLosses
Seridó 27 27 26 27 27 26 18 27 210 
Piancó 
Peixe 26 24 11 24 87 
Pataxó 27 27 27 27 27 27 27 26 27 27 269 
Paraú 27 26 26 23 11 27 144 
Média Piranhas Potiguar 23 11 40 
Médio P. Paraibano Potiguar 26 16 26 23 98 
Médio Piranhas Paraibano 27 26 16 26 25 26 149 
Espiranhass 27 27 25 27 26 25 27 200 
Difusas Baixo Piranhas 22 27 27 26 27 27 27 21 27 231 
Alto Piranhas 27 16 51 
Wins 60 264 183 126 230 172 121 70 39 219  
SubwatershedSeridóPiancóPeixePataxóParaúMédia Piranhas PotiguarMédio P. Paraibano PotiguarMédio Piranhas ParaibanoEspinharasDifusas Baixo PiranhasAlto PiranhasLosses
Seridó 27 27 26 27 27 26 18 27 210 
Piancó 
Peixe 26 24 11 24 87 
Pataxó 27 27 27 27 27 27 27 26 27 27 269 
Paraú 27 26 26 23 11 27 144 
Média Piranhas Potiguar 23 11 40 
Médio P. Paraibano Potiguar 26 16 26 23 98 
Médio Piranhas Paraibano 27 26 16 26 25 26 149 
Espiranhass 27 27 25 27 26 25 27 200 
Difusas Baixo Piranhas 22 27 27 26 27 27 27 21 27 231 
Alto Piranhas 27 16 51 
Wins 60 264 183 126 230 172 121 70 39 219  

Finalizing the calculations (applying the COPELAND method to the numbers of wins minus the number of losses), the watershed that obtained the best result and thus is the one in the best condition in terms of sustainability is the Piancó watershed (net result of 258), followed by Média Piranhas Potiguar (net result of 190). The worst and therefore least sustainable result (and with priority in actions in terms of management) was displayed by the Pataxó watershed (with a net result of −268).

Results observed in terms of watershed sustainability

In terms of results observed (Table 8), the Pataxó subwatershed demonstrated having the greatest need for intervention, hence being the least sustainable according to its lowest final score (−268), followed by Difusas Baixo Piranhas (−192) and Seridó (−150). Deepening the analysis of the least sustainable subwatershed (Pataxó), it can be noticed that in this watershed the hydrology indicator in terms of state (1,434.4 m3/hab*year) and of life (M.H.D.I = 0.58) can be highlighted, generating the necessity of future measures in terms of increasing the water availability as well as measures directed to improving the quality of life.

Table 8.

Final result.

Final rankingSubwatershedNet values (wins – losses)
Piancó 258 
Média Piranhas Potiguar 190 
Alto Piranhas 168 
Peixe 96 
Médio P. Paraibano Potiguar 74 
Paraú −18 
Médio Piranhas Paraibano −28 
Espinharas −130 
Seridó −150 
10 Difusas Baixo Piranhas −192 
11 Pataxó −268 
Final rankingSubwatershedNet values (wins – losses)
Piancó 258 
Média Piranhas Potiguar 190 
Alto Piranhas 168 
Peixe 96 
Médio P. Paraibano Potiguar 74 
Paraú −18 
Médio Piranhas Paraibano −28 
Espinharas −130 
Seridó −150 
10 Difusas Baixo Piranhas −192 
11 Pataxó −268 

PROMETHEE multi-criteria method

The application of the multi-criteria method PROMETHEE II has proven to be very effective in helping decision-making in watershed management, when well used and when the decision-maker is certain in relation to his objectives and preferences. The presented case study aimed to insert a more participatory approach to evaluate watershed performance through the use of indicators of the watershed sustainability index. It was observed that in the decision-making process one of the main aspects was the choice of weighting factors for each decision-maker.

In this sense, the search for the inclusion of society in this decision process was important, since it made it possible to know the priorities of the public by weighting the criteria in the evaluated scenarios. In the watershed committee meeting there was a significant response rate, with about 80% of participants answering the questionnaire, showing that in the presented case there is a willingness to contribute directly to the improvement of the planning unit.

The multi-criteria methodology presented advantages already observed in other studies. For example, an option may be very good from a hydrological point of view, but it may present a great risk to the environment. Therefore, the context itself directs the problem to an over-sorting approach, not admitting compensations between the criteria, thus alternatives are sought out to better balance the main factors involved with the decision-maker's preference structure.

Analysis of Copeland's multi-decision-maker method

By adopting this strategy (Copeland's method), in addition to the already mentioned advantages of the method (among which is being superior to Borda and Condorcet), results were sought considering individual decision-maker opinions and rankings in order to define the final ranking. It was possible to build flow matrices of gains and losses after multi-criteria analysis for each decision-maker subwatersheds' rankings and comparing one by one (pairwise) to achieve the final results.

The method proved to be effective as it represented the reality of opinions of the watershed committee members, which is marked by various different interests and, in this sense, the local sustainability analysis process through the watershed sustainability index better represents a broader participation of decision-makers.

Political outlines

Amidst the problems in the Brazilian Piranhas-Açu subwatershed, the application of the approach presented in this work has enabled a broad participation, as it has included the opinions of various different classes of members of the Brazilian Watershed Committee in the analysis of the indicators of the Watershed Sustainability Index, thus establishing a decision simulation for each member (PROMETHEE method), then aggregating it to a final outcome (Copeland method).

With the understanding that the members of the Brazilian Watershed Committee come from various segments of the population, it is clear that the adapted application of the WSI, with the possibility of analysing what really matters or not from the committee member's standpoint before the application, had a relevant effect on reliability and future acceptance, as they participated in the process and thus allowed for some democracy in building the application process.

In terms of management, it is noteworthy that the Piancó subwatershed, located in the Brazilian state of Paraíba, presented greater sustainability in contrast with Pataxó subwatershed (located in the Rio Grande do Norte state), showing important indications for the Federal Government in terms of defining in which political region to intervene. A curious fact that the methodology pointed out is precisely the fact that the Pataxó watershed is relatively close to a large reservoir known as Armando Ribeiro Gonçalves. In turn, Piancó has a different reservoir known as Curemas Mãe D'agua. However, it seems that the indicators point to a scenario where the equity and distribution of this resource is wider and, therefore, has reflected in a scenario of better sustainability. Pataxó's low sustainability opposed to Piancó's high sustainability demonstrates that it is not enough to be near the water, but a subwatershed should enjoy the asset through sustainable management of water resources and perhaps this is a good starting point for future analyses.

1

Here the authors have selected several water-related indexes and not necessarily indexes focused on sustainability in water resources.

2

International Hydrological Programme of UNESCO (The United Nations Educational, Scientific and Cultural Organization).

3

Among which is the fact of its originality in clearly having the watershed as the basis for its application.

4

Questionnaire available in Appendix II.

ANA
(
2012
).
Agência Nacional de Águas (Brazilian National Water Agency) Hidroweb
.
Available from: http://hidroweb.ana.gov.br (accessed 15 March 2015)
.
ANA
(
2016
).
Agência Nacional de Águas (Brazilian National Water Agency)
.
Piranhas-Açu Watershed Resources Plan
,
Brasília
.
Balali
V.
Zahraie
B.
Roozbahani
A.
(
2014
).
A comparison of AHP and PROMETHEE family decision making methods for selection of building structural system
.
American Journal of Civil Engineering and Architecture
2
(
5
),
149
159
.
Boaventura Neto
P. O.
(
2003
).
Grafos: teoria, modelos, algoritmos (Graphs: Theory, Models, Algorithms)
.
Editora Edgard Blücher
,
São Paulo
,
Brazil
, p.
328
.
Brans
J. P.
Vincke
P.
Mareschal
B.
(
1986
).
How to select and how to rank projects: the PROMETHEE method
.
European Journal of Operational Research
24
,
228
238
.
Brown
A.
Matlock
M. D.
(
2011
).
A Review of Water Scarcity Indices and Methodologies, University of Arkansas, The Sustainability Consortium, White Paper No. 106, Sustainability Consortium web [online]. Available from: http://www.sustainabilityconsortium.org/wpcontent/themes/sustainability/assets/pdf/whitepapers/2011_Brown_Matlock_Water-Availability-AssessmentIndices-and-Methodologies-Lit-Review.pdf (accessed 29 January 2015)
.
Caldas
P.
Ferreira
D. C.
Dollery
B.
Marques
R. C.
(
2018
).
Municipal sustainability influence by European Union investment programs on the Portuguese local government
.
Sustainability
10
(
4
),
910
.
doi:10.3390/su10040910
.
Caldas
P.
Ferreira
D. C.
Dollery
B.
Marques
R. C.
(
2019a
).
Economies of scope in Portuguese local government using an augmented Hicks–Moorsteen approach
.
Regional Studies
53
(
7
),
963
976
.
doi:10.1080/00343404.2018.1508872
.
Caldas
P.
Ferreira
D.
Dollery
B.
Marques
R.
(
2019b
).
Are there scale economies in urban waste and wastewater municipal services? a non-radial input-oriented model applied to the Portuguese local government
.
Journal of Cleaner Production
219
,
531
539
.
doi: 10.1016/j.jclepro.2019.02.076
.
Clímaco
J. C. N. A.
(
2003
).
Critical reflection on optimal decision
.
European Journal of Operational Research
153
,
506
516
.
De Araújo
D. C.
(
2016
).
Metodologia de análise de conflitos na gestão de águas pluviais urbanas com auxílio de métodos multicritérios e multidecisores (Methodology for Decision Support in Urban Rain Water Management Combining Multi-Criteria and Multi-Decision-Maker Methods)
.
Doctoral dissertation
,
Universidade Federal de Pernambuco
,
Recife
.
Gleick
P. H.
(
1998
).
Water in crisis: paths to sustainable water use
.
Ecological Applications
8
,
571
579
.
http://dx.doi.org/10.1890/10510761(1998)008[0571:wicpts]2.0.co;2
.
Gleick
P. H.
Loh
P.
Gomez
S.
Morrison
J.
(
1995
).
California Water 2020: A Sustainable Vision
.
Pacific Institute for Studies in Development, Environment, and Security
,
Oakland, CA
,
USA
.
Hák
T.
Janoušková
S.
Moldan
B.
(
2016
).
Sustainable development goals: a need for relevant indicators
.
Ecological Indicators
60
,
565
573
.
IBGE
(
2015
).
Instituto Brasileiro de Geografia e Estatística (Brazilian Institute of Geography and Statistics)
.
Available from: http://www.cidades.ibge.gov.br/xtras/home.php (accessed 18 March 2015)
.
Juwana
I.
Muttil
N.
Perera
B. J. C.
(
2012
).
Indicator-based water sustainability assessment – a review
.
Science of the Total Environment
438
,
357
371
.
Kangas
A.
Laukkanen
S.
Kangas
J.
(
2006
).
J. Social choice theory and its applications in sustainable forest management-a review
.
Forest Policy and Economics
9
,
77
92
.
Laukkanen
S.
Palander
T.
Kangas
J.
(
2004
).
Applying voting theory in participatory decision support for sustainable timber harvesting
.
Canadian Journal of Forest Research
34
,
1511
1524
.
Loucks
D. P.
(
2000
).
Sustainable water resources management
.
Water International
25
(
1
),
3
10
.
Lowe
M.
Whitzman
C.
Badland
H.
Davern
M.
Aye
L.
Hes
D.
Butterworth
I.
Giles-Corti
B.
(
2015
).
Planning healthy, liveable and sustainable cities: how can indicators inform policy?
Urban Policy and Research
33
(
2
),
131
144
.
MacGillivray
A.
(
2017
).
Accounting for change: indicators for sustainable development
. In:
Tomorrow's History
.
Zadek
S.
(ed.).
Routledge
,
Abingdon
,
UK
, pp.
122
129
.
Mays
L. W.
(ed.) (
2007
).
Water Resources Sustainability
.
McGraw-Hill
,
New York
,
USA
.
Nascimento
R. S.
(
2016
).
Modelo Conceitual para Gestão da Qualidade da água em Sistemas de Abastecimento de Médio Porte (Conceptual Model for Water Quality Management in Medium-Sized Supply Systems)
.
Doctoral dissertation
,
Universidade Federal de Campina Grande
,
Campina Grande
.
Pomerol
J. -C.
Barba-Romero
S.
(
2012
).
Multicriterion Decision in Management: Principles and Practice
.
Springer Science & Business Media
.
Postel
S. I.
(
2000
).
Entering an era of water scarcity: the challenges ahead
.
Ecological Applications
10
,
941
948
.
Sugiartawan
P.
Hartati
S.
(
2018
).
2018 Third international conference on informatics and computing (ICIC)
. In:
Group Decision Support System to Selection Tourism Object in Bali Using Analytic Hierarchy Process (AHP) and Copeland Score Model
,
Sumatra, Indonesia
, pp.
1
6
.
Tundisi
J. G.
(
2003
).
Água no século xxi: enfrentando a escassez (Water in the XXI Century: Facing Scarcity)
.
RiMa
,
São Carlos
,
Brazil
.
Velasquez
M.
Hester
P. T.
(
2013
).
An analysis of multi-criteria decision making methods
.
International Journal of Operations Research
10
(
2
),
56
66
.