Rethinking Brasília's water services: ‘new targets’ using the regulatory impact assessment (RIA) tool

Swyngedouw (2004) pointed out that the widely adopted market-oriented model of water services (WS) should be provided on an affordable and universal basis (Decker 2016). In fact, achieving the WS targets envisaged in the sustainable development goals, would not be possible without a proper embedding of these services within a public, social, financial and regulatory system backed by massive national and international public funding (Hutchings et al. 2018).

To fulfill the minimal efficiency considering the regulatory system (see more details in Berg (2013)), regulatory impact assessment (RIA) is understood by the authors as a tool that allows the evaluation of potential impacts arising from government action and assumes an important role since it helps to improve the knowledge base (Meuleman 2014), especially in WS. Any strategy or decision such as a primary law or a secondary regulation proposed by the government could have implications and change either the public or the private sector behaviors (Massarutto et al. 2013) with impacts on society under different points of view.

However, there is no register of RIA's (ex-ante, ex-tempore or ex-post) applicability worldwide in the utilities agenda, although studies on partial evaluation of legal reforms in this matter were held in Europe after fiscal crises at national scale. Therefore, this process should require not only practices but also a comprehensive understanding of the RIA conceptual model, its framework and support methods before decision-making. From that viewpoint, implementing a homogeneous RIA framework in all regulated sectors improves its compliance and social knowledge (Castro 2014) supporting effective outcomes and gaining legitimacy by the stakeholders.

In terms of structure, the WS in Brazil, together with wastewater, solid waste management and storm water management, is classified as part of a broader concept called basic sanitation under subnational regulation. Currently, the main investments come from the federal government. The sector is decentralized into municipal jurisdictions in which local administrations (subnational) are responsible for choosing or providing these services and define regulators. A few regulatory agencies for water supply and sanitation services were created after 1997, and Law no. 11445/2007 formalized their activities. Until 2015, 50 regulatory agencies were established for WS, out of which 22 were state agencies, one was a district agency, 24 were municipal agencies and three were municipal consortia (micro-regional) (ABAR 2015).

Concerning these figures, Brazil has always had a historical deficit in the provision of basic services (approximately 9.8 and 29.7 million households have no access to water supply and wastewater services, respectively), which fosters inequities and enhances actual social gaps (IBGE 2016). Brasília, which is the capital of the country and has a population of 2.98 million people (estimated by IBGE (2016)), has over 98% of the water supply coverage (SNIS 2015).

Although the performance of WS in Brasília does not represent the reality of the country, the Water, Energy and Wastewater Regulatory Agency (ADASA in Brazilian Portuguese) – the local regulatory agency – enacted an audacious and multiple dimensional target plan concerning this matter: Resolution law no. 08/2016 of July 4, which involves the social, economic, environmental and governance areas (ADASA 2016). On the one hand, the proposed Resolution law no. 08/2016 was developed in a context of non-stable coverage and quality of services and without good practices of governance. On the other, the logical framework adopted did not meet the good practices of regulatory analysis, e.g. RIA. Thus, this recent intervention represents an opportunity to attest RIA as a policy tool under a historical data (evidence), different viewpoint and an attractiveness perspective by stakeholders.

In this paper, the RIA approach allowed not only to assess the impact of each policy option through three stages, (i) status quo, (ii) assessment and (iii) consultation, but also to reduce bias along the process, e.g. myopic representation and anchoring bias (for more details, see Montibeller & von Winterfeldt (2015)). The policy options, titled ‘short term for 2018 (ST)’ and ‘long-term for 2020 (LT)’, were extracted from Resolution law no. 08/2016. In addition, two other options were built based on statistical analysis (based on the performance of WS in Brasília) and expressed with a 95% confidence interval.

The impacts of each option were assessed by attractiveness through a categorical-based evaluation technique (MACBETH) developed in a way that could adapt to a specified evaluating structure in order to appraise different possibilities (Pinto & Marques 2016). The judgment of attractiveness was obtained by preference elicitation from the customers, the regulators and provider's points of view observed along interactions between the authors and decision makers (DMs). The outputs were checked by a thorough and robust sensitivity analysis.

The RIA conceptual model used to assess Brasília's WS case provides an unprecedented and relevant example of RIA in the water sector. Furthermore, combining analytic and participatory methods enhanced the quality of information collected during the decision-making process.

The next section describes RIA followed by a brief contextualization of the water sector in Brazil and Brasília and of Resolution law no. 08/2016, in particular. The methodology adopted in this study was developed as described in the following section, and the next section provides an example of its application and its corresponding policy implications. The final section draws some concluding remarks.

The notion of RIA comes from the North American experience, where regulators must conduct an economic analysis of regulatory actions, send it to oversight regulatory units, publish it for notice, and comment (Dunlop & Radaelli 2016). Over 50 countries have implemented various forms of RIA as a mandatory step in ongoing or new acts, covering primary and secondary legislation – that is, the regulation of agencies – e.g. the environmental protection agency in the USA or health in Canada.

According to Alemanno (2016), RIA made it possible to render policy processes not only transparent and participatory but also more rational. This may be accelerated, through the increased availability of evidence supported by policy initiatives, and enhanced scrutiny of the preparatory process, leading to the adoption of a final rule contributing to better regulation and better awarding. However, the triumph of this decision depends on changing behaviors, and this is encapsulated in the term ‘compliance’, so that both the policy-maker at the design stage and the regulator at the delivery stage are seeking to promote the same thing.

Nowadays, the number of countries that adopted RIA is stable. A recent survey by OECD (2015) shows that the majority of OECD countries have established the requirement to conduct RIA in a legal or official document, but this does not follow in practice. This gap is more pronounced in the case of subnational regulation, as in the current case. Moreover, as to the aid methods, the benefit analysis (BA), risk assessment (RA) and cost-benefit analysis (CBA) are a requirement, respectively, in approximately 88, 60 and 29% of OECD. Outside OECD countries, among emerging economies and developing countries, there are several examples of attempts to introduce RIA as a systematic impact assessment of proposed new legislation or existing regulation. In many cases, RIA has been adopted as part of donor-financed projects and programs. In terms of numbers, estimates show that by 2009, some form of RIA has been adopted or discussed in at least 50 developing countries, a number that has almost certainly increased since then.

Despite being a vital policy tool tested in some countries and under quick evolution, as governments continually refine its design and application, RIA still has problems. Perhaps the main problem is the failure of its implementation and the inability to understand it (Jacobs 2016).

Although there are several definitions of RIA, this paper considers it as a policy tool used under an ex-ante, ex-tempore or ex-post time view. Moreover, four classical steps were defined regarding its framework. The first one named ‘the status quo (i)’ is composed of (a) the purpose, (b) options and (c) potential impact. ‘The purpose’ contains the extent of the situation and the complexity of the current problem. The assessment (ii) step is composed of the scenario and options analysis in each impact or criteria concerning functions of profits and losses. The consultation (iii) step should not be limited to letting stakeholders provide their point of view on impacts, but also on the problems definition, on policy objectives and on support to obtain data regarding the analysis. Finally, ‘review or final step’ aims to monitor and evaluate the results of regulations developed along the proposed frame (Carvalho et al. 2017, 2018). Given this academic approach, this paper focused on steps (i) to (iii), adapting the ‘consultation’ step within the multicriteria decision analysis (MCDA) modeling method and the participatory method (direct elicitation).

To accomplish the RIA process, three classical methods could be used: (i) methods to generate and analyze data on a specific impact; (ii) methods to integrate and aggregate data; and (iii) participatory methods to facilitate the interaction between different stakeholders (Jacob et al. 2011). However, the combination of methods depends on the problem's focus, the options, the available data, the internal resources, time and, finally, the analysts' familiarity with a certain method.

Considering the fast population growth, a prosperous economy and a changing climate, achieving sustainable universal water coverage would require a paradigm shift in current efforts (Monney & Antwi-Agyei 2018). According to a study published by the Ministry of Cities based on data collected from the SNIS – information system on water, wastewater and sanitation – by the end of 2013, the water supply coverage level in Brazil reached approximately 83% of households; the wastewater collection came to 50%, whereas the wastewater treatment remained way below those levels (SNIS 2015). Compared to other regions in the country, this panorama showed a great difference since the indicators in the center and southern parts of Brazil were much better (89.6 and 89.4% respectively, for water supply and 50.2 and 41.4% for wastewater treatment), whereas in the northern and northeastern regions, the reality was much worse (56.9 and 73.4% and 16.4 and 32.1%) (SNIS 2015).

Currently, the water and wastewater market in Brazil consists of 28 state water utilities, six regional companies, and 1,474 local water utilities. In most of them – almost 1,000 – the water supply and wastewater collection and treatment are provided directly by the municipalities. The 28 state water utilities encompass 4,002 municipalities in the water sector and 1,292 in the wastewater sector (SNIS 2015).

Brasília has been experiencing an increasing rate of water coverage (0.74%) and a decrease of wastewater services (0.16%) since 2000. In 2014, over 98 and 82% of physical accessibility in terms of water and wastewater services, respectively, were achieved (SNIS 2015). This result required a constant interaction among CAESB, the water utility company of Brasília; ADASA, the water, energy, and sanitation regulatory agency of the Federal District; all regional governments and local associations to ensure and improve the quality of the provided services, safeguard economical and infrastructural sustainability and protect customer interests.

ADASA, which is linked to SEMA, an environmental secretary of the Federal District was created through Law no. 3365/04, and later its competences were extended by Law no. 4285/08. Currently, ADASA is responsible for simultaneous regulatory functions over water resources and water and wastewater services including solid waste and drainage. Besides, it also operates in sectors related to gas, petrol, and oil. Although ADASA achieved prominent aspects in terms of governance, regarding laws and processes followed by an agency, there is still a gap in regulatory substance, such as lack of tools and rules available to regulate CAESB.

Recently, Resolution law no. 08/2016 was published concerning the methodology for evaluating the performance of water and wastewater services in Brasília, as well as the general procedures between ADASA and the state water and wastewater company (CAESB). Additionally, this secondary legislation established targets to comply with the good practices. A total of five dimensions, (i) customer interests, (ii) infrastructural sustainability, (iii) economic and financial management, (iv) environmental sustainability and (v) governance, were carried out. Two policy options were developed by ADASA. The first one, named ST, represents the current conditions in WS in Brasília, i.e. without clear incentives to CAESB improve coverage, quality of services and its effects on the tariffs. The second, called LT, was a very ambitious proposal in terms of coverage, quality of service and other criteria (infrastructure and environmental) which should reflect in incentives and water tariff changes. Differences between the options show that the process was not well structured, i.e. without an intermediate option and with a huge gap in terms of incentive. Analyzing it from a different perspective, it should be an example of the theory of capture of the regulator by the provider.

Thus, it is fair to say that the current conditions of the water sector in Brasília enable a chance to observe a national experiment, which will provide the evidence basis to assess the impact of different policy options under distinct perspectives, including several objectives and dimensions. Appendix 1 (available with the online version of this paper) highlights the hierarchical structure adopted, developing the concept from the ultimate goal of the decision at hand through a descending order of specificity.

According to Resolution law no. 08/2016, ‘Service provider’ refers to protecting the customer interests through accessibility and quality of service. This dimension is often associated with social sustainability of services. ‘Infrastructural sustainability’ promotes the suitable balance between resilience, robustness and performance of the system evaluating the current practices on this matter. The importance of ‘Environmental sustainability’ relies on avoiding or minimizing pressure on the natural resources through the efficiency and sustainability practices.

An adapted RIA framework (Figure 1) was developed through three steps, (i) status quo, (ii) assessment, and (iii) consultation, as discussed above under ‘RIA's definition’.

The projection performed by MATLAB^® software was considered as an input of each policy option, i.e. ‘no policy change’ (NPC) 2018 and ‘NPC 2020’. Furthermore, two other alternatives were built during the MCDA process by the preferences of DMs. Consequently, at the end, six policy options were checked.

Following Resolution law no. 08/2016 objectives, ten criteria were grouped into three dimensions (Appendix 1). These dimensions of providing WS helped meet the social, economic and security goals in an environmentally sustainable manner.

Since the criteria previously defined by Resolution law no. 08/2016 are value-oriented and need to be operationalized, descriptors and value functions have to be assigned considering that they are clearly not linear across their ‘scale’, depending on the condition itself. Those descriptors (see Appendix 1) are organized in a set of possible impact levels related to the fundamental points of view (criteria).

To overcome some bias on the part of the DMs and stakeholders in the policy decision process (see Montibeller & von Winterfeldt 2015), a consultation step was developed by direct elicitation techniques (Greco et al. 2016) from different DMs under their experience as customers, regulators and service providers.

This step is divided into three stages: (i) structuring issues, (ii) scale transition, and (iii) weighting coefficient procedure. The first stage focuses on the reference levels. The second stage, perhaps the main one, corresponds to the transition from ordinal to cardinal information, revealing the origin of the notion of strength preference. The last stage refers to capturing the variation of the weight as the value measure swings (see Montibeller & von Winterfeldt 2015) from the lowest to the highest value level on the scale (Parnell 2009). The interaction between the facilitator and the DM was done through web survey and structure interviews.

The ‘Assessment’ step consists of the analysis of the options in each impact criterion. Using decision-methods to support regulations should help regulatory authorities to choose the one that best meets the policy objectives. The ability to choose the most appropriate method depends on the combination of the following aspects: (i) problem's definition (Greco et al. 2016), (ii) expected results and secondary questions (Roy & Słowiński 2013), (iii) capacity to address bias (Montibeller & von Winterfeldt 2015), (iv) time, analyst's familiarities and (v) available resources.

The word ‘problematic’ refers to the way in which decision aid (DA) is envisaged (Greco et al. 2016). (P. α) exposes the problem in terms of best choice. In (P. β), categories are designed prior to receiving actions (A), which will or may be processed in the same way during the following step. (P. γ) is oriented towards and relies on a partial or complete ordering (pre-order) on A that may be considered as a suitable instrument for comparing actions pairwise (Greco et al. 2016).

Furthermore, to select the MCDA method, the analyst should start by reflecting on the best and only way to answer the following essential question: ‘what type(s) of results is this method expected to produce?’ In many cases, answering this question may lead to a shortlist of more than one method. Therefore, using the ‘five other key questions to choose the right method’ as well as the ‘secondary questions’ found in Roy & Słowiński's (2013) work, and analyzing their answers would enable a particular method to be chosen.

This exercise of RIA is particularly suitable for (P. α) providing the best alternative selection process, although there is no need for an option that has the best results for every criteria. The options may differ in scope, which is they may achieve several objectives, and no option will obviously be the best to achieve all objectives. Moreover, the following features such as utility score, performance scale properties, preference information required, imprecision, uncertainty, admitted compensation and robustness of the method, were admitted to supporting the authors' decision method. Bearing those characteristics in mind, the evaluation technique (MACBETH) was selected to perform the RIA's assessment step.

The results of RIA's approach allow DMs to understand the scenario, the inherent dimension tradeoffs defined by Resolution law no. 08/2016, and how policy options have a different local score, as well as the impact of each criterion on the overall suitability results under customers, regulators and services providers' perspectives.

Performance profiles for each policy option in each criterion are presented in Appendix 1. The difference between policy options from Resolution law no. 08/2016 and statistical influence with their respective characteristics was more emphasized in ‘adequacy of treatment capacity ATC’ – linked to the statistical influence – ‘fulfilment of water intake licensing – FWIL’ and ‘sludge disposal – SL’. The ‘Neutral’ and ‘Good’ reference levels, which were developed during the interaction modeling with ‘decision makers DMs’, had reference scores between 0 and 100.

It was appropriate to inform and detail the timeline of interaction with DMs before each stage of the consultation process. During the questionnaire test phase, two experts were invited to improve the form and content of the draft version according to the requisites of the MCDA modeling method.

Although the authors had previously defined 60 days to conclude the direct elicitation process, it took 85 days, of which 23 were required to develop the questionnaire, including the test phase. The frequency of interaction in each stage did not allow them to invest in a parallel consultation. The interval between the first stage and the third stage was influenced by the difficulty to identify other DMs that represented that viewpoint.

After describing the criteria performance for each policy option, which is the keystone of a global additive appraisal model, it was necessary to create the definition of scales attractiveness or value functions for each criterion, which convert performance levels into a local score. First, the authors requested the DMs to point out the lower (‘neutral’) and upper (‘good’) reference level considering each criterion's performance.

Here, the authors invest in a one-on-one discussion with DMs to explore the concept of lower and upper reference. The idea is just to identify where the gap is more advanced in each criterion and calibrate it with the previous value.

The innovative (de)biasing technique presented above allows not only to overcome some bias, e.g. myopic problem representation, anchoring, etc., but also to easily understand how policies would affect stakeholders in different ways according to their preferences expressed through the local score.

During this stage, the modeling of DM's preferences aims to translate impacts into value scores, indicating the attractiveness of an impact when compared to the reference levels defined above under ‘Structuring issues’. The authors allowed some interaction between the facilitator and DMs to discuss and solve judgment inconsistency when necessary. Appendices 2–4 (available with the online version of this paper) show the preference scale for the continuity of service criteria under customers', regulator's and provider's perspective.

After this procedure, the authors checked the data consistency with the DM to verify whether reference levels agree or not with the value function obtained from the DM's judgment.

There are several procedures to assign weighting coefficients to the criteria, e.g. swing weighting method, trade-off, etc. (Greco et al. 2016). Moreover, Montibeller & von Winterfeldt (2015) identified a family of biases – some were previously mentioned in this article – that could affect the elicitation procedure in the decision analysis processes. According to these authors, using swing weighting with multiple stakeholders' opinions could provide different degrees of detail that could improve the decision quality (as confirmed above under ‘Scale transition’ by the difference between each perspective's value function). During the third stage of this study, the differences between neutral and good attractiveness for every pair of criteria were obtained from the most to the least attractive criteria swing.

The weighting matrix based on elicit qualitative judgments from DMs concerning the attractiveness difference between swings was repeated row-by-row until the matrix of judgments was completed. It defines the importance and variation in the decision context to aid weight assessment and communication. The DM was, in each perspective, asked to examine and confirm the weighting coefficients to validate results (see all results in Appendix 5, available online). This part of the research was challenging because the DMs were not familiar with such a concept and swing exercise. To complete all stages (by an additive model) a total of 62 days were required. Figure 2 provides an overall score for each policy option under each perspective.

Hence, researchers could see that there was a policy option, ‘long term – LT’ which stood out from the other alternatives. According to the customers' and regulators' perspectives, ‘LT’ surpassed ‘Good’ on the reference level. On the contrary, from the providers' perspective, the ‘LT’ policy option was far from ‘Good’, which shows the contrast between the evaluated perspectives.

Furthermore, policy options ‘no policy change – NPC 2018’ and ‘NPC 2020’ were observed in all perspectives and were prone to much reflection regarding the ‘short term – ST’ option presented in Resolution law no. 08/2016. From that point of view, such overall scores show how the impact of each policy option changed under the adopted perspective.

According to Figure 2, the RIA framework shows that the options obtained from Resolution law no. 08/2016 were far from expected in all perspectives assessed in terms of ST in a comparative analysis with NPC. In fact, such a process conducted by ADASA without a clear framework, evidence or transparency produces outcomes that are not consistent with the stakeholders' objectives. Before drawing up policy implications and concluding remarks, an additional analysis (sensitivity and robustness) was completed and detailed in terms of the LT and NPC policy options (Figure 2 – top).

The sensitivity analysis was conducted between the first and the second option in all perspectives, i.e. ‘LT × NPC’. As an example, Appendix 6 (available online) shows the sensitivity analysis on weight concerning the affordability level from a different perspective as to the ‘affordability level’ criterion.

By performing a sensitivity analysis on the weight of ‘affordability level’ between ‘LT × NPC 2018’ policy option, it is possible to observe that its variation – keeping identical proportions for the weights of the remaining criteria – only introduces changes in the ordering of the best option if increased to a level greater than 82% (above the current value), as shown in Table 1. All criteria between the combination of ‘LT × NPC 2018’ and ‘LT × Good’ policy options were checked to support this analysis.

Observing ‘LT × NPC 2018’, from the regulator's perspective, over 74.2% of ‘affordability level – AL (above the current value)’ is necessary to change results. Regarding the provider's perspective, 56.1% of AL is desirable to modify the portfolio of policy options. In all perspectives ‘physical accessibility – PA’ is stable, i.e. there are no changes in the ranking.

When ‘LT × Good’ policy options are compared in terms of ‘adequacy of treatment capacity – ATC’ the variation of the results would be achieved if this criterion increased to levels higher than 35.2%. If the same criterion surpasses 22.4% of the policy options, the list would change under the regulator's perspective. Pragmatically, the requirements for changing the order of best option need a moderate modification of the weight criteria in all appraised transitions.

Finally, Table 1 illustrates the strength of customer's position on weights for each criterion, which was not captured by the analysis developed by ADASA, before enacted Resolution law no. 08/2016.

A table of global comparisons of the actions was integrated into the M-MACBETH^® software. To explore the extent to which assumptions could be drawn due to varying amounts of differing degrees of imprecision or uncertainty, the robustness analysis proposed carries on the ordinal, MACBETH and cardinal information because of its capacity to represent not only the criteria's order but also the judgments and weighting coefficients.

When uncertainty is involved in the procedure of impact estimation, an impact table, for example, provides an overview of the options impact on the criteria extracted by Resolution law no. 08/2016 from the customers' perspective. ‘LT’ dominates () ‘ST’ and is globally more attractive () than all of the other policy options (Appendix 7, available online). However, if researchers increase the imprecision of local cardinal information (97%), the model will no longer have enough information to determine if ‘Good’ is globally more attractive than ‘LT’ (or vice versa). In other words, ‘Good’ dominated ‘Neutral’. Pragmatically, these conditions presented on the ‘dominance table’ proved that the ‘LT’ policy option was robust, i.e. a strong variation is needed to change its relation of dominance and attractiveness.

From the regulator's perspective, ‘LT’ dominated ‘Neutral’ and was also globally more attractive than all of the other policy options. There was no change in the local cardinal information that modified the robustness of the final solution. Regarding the providers' perspective, changing only 2% of the weight in each criterion (PA, SEC, and WLDS) only denoted incomparability between ‘NPC 2018’ and ‘NPC 2020.’ Moreover, by changing the weight of each criterion (CS, QSW, CWWWS, ATC, and FWIL) by at least ±1%, the composition of the final solution changed and denoted incomparability between ‘NPC 2018’ and ‘NPC 2020’.

The ‘LT’ policy option is supported by the robustness analysis providing that the nature of stakeholders are present in sufficient conditions of changing imprecision or uncertainty. Therefore, results (LT) show that this method leads to robust conclusions suitable to develop recommendations under different perspective and policy options design for the current analysis.

In addition, an analysis of the efficient frontier was made in order to check the locus of all possible policy options formed by the proposed dimensions obtained from Resolution law no. 08/2016 (see graphs in Appendices 8–10, available online). According to the customers' perspective (Appendix 8(a)), the effects of ‘LT’ were more positive in terms of ‘service provider’ than ‘infrastructural sustainability’.

Regarding the regulator's perspective, ‘LT’ was more effective in ‘environmental sustainability’ than the ‘infrastructural sustainability’ (Appendix 9(c)). The providers' point of view showed that ‘LT’ was the best option in an intersection between ‘service provider’ and ‘environmental sustainability’ (Appendix 10(b)). Thus, regardless of strategy combination, ‘LT’ was overall well positioned, which allowed the authors to confirm the relative merit of this policy option. Moreover, since 2014 the gap between the ‘affordability level’ and tariffs has been increasing (7%, 2014–2018). Such a situation reinforces the idea that the current alternative does not achieve the previously goal defined by ADASA (3%).

Based on the robustness analysis from the customer and regulators' perspective, it is fair to say that ‘LT’ was a robust policy option, regarding the providers' perspective. Although the dominant conditions were not robust between the third and fourth policy options, ‘LT’ remained in the first position.

To summarize, looking back to Figure 2, LT represents the best policy option with a considerable distance from ST (current policy options from Resolution law no. 08/2016), e.g. 217 units in customers' perspective; 154.2 units in terms of regulator's perspective and 160.7 units regarding providers' perspective.

The result found with the RIA approach exercise – based on a conceptual and framework model – is positive, since it recognized the practical application and experience of the whole process in an authentic context.

Using the coefficient of determination as well as the confidence interval to evaluate the stability of complementary ‘NPC’ policy options not only improved the analysis but also yielded reliable information before moving on to the assessment stage. Although the policy options are inserted prior to the assessment stage, the MCDA modeling method let the DMs create ‘Neutral’ and ‘Good’ references during the interaction process, which allowed them to make comparisons at the end of the process.

Furthermore, in order to overcome common bias in this study, different perspectives and (de)biasing techniques, such as structuring issues and swing weighting, were used to improve the quality of the model analysis following Montibeller & von Winterfeldt's (2015) suggestions.

As highlighted before, using MCDA modeling in the RIA ‘assessment’ step offers an ideal approach to evaluate problems shaped by several objectives and political context describing performances, e.g. social and environmental without forcing conversion to monetary units.

The MACBETH method was helpful in designing a consistent valuation/selection criterion; the options and the analysis of sensitivity and robustness performed before making the final decision.

The weight sensitivity analysis supported the superiority of the first policy option, which would not affect the overall result of the model. The robustness analysis of the outputs denoted that as much as the authors varied the percentage of the local cardinal information, the best policy option (LT) never fell to the third and fourth positions in the final portfolio.

Despite the limits of this exercise, the RIA approach suggests that policy options extracted from Resolution law no. 08/2016 could be reviewed in order to ensure successful implementation. Furthermore, based on the authors' analysis the following remarks should be considered:

• As for the global impact, from customers and regulators' perspective, the ‘LT’ policy option is the first one chosen, because: (i) weighting results in terms of PA, AL and CS and (ii) best performance in terms of cost/benefit between the service provider and infrastructural and environmental sustainability. In other words, LT means: good services, good prices and coverage. However, these improvements require huge investments long term. ‘Good’, is the following option, based on decision-makers' preferences under that same perspective, which reinforces the importance of building solutions by means of DMs' interaction.

• The most important challenge to overcome considering all perspective is ‘rethinking’ the short term ‘ST’ targets take into account long term ‘LT’ results.

• From the providers' perspective, the ‘Good’ policy option stood out among all other alternatives, although a change of only ±1% in some criteria's weight does not promote a change in the first or in the second policy options, reflecting the need to rethink the new targets proposed by this legal act/Resolution law no. 08/2016. In other words, the current options does not achieve the expected impact under the concessionaire's perspective.

• Although the participation of more DMs make the decision processes more credible, it could be resource consuming if not well designed and carefully planned.

• The DMs' preferences showed interesting features, i.e. the ‘service provider’ aspect was common and predominant in all of the studied perspectives. Both customers' and providers' perspectives highlighted infrastructural sustainability. Finally, DMs who represented customers and regulators' perspectives tended to emphasize environmental sustainability. In fact, the regulator does not adjust policy based on prior activity (Henze et al. 2012).

• As mentioned before, the distance between LT and ST shows how far the customers' perspective is from the reality of the Resolution law no. 08/2016. As discussed above under ‘Brasília water sector in brief’, the results demonstrate how the incentives of each policy option are diverse according to the coverage, quality of service and costs. In fact, LT reduces the gap between expected stakeholders involved in such work.

Although the DMs had a certain difficulty understanding the RIA model used in this study, it is noticeable that the multi-methodological approach can contribute to an inclusive learning process that promoted the legitimacy of the output as well as a solid justification of the analysis. Therefore, it was not surprising to hear from some DMs later interviewed that the RIA approach structure facilitated understanding the decision process of multiple objective problems.

The authors' practical involvement in designing the RIA didactic capability policy tool allowed DMs to review targets in order to meet their perspective; contributed to generating a supportive reference to justify more studies in this area, and consequently, to discuss solutions.

In addition, according to this study (looking for the distance between the LT and ST), when DMs neglect prescriptive tools that: (i) could improve their understanding of the preliminary scenario; (ii) organize the adequate options, criteria and its descriptors; and (iii) the participatory process, governmental actions in some cases are unable to solve complex problems. Efforts in order to create incentives in WS, as the proposed LT, should always be fostered, even if they require an extensive data collection, not necessarily either straightforward or easy, mainly for lower developed regions, as they will pay clear dividends. Naturally, with increasing data, a structuring process along with accurate perspective of stakeholders' better regulations can be developed.

Therefore, the analysis enabled authors to critically assess the new targets proposed by Resolution law no. 08/2016, or ‘rethinking these’, i.e. the current alternative (presented by Resolution law no. 08/2016) is not efficient, because the incentives are far from the other options (created by the RIA's framework) and this is not legit. The current revisiting analysis by ADASA and the current water crises in Brasília reinforce that through a suitable approach based on evidence and stakeholders' involvement should give more rationale to regulators arising legal acts in complex sectors, as in the case of water services. It enables decisions that are more reliable when the DMs face a complex problem and when society requires more accountability and effectiveness. The RIA approach applied to the environmental regulatory system should serve as a starting point to promote discussion, attract public interest for policymakers, and align different perspectives under several objectives, as it is common in water management problems. To conclude, the proposed method can be considered a suitable decision support system and useful regulatory tool.