This paper investigates what information water resource managers think they need to make decisions on climate change adaptation. This is achieved through a hypothetical case study where participants, all actual water resource managers or in research, practitioner or administration roles linked to Australian water resources management, were given theoretical future climate scenarios and asked to make decisions based on the available information. The case study provided useful insights into why there is little evidence of effective climate change adaptation being implemented despite significant advances in climate impacts and adaptation science over the last decade. It was found that in order to bridge the gap between climate change adaptation recommendations and successful implementation at practitioner level there is a demand for: improved translation, communication and packaging of existing climate science information into sector- and location-specific impacts (e.g. hydrological interpretation of climate model rainfall projections and the associated uncertainties); attribution of historical and future hydroclimatic changes (e.g. not just what has happened or is going to happen but why and the confidence and likelihoods surrounding that); quantification of costs and benefits of any decision; and understanding of the social, political, and environmental contexts and level of acceptance associated with any decision.

## INTRODUCTION

Successful adaptation outcomes are supported by decision making that is informed by the best available climate science (e.g. Burton 1997; Sarewitz & Pielke 1999; Patt & Dessai 2005; Power et al. 2005; Meinke et al. 2009). Useful research and insights already exist on science-policy interfaces (e.g. McNie 2007; van den Hove 2007), boundary workers or knowledge brokers (e.g. Jasanoff 1990; Cash et al. 2003; Hegger et al. 2012; Lemos et al. 2012) and the barriers to governance of climate adaptation (e.g. Meinke et al. 2009; Biesbroek et al. 2011). However, a fundamental gap still exists between the information that climate science provides and the information that is practically useful for end-users and decision makers (e.g. Pielke 2007; Shaw et al. 2009; Kundzewicz & Stakhiv 2010; Kiem & Verdon-Kidd 2011; Mote et al. 2011; Pielke & Wilby 2012; Kiem & Austin 2013a, 2013b). In some cases it appears (or people believe) that uncertainty associated with climate science is not adequately quantified and communicated or that the climate information is simply ‘too uncertain’ to be of any practical use. Due to high uncertainty surrounding precipitation projections this disconnect is emphasised within the Australian water resource management and agricultural sectors and has been identified as a major barrier preventing well documented facts, themes and recommendations from being translated into successful adaptation outcomes (e.g. Ziervogel & Downing 2004; Kiem et al. 2010a, 2010b; Ziervogel et al. 2010; Dilling & Lemos 2011; Wågsæther & Ziervogel 2011; Lemos et al. 2012; Rickards 2012).

A recent project conducted for the Australian Government's National Climate Change Adaptation Research Facility (NCCARF; www.nccarf.edu.au/publications/decision-making-under-uncertainty) brought together climate scientists (information providers) and a group of climate science end-users (i.e. either actual water resource managers or in research, practitioner or administration roles linked to Australian water resources management) to develop a dialogue and improve understanding about what climate information is required, and what information climate science can provide now and can be expected to provide over the next 5–10 years. Aspects of climate science that are likely to remain highly uncertain were also identified. Key findings from the larger NCCARF project included that uncertainty in climate science is seen as a key barrier to adaptation and that it is a multi-faceted issue – with challenges identified in terms of communication of uncertainty, misunderstanding of uncertainty and the lack of tools/methods to deal with uncertainty (see Kiem et al. (2014) for further details). Both producers of climate information and end-users of this information felt that uncertainty was not well communicated and that there were also key differences surrounding uncertainty in terms of expectations for the future – most end-users were of the belief that uncertainty would reduce within the next 5–10 years. However, producers of this information were well aware that this is most likely not the case, a point that has been demonstrated in many previous studies (e.g. Parry et al. 2007; Randall et al. 2007; Stainforth et al. 2007; Koutsoyiannis et al. 2008, 2009; Blöschl & Montanari 2010; Montanari et al. 2010; Verdon-Kidd & Kiem 2010; Kiem & Verdon-Kidd 2011; Brown & Wilby 2012; Stephens et al. 2012). Many end-users were also of the opinion that uncertainty needs to be reduced in order to develop effective adaptation strategies and it is still often the case that uncertainty is used as a reason (or excuse) to operate under ‘business as usual’. This is important as some decision makers may be waiting for uncertainty in climate information to reduce before they take action on adaptation and risk planning. However, this reduction in uncertainty may never eventuate, or may happen long after the optimum time for action.

Traditionally, in carrying out research on human-induced climate change and its impacts, scientists have followed a pathway that starts with the specification of greenhouse-gas emissions and ends with possible impacts and possible response strategies. From the perspective of high-level policy and decision makers through to practitioners working on implementing adaptation strategies, the problem is that each step in this pathway has an associated uncertainty. More importantly, these uncertainties compound at each step meaning that by the time the step of projecting climate change impacts at spatial scales relevant for decision making is reached, the uncertainties have exploded (e.g. Jones 2000; Brown & Wilby 2012).

Despite the significant advances made in the climate science and adaptation field, as well as the science-policy-practice interface, operationalising scientific knowledge for robust and successful climate change adaptation policy or decision making remains a challenge, and numerous studies exist that demonstrate or speculate why (e.g. Hulme & Dessai 2008; Boezeman et al. 2013; Dewulf 2013; Kiem & Austin 2013b; van Enst et al. 2014). Successful climate change adaptation, specifically water resource management in a variable and changing climate, requires more than simply translating scientific knowledge into practical knowledge. There are multiple other issues, interdependencies, feed-backs and overlaps to consider such as the existing political situation (local, state and national), the cost and benefits of the various adaptation options, demographics, social and cultural issues and historical and current controversy associated with some water resource management strategies (e.g. desalinisation versus a new dam, recycled water for drinking water, etc.).

It is acknowledged that many studies exist that focus on trans- and multi-disciplinarity, co-production of knowledge and the need for translating scientific knowledge into practical knowledge (and how best to do it), but previous research (e.g. Ziervogel & Downing 2004; Pielke 2007; Kundzewicz & Stakhiv 2010; Ziervogel et al. 2010; Dilling & Lemos 2011; Kiem & Verdon-Kidd 2011; Mote et al. 2011; Wågsæther & Ziervogel 2011; Lemos et al. 2012; Pielke & Wilby 2012; Rickards 2012; Kiem & Austin 2013a, 2013b) strongly suggests that this work, while undoubtedly important, is still mostly academic or theoretical and that very little actual successful implementation exists on the ground or in real life.

Therefore, this paper investigates the following question: what information do Australian water resources managers think they need to make decisions on climate change adaptation and how does this relate to the information currently provided by climate science? A secondary aim is to better understand what water resource managers in Australia actually consider when they are making decisions related to climate change adaptation and why, at least for the Australian water resources management sector, the well documented climate adaptation themes and recommendations continue to emerge with little evidence of effective climate change adaptation strategies actually being implemented. We explore this knowledge gap from the perspective of water resources management in Australia in order to better understand why resilience and adaptive capacity levels are not showing any noticeable improvements despite the significant advances in climate science (both impacts and adaptation science) over the last decade. We also discuss (below under the section ‘Lessons learned from the hypothetical case study’) how the situation might be improved to increase the actual occurrence (i.e. at the practitioner or industry level) of successful climate change adaptation in the water resources sector in Australia.

## METHOD

The research question is addressed using a hypothetical case study, conducted during a workshop associated with the NCCARF ‘Decision making under uncertainty’ project (www.nccarf.edu.au/publications/decision-making-under-uncertainty). Climate scientists and water resource managers (or researchers, practitioners or administrators working in roles linked to water resources management) were brought together for a 2-day (12–13 April 2012) workshop in Canberra, Australia, to discuss decision making under uncertainty and the gap that currently exists between the information that climate science provides and the information that is practically useful for water resource management. The overall aims of the workshop were to:

• improve climate scientists’ understanding about what climate information is required by water resource managers and what format the information needs to be provided in;

• improve water resource managers’ understanding of what information climate science can currently provide, the limitations of the science and the uncertainties associated with the outputs;

• develop a better understanding of what climate science can realistically be expected to provide over the next 5–10 years and what probably will never be possible;

• learn about the decision making process and how uncertainty is currently dealt with.

Table 1 shows a list of workshop participants and their role within their respective organisations. Participants were a mix of climate scientists and researchers, high-level civil servants and senior experts in government agencies, or industry practitioners involved in water resources management in Australia. No participants were politically elected officials and as such the term ‘decision making’ refers to decisions made at the practitioner level rather than the higher-level policy decisions (although many of the participants regularly provide advice and information to politically elected officials and so were very familiar with the high-level decision making process).

Table 1

Workshop participant list

Name Organisation Institutional role
Alan Randall University of Sydney Professor – economics, uncertainty, decision making
Alice Howe Lake Macquarie City Council (LMCC) Local government natural resources manager (Sustainability)
Andrew Davidson New South Wales Office of Water (NOW) Water resources manager
Anthony Swirepik Department of Climate Change & Energy Efficiency (DCCEE) Water resources manager
Bertrand Timbal Centre for Australian Weather and Climate Research (CAWCR) Climate scientist
Brendan Berghout Hunter Water Water resources manager
Bruce Rhodes Melbourne Water Water resources manager
Bryson Bates Commonwealth Scientific and Industrial Research Organisation (CSIRO) Hydrologist and water resources engineer
Chris Lee New South Wales Office of Environment & Heritage (OEH) Government administration – impacts and adaptation
David Griggs Monash University Climate scientist
David Post Commonwealth Scientific and Industrial Research Organisation (CSIRO) – Land and Water Hydrologist
Eleanor McKeough Melbourne Water Water resources manager
Francis Chiew Commonwealth Scientific and Industrial Research Organisation (CSIRO) – Land and Water Hydrologist and water resources engineer
Greg Hertzler University of Sydney Professor – economics, uncertainty, decision making
James Ward University of South Australia Water resources engineer and environmental scientist
Jane Chrystal Central West Catchment Management Authority (CMA) Natural resources manager
Jason Evans University of New South Wales (UNSW) Climate scientist
Jean Palutikof National Climate Change Adaptation Research Facility (NCCARF) Researcher – climate change impacts, economic and planning issues
Ken Day Queensland Climate Change Centre of Excellence Government research and administration – natural resources management
Mike Roderick Australian National University (ANU) Climate scientist
Neville Nicholls Monash University Climate scientist
Penny Whetton Commonwealth Scientific and Industrial Research Organisation (CSIRO) – Marine and Atmospheric Climate scientist
Peter Hayman South Australian Research and Development Institute (SARDI) Climate and agriculture scientist
Rae Moran Victoria Department of Sustainability and Environment (DSE) Water resources manager
Rod McInnes Sydney Catchment Authority (SCA) Water resources manager
Sam Capon National Climate Change Adaptation Research Facility (NCCARF) Researcher – climate change adaptation
Shahadat Chowdhury New South Wales Office of Water (NOW) Water resources manager
Tim Capon University of Sydney Researcher – economics, uncertainty, decision making
Todd Sanderson University of Sydney Researcher – economics, uncertainty, decision making
Name Organisation Institutional role
Alan Randall University of Sydney Professor – economics, uncertainty, decision making
Alice Howe Lake Macquarie City Council (LMCC) Local government natural resources manager (Sustainability)
Andrew Davidson New South Wales Office of Water (NOW) Water resources manager
Anthony Swirepik Department of Climate Change & Energy Efficiency (DCCEE) Water resources manager
Bertrand Timbal Centre for Australian Weather and Climate Research (CAWCR) Climate scientist
Brendan Berghout Hunter Water Water resources manager
Bruce Rhodes Melbourne Water Water resources manager
Bryson Bates Commonwealth Scientific and Industrial Research Organisation (CSIRO) Hydrologist and water resources engineer
Chris Lee New South Wales Office of Environment & Heritage (OEH) Government administration – impacts and adaptation
David Griggs Monash University Climate scientist
David Post Commonwealth Scientific and Industrial Research Organisation (CSIRO) – Land and Water Hydrologist
Eleanor McKeough Melbourne Water Water resources manager
Francis Chiew Commonwealth Scientific and Industrial Research Organisation (CSIRO) – Land and Water Hydrologist and water resources engineer
Greg Hertzler University of Sydney Professor – economics, uncertainty, decision making
James Ward University of South Australia Water resources engineer and environmental scientist
Jane Chrystal Central West Catchment Management Authority (CMA) Natural resources manager
Jason Evans University of New South Wales (UNSW) Climate scientist
Jean Palutikof National Climate Change Adaptation Research Facility (NCCARF) Researcher – climate change impacts, economic and planning issues
Ken Day Queensland Climate Change Centre of Excellence Government research and administration – natural resources management
Mike Roderick Australian National University (ANU) Climate scientist
Neville Nicholls Monash University Climate scientist
Penny Whetton Commonwealth Scientific and Industrial Research Organisation (CSIRO) – Marine and Atmospheric Climate scientist
Peter Hayman South Australian Research and Development Institute (SARDI) Climate and agriculture scientist
Rae Moran Victoria Department of Sustainability and Environment (DSE) Water resources manager
Rod McInnes Sydney Catchment Authority (SCA) Water resources manager
Sam Capon National Climate Change Adaptation Research Facility (NCCARF) Researcher – climate change adaptation
Shahadat Chowdhury New South Wales Office of Water (NOW) Water resources manager
Tim Capon University of Sydney Researcher – economics, uncertainty, decision making
Todd Sanderson University of Sydney Researcher – economics, uncertainty, decision making

Workshop participants (Table 1) were selected so as to provide a comprehensive background on the state of current climate science, whilst also providing a summary of the usefulness of climate science from the perspective of water resource managers. Participants were selected because of their prominence in their particular field of climate science or their specific role as a water resource-related manager or decision maker (e.g. representing a particular sector or level of government).

To ensure independence and decrease the potential for bias, the workshop was conducted by professional and experienced facilitators from Global Learning (for more information see www.globallearning.com.au/) rather than members of the project team (i.e. authors of this paper). The computer software iMEET! (www.imeet.com.au/public/) was also used at the workshop to increase productivity and efficiency during the workshop and afterwards when reporting. The iMEET! software captures discussions and makes the information available immediately, allowing rapid organisation, analysis, evaluation and evolution of ideas. This style of facilitation ensured all participants were given the opportunity to voice their concerns but had the added benefit of preventing discussion being dominated by a limited number of individuals.

The first day of the workshop and the first session of the second day covered water resource managers’ experiences in using climate information and also involved several leading climate scientists giving their views on the ‘state of climate science and its application in Australia’. The final sessions of the second day were devoted to decision making under uncertainty and included investigating differences in participants’ perceptions on knowledge and uncertainty (see Kiem et al. (2014) for details), a hypothetical case study on decision making under uncertainty, and a wrap-up session where the key themes that emerged were discussed and given priority rankings.

The focus of this paper is the hypothetical case study which required all participants to engage in an interactive exercise that highlighted the information required and different methods water resource managers use to make decisions, even when faced with significant uncertainties. As such, the aims and results of this paper represent a subset of the goals and outcomes of the overall workshop. Specifically, the aim of the hypothetical case study was to gain insights into the following:

• What information water resource managers think they need to make decisions.

• How water resource managers make decisions and how climate scientists think water resource managers make decisions.

• If decisions change given different circumstances or additional information, and if so, what are the implications of this?

• Both during and after an event, how people assess whether the decision made was a success or failure.

• Strengths and weaknesses of current or traditional water resource management approaches.

All participants (i.e. both climate scientists and water resource managers) were told to consider themselves as water resource managers belonging to an urban coastal water authority in the year 2012. In Phase 1 of the hypothetical case study, participants were told that the best information strongly indicated that population and associated demand on water resources were projected to increase and annual average rainfall (and water availability) was projected to decrease such that supply probably will not meet demand 20 years from now (see Figure 1).
Figure 1

Hypothetical case study (Phase 1): historical, current and projected water availability for a theoretical urban coastal water authority (line = historical water availability, the squares on the right represent upper bound (top), lower bound (bottom) and best estimate (middle) for 2030 water availability based on hypothetical future climate projections).

Figure 1

Hypothetical case study (Phase 1): historical, current and projected water availability for a theoretical urban coastal water authority (line = historical water availability, the squares on the right represent upper bound (top), lower bound (bottom) and best estimate (middle) for 2030 water availability based on hypothetical future climate projections).

Given this information participants were then asked:

• What information do you need to decide on options for adaptation?

• What are your options for ensuring the region does not run out of water (or at least prolonging or decreasing the chance of that situation occurring)?

• Of the available options what extra information do you need in order to make a decision as to what to do?

• If you have to make a decision now what would it be and how did you use the information available to you?

Once responses to Phase 1 questions had been entered into iMEET! (and locked so no editing could take place as extra information became available), Phase 2 of the hypothetical case study required participants to imagine that it was now 2020 (i.e. almost a decade after the previous decisions were made) and the following extra information was now available:

• Whatever option you decided on in 2012 is now well on the way to being implemented;

• o except if it was a new dam – there was an election in 2013 and the budget cuts and ministerial reshuffling associated with the change of government and increased mining in potential dam sites meant the new dam never got started.

• Eight of the last 10 years have seen above average rainfall and extensive flooding in your region and people are beginning to question the need for recycled water or desalination plants.

• Population and water demand are still projected to increase as in 2012 but the updated climate projection is now for a climate 10 years from now (i.e. 2030) that looks much the same as the 1990–2010 climate – so there is still a chance of running out of water (due to increasing demand) but that chance is markedly lower than when you previously decided what to do back in 2012.

Participants were then asked to consider the same questions posed previously (i.e. during Phase 1) and describe how their response would or would not change and why, given this new information.

The final stage of the hypothetical case study, conducted after responses to Phase 2 were locked into iMEET!, required participants imagining that it was now 2030 and they were presented with Figure 2 which illustrates the ‘reality’ (i.e. what actually happened from the time they were first required to make a decision through to the time horizon they were making their decision about). Participants were then asked to comment on their original and subsequent decisions:
• What would you have done differently if you knew the information in Figure 2 broadly in 2012? That is, what would you have done at 2012 if you were told that the period 2010 to 2030 would be characterised by a wet 2010s, short/intense drought in the early 2020s, followed by a wet 2025/26 and rapid drying from 2027 on?

• What would you have done differently if you knew the information in Figure 2 exactly in 2012?

• In hindsight, what could you have done differently without knowing anything more but which would have left you better prepared for what eventuated?

• Of the information you thought you needed, if it had been provided would you have actually been able to make a better decision?

• Rate your decisions at the two points in time as you would if you were at 2030 and reflecting back.

Figure 2

Hypothetical case study (Phase 3): water availability for a theoretical urban coastal water authority 1990–2030.

Figure 2

Hypothetical case study (Phase 3): water availability for a theoretical urban coastal water authority 1990–2030.

## RESULTS

### Hypothetical case study – Phase 1: decisions made at 2012 for 2030

Figure 3 summarises the participant responses made during Phase 1 (illustrated in Figure 1) of the hypothetical situation described above. A key insight to emerge was that despite the limited and vague information given, all participants were able to identify options to address the situation and all participants were able to make a decision. This in itself is not surprising, given that decisions are regularly made under uncertainty or with only partial knowledge about likely consequences (e.g. investment decisions, career decisions, decisions concerning health, etc.). However, in practice, and contrary to the results of the hypothetical case study, when it comes to climate science many are still reluctant to consider climate change adaptation until the science is more certain (e.g. Jacobs et al. 2005; Kiem & Austin 2013a, 2013b). Unfortunately, uncertainty surrounding climate science will not disappear (e.g. Parry et al. 2007; Randall et al. 2007; Stainforth et al. 2007; Koutsoyiannis et al. 2008, 2009; Blöschl & Montanari 2010; Montanari et al. 2010; Verdon-Kidd & Kiem 2010; Kiem & Verdon-Kidd 2011; Brown & Wilby 2012; Stephens et al. 2012) and novel frameworks for climate adaptation decision making under uncertainty and research aimed at translating uncertainty into risk are urgently required. A lot can be learned from the extensive body of knowledge relating to assessing and dealing with climate risks (i.e. climate risk in general as opposed to impacts and risks associated with anthropogenic climate change), much of which pre-dates the emphasis on climate change adaptation (e.g. Hammer et al. 2000; Hayman 2000; Cash et al. 2003; McKeon et al. 2004; Adger et al. 2005; Meinke et al. 2006, 2009; Hayman et al. 2007).
Figure 3

Responses at 2012 to the hypothetical situation illustrated in Figure 1 (i.e. Phase 1).

Figure 3

Responses at 2012 to the hypothetical situation illustrated in Figure 1 (i.e. Phase 1).

Also important to note is that when asked what extra information was required to (a) determine plausible options or (b) make a decision (or make a better decision), the majority of responses did not indicate a need for more climate scenario information or even more accurate climate scenario information. Rather, the emphasis was on the need for improved translation and packaging of the existing climate science information into sector- and location-specific impacts (e.g. hydrological interpretation of climate model rainfall projections), attribution of historical and future hydroclimatic changes (e.g. not just what has happened or is going to happen but why and the confidence and likelihoods surrounding that), the costs and benefits of any decision, and understanding of the social, political, and environmental contexts and level of acceptance associated with any decision.

### Hypothetical case study – Phase 2: decisions made at 2020 for 2030

The key insights to emerge from Phase 2 (i.e. at 2020) were that even with the minimal information given, the decisions made in 2012 were still valid and relevant in 2020. The extra information deemed necessary to determine options or make a decision in 2020 was similar to 2012 (e.g. more information on climate drivers and attribution of recent and historical events, better quantification and understanding of the uncertainty associated with the future scenarios (climatic and otherwise), more information on derived climate variables (i.e. not just the information coming out of a climate model but the secondary impacts of those projected changes on things like streamflow, evaporation, water demand, hydrological conditions, vegetation, etc.)).

The decisions made at 2020 were mostly a continuation of the decisions made at 2012, despite the changed climate outlook and wet conditions experienced just prior to the time the decisions were made. Interestingly, despite the recent wetter than average conditions (i.e. full water storages) the main factor driving participants’ decisions was to make sure they were prepared for the worst case scenario (e.g. running out of water). Participants seemed to interpret the recent wet conditions as just a short-term excursion from what they still perceived as an overall drying trend (i.e. they put more weight in the future climate projection than the recent or current conditions). This result is in line with other comments made at the workshop in relation to current decision making and engineering methodologies, and can be summarised as ‘currently, we always resolve uncertainty by overbuilding’. The viability and efficiency of this approach is questionable (e.g. time and funding may not be available, cost of ‘overbuilding’ may outweigh the cost of the risk you are trying to protect against, ‘overbuilding’ to protect against a threat may limit or reduce the ability to make the most of opportunities, what is thought to be an ‘overbuild’ may give a false sense of security if the risk assessment is flawed or if, as recent research suggests (e.g. Kiem & Verdon-Kidd 2013; Verdon-Kidd & Kiem 2014; Verdon et al. 2014; Ho et al. 2015a, 2015b; Vance et al. 2015), the risk changes over time).

An interesting and important issue that arose during the hypothetical exercise is the need for information on public perceptions and acceptance of climate science, and any proposed adaptation strategies based on that science, if the climate scientists are thought to have ‘got it wrong’. This emphasises the need for clarification on terminology associated with issues like uncertainty and risk, and improved communication and education aimed at explaining that projected scenarios are possible futures as opposed to what will happen. Further, climate science is usually not the only consideration in making climate-sensitive decisions, or even the most important (Power et al. 2005). There is increasing support, from the results obtained here and findings from previous studies (e.g. Adger et al. 2005; Füssel 2007; Kiem & Austin 2013b), for the suggestion that the gap is not just between the science and the decision makers, rather it is that the decision has to be socially, politically, economically and environmentally acceptable for it to be implemented. Even in a perfect world where scientists provide useful information to water resource managers and water resource managers subsequently make robust climate change adaptation decisions based on that science, if people do not like the decision there will always be difficulty in getting that decision implemented (e.g. every time a desalinisation plant or reservoir is proposed, when water trading or allocation schemes are introduced, when sea-level inundation or flood management policies are changed, etc.). This point that very little climate change adaptation is going to happen unless it is politically and socially acceptable was raised many times during the hypothetical case study, and this can only happen when concepts of natural variability and uncertainty within broader scale climatic changes are fully understood (e.g. the need for a new dam does not necessarily disappear just because above-average rainfall is received for a few years, the dangers of building on a floodplain do not disappear just because it has not flooded there for 30 years, etc.). These key themes and barriers also emerge consistently in the literature around the gap between climate science and effective climate change adaptation (e.g. Meinke et al. 2009; Rickards 2012; Kiem & Austin 2013a, 2013b) and a formalised ‘knowledge broker’ program has been proposed to help bridge this gap (Kiem et al. 2014, submitted).

### Hypothetical case study – Phase 3: hindsight evaluation made at 2030

Figure 4 summarises the participant responses to Phase 3 (i.e. at 2030) of the hypothetical situation described above and Table 2 shows how participants rated the decisions made at 2012 and 2020. Figure 4 (and Table 2) suggests that most were happy with their decisions made at 2012 and would not have made a very different decision if they knew broadly what was going to happen out to 2030. However, some minor improvements that would have been made include: improved community engagement (since it is easier dealing with the community when you have confidence you know at least approximately what is going to happen); holding off on expensive plans (e.g. desalination, new dam); and concentration on demand management. These results suggest that it is possible to make decisions and achieve effective climate change adaptation by considering and planning for a wide range of plausible futures by developing robust and flexible responses that are consistently reviewed moving forward in time.
Table 2

Responses at 2030 to the hypothetical situation illustrated in Figure 2 (i.e. Phase 3) – rate your decisions at the two points in time as you would if you were at 2030 and looking back

Rate your decisions as if you were at 2030 and looking back
• Good decisions were made, these were: keeping options open; minimising large premature commitments; maintaining alertness to both demand and supply side management
• We had done well as we had made a reasonably robust and defendable decision on information at the time and provided for long-term resilience through integrated water management and climate-independent options in addition to increasing liveability and improving aquatic eco system health. However, just because we think it is the right decision(s) does not necessarily mean it is the ‘best’ decision. For example, the most efficient decision would have been to ride it out, but this would have promoted a wait-and-see response to climate that would not lead to efficient longer-term decisions and would not have gained other benefits of integrated water management
• 2012: A bit risky to leave it to the community, especially as they might take the ‘soft option’ of avoiding costly infrastructure (in this case it would have paid off but that is just luck). Perhaps some small-scale infrastructure investment would have been more sensible. 2020: Lucky again that, despite high rainfall for recent years, the community accepted a moratorium on growth/development to preserve the town's water supply
• The relatively low cost of a small but expandable desalination plant would have been a favoured option throughout unless mitigating factors (e.g. a lack of public/political support) had been found
• The decisions made would not have been perfect but the decision making process would rate highly. We did not run out of water so we get our annual bonus!!
• Satisfactory at both points in time
• From the 2030 perspective our decision making process was good at 2012 and 2020. It allowed contingency plans to be activated only if required and prevented investment too early
Rate your decisions as if you were at 2030 and looking back
• Good decisions were made, these were: keeping options open; minimising large premature commitments; maintaining alertness to both demand and supply side management
• We had done well as we had made a reasonably robust and defendable decision on information at the time and provided for long-term resilience through integrated water management and climate-independent options in addition to increasing liveability and improving aquatic eco system health. However, just because we think it is the right decision(s) does not necessarily mean it is the ‘best’ decision. For example, the most efficient decision would have been to ride it out, but this would have promoted a wait-and-see response to climate that would not lead to efficient longer-term decisions and would not have gained other benefits of integrated water management
• 2012: A bit risky to leave it to the community, especially as they might take the ‘soft option’ of avoiding costly infrastructure (in this case it would have paid off but that is just luck). Perhaps some small-scale infrastructure investment would have been more sensible. 2020: Lucky again that, despite high rainfall for recent years, the community accepted a moratorium on growth/development to preserve the town's water supply
• The relatively low cost of a small but expandable desalination plant would have been a favoured option throughout unless mitigating factors (e.g. a lack of public/political support) had been found
• The decisions made would not have been perfect but the decision making process would rate highly. We did not run out of water so we get our annual bonus!!
• Satisfactory at both points in time
• From the 2030 perspective our decision making process was good at 2012 and 2020. It allowed contingency plans to be activated only if required and prevented investment too early
Figure 4

Responses at 2030 to the hypothetical situation illustrated in Figure 2 (i.e. Phase 3).

Figure 4

Responses at 2030 to the hypothetical situation illustrated in Figure 2 (i.e. Phase 3).

Surprisingly, given repeated calls (in earlier parts of the workshop, as detailed in Kiem et al. (2014)) by end-users for more precise and more accurate information when asked what they need to enable robust climate change adaptation, participants in the hypothetical exercise found that having more exact information would not actually alter their decisions (at least in the hypothetical situation used here). The implications of this are significant given the large amounts of time and money being spent on downscaling products and also given the number of end-users waiting for more specific information before they alter from ‘business as usual’ adaptation approaches. That is, more exact information will not necessarily enable better or easier decisions, nor will it necessarily lead to climate change adaptation beyond what is already being done. This is consistent with findings in other studies (e.g. Brown et al. 2011; Wilby 2011; Pielke & Wilby 2012) and is discussed below in the ‘Conclusions’ section.

The responses to the questions asking participants to determine if they would have made a better decision if provided with the information they thought they needed indicates again that it is not so much a need for more climate information as it is about needing to improve the packaging of available information and the communication surrounding it. Also important, and currently lacking, is information on how the projected changes to primary climate variables translates into changes in the things that actually matter to water resource managers. That is derived climate variables (e.g. flood risk, drought risk, reservoir inflows, bushfire risk, vegetation changes, water demand, ecological and environmental impacts) and non-climatic influences such as population growth, social acceptance, political contexts, and economics. This is similar to findings from previous projects (e.g. Hulme & Dessai 2008; Rickards 2012; Boezeman et al. 2013; Dewulf 2013; Kiem & Austin 2013b; van Enst et al. 2014; Gramberger et al. 2015; Kok et al. 2015) which demonstrate that successful adaptation to climate change is more than just dealing with climate impacts.

The comments in Figure 4 and Table 2 also highlight that no matter how good the science is or how good the climate change adaptation decision seems to be, unless the decision is robust and takes into account all climatic and non-climatic factors, implementation will be difficult (e.g. controversy and debate surrounding the recently adopted Murray-Darling Basin Plan (www.mdba.gov.au/what-we-do/basin-plan/development) and the proposed and subsequently rejected Tillegra Dam near Newcastle, New South Wales (http://majorprojects.planning.nsw.gov.au/index.pl?action=view_job&job_id=1687)).

### Lessons learned from the hypothetical case study – what information do Australian water resources managers think they need to make decisions on climate change adaptation and how does this relate to the information currently provided by climate science

To determine the priority actions emerging from the hypothetical case study, participants were presented with a list of issues that arose throughout the exercise and were asked to vote on what they felt were the most important and second most important – voting was two votes per person (with voting for the same thing twice not permitted) submitted confidentially through iMEET! Table 3 lists the issues that were voted on sorted to indicate the most popular (i.e. highest priority) issues at the top.

Table 3

List of key issues arising from the hypothetical case study ranked according to number of votes

Number of votes Key issue to arise from the hypothetical case study
12 The need for ‘knowledge broker’ to fill the space between developers of climate science information and application of that information by end-users
The requirement for dialogue between providers and end-users to bridge the gap effectively. This is an extremely complex, time-consuming and resource intensive task which has not been factored into any plans or strategies
Greater consideration of baseline risk and accounting for non-stationarity when developing climate projections
Improved packaging of climate projections (e.g. climate futures)
Improved understanding of natural variability drivers and impacts and how that might change in future
More focus on tools and methods to integrate between projections and decision making
Continue open and frank dialogue between scientists and end-users in all climate projects
Better communication of climate science – not just better PowerPoint slides or glossy brochures, but delivery of practical information to end-users and feedback to climate scientists regarding the end-user needs
To better understand how decisions are made
Improved capacity to deal with wide diversity of end-users (each of which need a different approach and the science community has no capacity to deliver this)
More focus ($$) on attribution of current/recent/historical extremes Focus on plausible scenarios rather than more precise information Black swans … what to do? Ignore and hope for best? Accept the gap is real and unrealistic to expect that to close … what now?? More focus ($$) on downscaling
More focus ($$) on next round of GCM outputs (e.g. CMIP6) Identification of plausible regional adaptation options More focus ($$) on GCM/RCM model selection/evaluation
More focus ($$) on emission scenarios Insights/quantification of relative importance of different sorts of uncertainties (including non-climatic) Number of votes Key issue to arise from the hypothetical case study 12 The need for ‘knowledge broker’ to fill the space between developers of climate science information and application of that information by end-users The requirement for dialogue between providers and end-users to bridge the gap effectively. This is an extremely complex, time-consuming and resource intensive task which has not been factored into any plans or strategies Greater consideration of baseline risk and accounting for non-stationarity when developing climate projections Improved packaging of climate projections (e.g. climate futures) Improved understanding of natural variability drivers and impacts and how that might change in future More focus on tools and methods to integrate between projections and decision making Continue open and frank dialogue between scientists and end-users in all climate projects Better communication of climate science – not just better PowerPoint slides or glossy brochures, but delivery of practical information to end-users and feedback to climate scientists regarding the end-user needs To better understand how decisions are made Improved capacity to deal with wide diversity of end-users (each of which need a different approach and the science community has no capacity to deliver this) More focus ($$) on attribution of current/recent/historical extremes
Focus on plausible scenarios rather than more precise information
Black swans … what to do? Ignore and hope for best?
Accept the gap is real and unrealistic to expect that to close … what now??
More focus ($$) on downscaling More focus ($$) on next round of GCM outputs (e.g. CMIP6)
Identification of plausible regional adaptation options
More focus ($$) on GCM/RCM model selection/evaluation More focus ($$) on emission scenarios
Insights/quantification of relative importance of different sorts of uncertainties (including non-climatic)

Red indicates comments related to improved communication and packaging of climate information. Blue indicates comments related to needs for better understanding and quantification of baseline risk, natural variability and non-stationarity. Green indicates comments related to development of tools and methods to integrate between projections and decision making.

Table 3 indicates there were three primary issues that participants felt were important to address in order to enable better decision making under uncertainty and to bridge the gap between climate change adaptation recommendations and successful implementation at practitioner level:

1. Improved communication and packaging of climate information (note comments related to this are highlighted in red in Table 3). Participants commented that this was not just more glossy brochures and presentations by climate scientists, rather the role of a ‘knowledge broker’ was identified that could operate in the space between the two parties. The role of the ‘knowledge broker’ would be to package, translate (both ways) and transform climate information, a result and recommendation that is in line with other recent studies (e.g. Dilling & Lemos 2011; Lemos et al. 2012; Rice et al. 2012; Kiem et al. 2014, 2015).

2. A better understanding and quantification of baseline risk, natural variability and non-stationarity (note comments related to this are highlighted in green in Table 3). This issue arose on multiple occasions throughout the hypothetical case study and it is clear that guidance needs to be developed in order for end-users to integrate this into their climate impact assessment and adaption processes – in particular how the natural variability might change.

3. Development of tools and methods to integrate between climate change projections and decision making, and the social, political, and environmental contexts and level of acceptance associated with any decision (highlighted in blue in Table 3). This issue is the focus of a parallel project funded by NCCARF ‘Understanding end-user decisions and the value of climate information under the risks and uncertainties of future climates’ (refer to www.nccarf.edu.au/content/decisions-under-climate-risks and www.adaptation-decisions.com for further information).

Advances towards solving the first problem may be achieved via a collaborative effort between end-users and scientists to first define a model for this ‘knowledge broker’ (in particular define the role, form and communication methods of this entity) and then move forward in applying this model (through attracting funding, setting up committees, etc.) – this is not a new concept (e.g. Dilling & Lemos 2011; Lemos et al. 2012; Rice et al. 2012) but is something that has proven difficult to instigate and sustain in Australia (see Kiem et al. (2014, 2015) for further details).

The second issue relating to baseline risk, non-stationarity and variability requires advances both in the research and the translation of the research into practical guidelines. At this point in time we still do not completely understand what causes hydroclimatic variability in Australia or indeed the full range of variability that we can expect (Power et al. 2005; Verdon-Kidd & Kiem 2010; Kiem & Verdon-Kidd 2011; Gallant et al. 2012). Further, in order to better define the baseline, case study participants agreed that additional research needs to be conducted into multi-decadal variability (which must include extension of the instrumental record via paleoclimate information). Finally this information needs to be translated into a series of recommendations that can be transferred to climate risk/impact studies (e.g. flooding, drought planning, infrastructure design, etc.). It is clear that this issue needs to be addressed via an emphasis on research in this field followed by the development of tools/methods to apply this knowledge and that funding should be directed in such a way. At present a large amount of climate-related funding is being focused towards generating regional projections and the next round of climate model outputs. However, only one participant actually voted that these were a critical issue compared to the other problems listed. While more precise or more accurate climate science information could be useful, the high priority issues and questions identified in Table 3 also need to be addressed (i.e. funded) to improve decision making under uncertainty and to decrease the gap between climate change adaptation recommendations appearing in the literature and what is actually happening on the ground.

During the hypothetical case study it was clear that uncertainty means different things to different people – with some scientists particularly stressing the difference between uncertainty, precision and accuracy. Water resource manager understanding of uncertainty may not have changed due to the hypothetical case study. However, some water resource managers felt they had gained ‘better clarity through technical definitions’. From a climate scientist perspective, while views on uncertainty also did not necessarily change, the importance of non-climate-related uncertainty was reinforced (and in some cases introduced). This is a satisfactory result as prior to the hypothetical case study water resource managers said that one thing that would make the exercise valuable is ‘an appreciation that climate change adaptation is based on more than just climate’. Based on the following comments given by scientists after the hypothetical case study it seems that this was achieved:

The hypothetical case study exercise exceeded expectations. It was thought-provoking and led to a better understanding of the issues facing decision makers, especially that the uncertainties are not just in the science’ (Scientist).

The hypothetical case study reinforced my view on uncertainty and reinforced that climate change uncertainty is not the only uncertainty and not always the most important’ (Scientist).

I still tend to view uncertainty the same as before the hypothetical case study with respect to climate change projections. However, I now recognise that there are political, social and demographic issues affecting decision makers that may be of equal or greater magnitude than scientific uncertainty’ (Scientist).

Another insight to emerge from the hypothetical case study was that water resource managers recognise that there is and always will be irreducible uncertainty associated with climate forecasts and climate change projections in Australia, and that effective climate change adaptation must be able to prepare for and manage Australia's highly variable climate. Australian water resource managers have been criticised for taking the stance that there is too much uncertainty in the climate science to implement major (i.e. expensive and in some cases controversial) climate change adaptation strategies. Further, it has also been suggested that delaying or forestalling a decision is a common means of dealing with uncertainty (e.g. Lipshitz & Strauss 1997). If we were to persist with the uncertainty-laden (e.g. Jones 2000) climate model based ‘predict-then-plan’ or ‘scenario-first’ decision making process then this is perhaps true – especially given the complexities of climate science, the many climate drivers and interactions that are not properly understood, and the errors/biases within and brevity (in the context of multi-decadal to centennial time-scales that climate change works on) of observed climate data sets. Therefore, as indicated in Table 3, there is a clear need to better understand, quantify and attribute the natural variability of the Earth's climate system (i.e. baseline risk, non-stationarity, natural variability). Computer models enhance understanding but cannot, for a system as complex as the Earth's climate system, be expected to provide realistic projections (let alone predictions) at the temporal and spatial scales useful for water resource managers – at least until our fundamental understanding of that system is improved and technological issues (e.g. computing speed, data storage and access, etc.) are overcome. Hence, current climate models provide a comparative analysis platform rather than a definitive projection platform. This makes climate models useful tools for evaluating the relative importance of various processes (or aspects associated with various decisions) but it must be recognised that this is very different to using climate model outputs as definitive futures and implementing adaptation strategies based on that. This study demonstrated that if these realities of current climate science modelling and understanding are comprehended by end-users, that it is possible to make decisions that proved, in the hypothetical case study conducted here, to be robust despite minimal information and high uncertainty at the time the decision had to be made. So again the need is not so much for more climate science information as it is for better use and communication of existing knowledge, as well as acknowledgement of what we do not know and the limitations of current climate science and climate model outputs.

## CONCLUSIONS

An important message emerging from this study was the need for clear, two-way communication between climate scientists and end-users. This reinforces the findings of many other studies (as outlined in the Introduction) that have demonstrated the importance of trans- and multi-disciplinarity, co-production of knowledge and the need for translating scientific knowledge into practical knowledge (and how best to do it). However, this result also suggests, at least from the perspective of water resources management in Australia, that the translation of science into practically useful information and successfully implemented adaptation strategies remains a significant issue and possibly a major reason why resilience and adaptive capacity levels are not showing any noticeable improvements despite the significant advances in climate science over the last decade. The results of this study (especially Table 3) clearly identify the need for continued dialogue between scientists and water resource managers, with scientists delivering information in a format that makes sense to water resource managers without ‘dumbing down’ nuances and uncertainties and water resource managers being given the opportunity to provide scientists with feedback regarding their needs. Water resource managers also need to be informed about what is feasible and what is not with respect to climate science now and into the future. Some scientists identified that this is an ‘extremely complex, time-consuming and resource-intensive task’, with a wide diversity of end-users, each of whom needs a different approach, and that the science community does not have the capacity (or incentives) to deliver on these needs. This is most likely true, but the need for ‘knowledge brokers’ and ‘technical/discipline translators’ to facilitate communication between climate scientists and end-users (e.g. water resource managers) identified here (and in many previous studies) is real – and resilience and adaptive capacity levels are unlikely to improve until this need is satisfactorily addressed (see also Kiem & Austin 2013a, 2013b; Kiem et al. 2014, 2015).

What was also clear from the hypothetical case study is that water resource managers are not actually seeking firm numbers at fine spatial and temporal resolutions at specified time horizons in the future in order to be able to make robust decisions. Participants agreed and accepted that definite answers about the future are not currently possible and that any future scenario or climate model projection will always be associated with significant uncertainty. However, much more certainty and more useful information is required about the risk water resource managers are accepting and the ability to recover if an uncertain event does occur (i.e. resilience) (e.g. Kiem & Verdon-Kidd 2011; Guillaume & Jakeman 2012). Alternative, ‘decision-led’ as opposed to ‘scenario-first’ approaches to assessing and dealing with climate risks do exist (e.g. Koutsoyiannis et al. 2008, 2009; Milly et al. 2008; Verdon-Kidd & Kiem 2010; Brown et al. 2011; Kiem & Verdon-Kidd 2011; Wilby 2011; Brown & Wilby 2012; Gramberger et al. 2015; Kok et al. 2015; Mortazavi-Naeini et al. 2015; www.mediation-project.eu/). These approaches, if investigated along with the key priorities emerging in this study (see Table 3), could improve our understanding of climate-related risk and at the same time improve the capabilities of climate models and their applicability to decision making and effective climate change adaptation. What is crucial is that effort is made towards ensuring that what scientists are certain about, or focused on, corresponds to what end-users and decision makers want to be more certain about – even if that is just quantification of the uncertainty, or a better understanding of the causes and structure of the uncertainty or clear statements about what we do and do not know.

## ACKNOWLEDGEMENTS

This work was carried out with financial support from the Australian Government (Department of Climate Change and Energy Efficiency) and the National Climate Change Adaptation Research Facility (NCCARF). Refer to www.nccarf.edu.au/publications/decision-making-under-uncertainty for full project details.

## REFERENCES

REFERENCES
W. N.
Arnell
N. W.
Tompkins
E. L.
2005
.
Glob. Environ. Change
15
,
77
86
.
Biesbroek
R.
Klostermann
J.
Termeer
C.
Kabat
P.
2011
.
Clim. Law
2
(
2
),
181
199
.
Blöschl
G.
Montanari
A.
2010
Hydrol. Process.
24
,
374
381
.
Boezeman
D.
Vink
M.
Leroy
P.
2013
.
Environ. Sci. Policy
27
,
162
171
.
Brown
C.
Wilby
R. L.
2012
.
Eos Trans. AGU
93
(
41
),
401
.
Brown
C.
Werick
W.
Leger
W.
Fay
D.
2011
.
J. Am. Water Resour. Assoc.
47
(
3
),
524
534
.
Burton
I.
1997
.
Clim. Change
36
,
185
196
.
Cash
D. W.
Clark
W. C.
Alcock
F.
Dickson
N. M.
Eckley
N.
Guston
D. H.
Jäger
J.
Mitchell
R. B.
2003
.
100
,
8086
8091
.
Dewulf
A.
2013
.
WIREs Clim. Change
4
,
321
330
.
Dilling
L.
Lemos
M. C.
2011
.
Glob. Environ. Change
21
(
2
),
680
689
.
Füssel
H. M.
2007
.
Sustain. Sci.
2
,
265
275
.
Gallant
A. J. E.
Kiem
A. S.
Verdon-Kidd
D. C.
Stone
R. C.
Karoly
D. J.
2012
.
Hydrol. Earth Syst. Sci.
16
,
2049
2068
.
Gramberger
M.
Zellmer
K.
Kok
K.
Metzger
M.
2015
.
Clim. Change
128
(
3–4
),
201
214
.
Guillaume
J. H. A.
Jakeman
A. J.
2012
Providing scientific certainty in predictive decision support: the role of closed questions
. In:
Proceedings 6th International Congress on Environmental Modelling and Software (iEMSs)–Managing Resources of a Limited Planet: Pathways and Visions under Uncertainty
.
Leipzig
,
Germany
.
Hammer
G. L.
Nicholls
N.
Mitchell
C.
(eds)
2000
Applications of Seasonal Climate Forecasting in Agricultural and Natural Ecosystems – The Australian Experience
.
,
Dordrecht
,
The Netherlands
.
Hayman
P. T.
2000
Communicating probabilities to farmers: pie charts and chocolate wheels
. In:
A Manual of Tools for Participatory R&D in Dryland Cropping Areas
(
Petheram
R.
, ed.).
Rural Industries Research and Development Corporation
,
Canberra, Australia
, pp.
133
136
. .
Hayman
P. T.
Crean
J.
Parton
K. A.
Mullen
J. M.
2007
Aus. J. Agric. Res.
58
,
975
984
.
Hegger
D.
Lamers
M.
van Zeijl-Rozema
A.
Dieperink
C.
2012
.
Environ. Sci. Policy
18
,
52
65
.
Ho
M.
Kiem
A. S.
Verdon-Kidd
D. C.
2015a
.
Water Resour. Res.
51
(
10
),
8362
8379
. Doi:10.1002/2015WR017058.
Ho
M.
Kiem
A. S.
Verdon-Kidd
D. C.
2015b
.
Water Resour. Res.
51
(
10
),
8380
8396
. Doi:10.1002/2015WR017059.
Hulme
M.
Dessai
S.
2008
.
Environ. Sci. Policy
11
(
1
),
54
70
.
Jacobs
K.
Garfin
G.
Lenart
M.
2005
.
Environ. Sci. Policy Sustain. Dev.
47
,
6
21
.
Jasanoff
S.
1990
The Fifth Branch: Science Advisers As Policymakers.
Harvard University Press
,
Cambridge
,
302
p.
Jones
R. N.
2000
.
Clim. Change
45
,
403
419
.
Kiem
A. S.
Austin
E. K.
2013a
.
Glob. Environ. Change
23
(
5
),
1307
1316
.
Kiem
A. S.
Austin
E. K.
2013b
.
Clim. Res.
58
,
29
41
.
Kiem
A. S.
Verdon-Kidd
D. C.
2011
.
Water Resour. Res
.
47
,
W00G06
.
Kiem
A. S.
Verdon-Kidd
D. C.
2013
The importance of understanding drivers of hydroclimatic variability for robust flood risk planning in the coastal zone
.
Aus. J. Water Resour.
17
(
2
),
126
134
.
Kiem
A. S.
L. E.
Sherval
M.
Verdon-Kidd
D. C.
Clifton
C.
Austin
E.
McGuirk
P. M.
Berry
H.
2010a
Drought and the Future of Rural Communities: Drought impacts and adaptation in regional Victoria, Australia
.
Report for the National Climate Change Adaptation Research Facility (NCCARF)
,
Gold Coast
,
Australia
. .
Kiem
A. S.
Verdon-Kidd
D. C.
Boulter
S.
Palutikof
J.
2010b
Learning From Experience: Historical Case Studies and Climate Change Adaptation
.
Report prepared for the National Climate Change Adaptation Research Facility (NCCARF)
,
Gold Coast
,
Australia
. .
Kiem
A. S.
Verdon-Kidd
D. C.
Austin
E. K.
2014
.
Clim. Res.
61
,
57
74
.
Kiem
A. S.
Verdon-Kidd
D. C.
Austin
E. K.
Moran
R.
Bates
B.
Hayman
P. T.
2015
A ‘knowledge broker’ to bridge the gap between end-user needs and information from climate science: an Australian perspective
.
Clim. Risk Manage
.
[submitted]
.
Kok
K.
Bärlund
I.
Flörke
M.
Holman
I.
Gramberger
M.
Sendzimir
J.
Stuch
B.
Zellmer
K.
2015
.
Clim. Change
128
(
3–4
),
187
200
.
Koutsoyiannis
D.
A.
Mamassis
N.
Christofides
A.
2008
.
Hydrol. Sci. J.
53
(
4
),
671
684
.
Koutsoyiannis
D.
Montanari
A.
Lins
H. F.
Cohn
T. A.
2009
.
Hydrol. Sci. J.
54
(
2
),
394
405
.
Kundzewicz
Z. W.
Stakhiv
E. Z.
2010
Hydrol. Sci. J.
55
,
1085
1089
.
Lemos
M. C.
Kirchhoff
C. J.
V.
2012
.
Nat. Clim. Chang.
2
(
11
),
789
794
.
Lipshitz
R.
Strauss
O.
1997
.
Organ. Behav. Hum. Decis. Process.
69
(
2
),
149
163
.
McKeon
G.
Hall
W.
Henry
B.
Stone
G.
Watson
I.
2004
Pasture Degradation and Recovery in Australia’s Rangelands: Learning from History.
Queensland Department of Natural Resources, Mines, and Energy
,
Brisbane
,
Australia
,
256
p.
McNie
E. C.
2007
.
Environ. Sci. Policy
10
(
1
),
17
38
.
Meinke
H.
Nelson
R.
Kokic
P.
Stone
R.
Selvaraju
R.
Baethgen
W.
2006
.
Clim. Res.
33
,
101
110
.
Meinke
H.
Howden
M.
Struik
P. C.
Nelson
R.
Rodriguez
D.
Chapman
S. C.
2009
.
Curr. Opin. Environ. Sustain.
1
,
69
76
.
Milly
P. C. D.
Betancourt
J.
Falkenmark
M.
Hirsch
R. M.
Kundzewicz
Z. W.
Lettenmaier
D. P.
Stouffer
R. J.
2008
Science
319
,
573
574
.
Montanari
A.
Blöschl
G.
Sivapalan
M.
Savenije
H.
2010
Getting on target
.
Public Serv. Rev. Sci. Technol.
7
,
167
169
.
Mortazavi-Naeini
M.
Kuczera
G.
Kiem
A. S.
Cui
L.
Henley
B.
Berghout
B.
Turner
E.
2015
.
Environ. Model. Softw.
69
,
437
451
.
Mote
P.
Brekke
L.
Duffy
P. B.
Maurer
E.
2011
.
Eos Trans. AGU
92
(
31
),
257
.
Parry
M. L.
Canziani
O. F.
Palutikof
J. P.
van der Linden
P. J.
Hanson
C. E.
2007
Climate Change 2007: Impacts, Adaptation and Vulnerability
.
Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change
.
Cambridge University Press
,
Cambridge
,
UK
, p.
976
.
Patt
A.
Dessai
S.
2005
.
Comptes Rendus Geosci.
337
,
425
441
.
Pielke
R. A.
Jr
2007
The Honest Broker–Making Sense of Science in Policy and Politics
.
Cambridge University Press
,
Cambridge
,
UK
,
198
p.
Pielke Sr
R. A.
Wilby
R. L.
2012
?
Eos Trans. AGU
93
(
5
),
52
.
Power
S. B.
B.
Nicholls
N.
2005
.
Bull. Am. Meteorol. Soc.
86
,
839
844
.
Randall
D. A.
Wood
R. A.
Bony
S.
Colman
R.
Fichefet
T.
Fyfe
J.
Kattsov
V.
Pitman
A.
Shukla
J.
Srinivasan
J.
Stouffer
R. J.
Sumi
A.
Taylor
K. E.
2007
Climate models and their evaluation
. In:
Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change
(
Solomon
S.
Qin
D.
Manning
M.
Chen
Z.
Marquis
M.
Averyt
K. B.
Tignor
M.
Miller
H. L.
, eds).
Cambridge University Press
,
Cambridge
,
UK
and
New York
,
USA
.
Rice
J. S.
Moss
R. H.
Runci
P. J.
Anderson
K. L.
Malone
E. L.
2012
.
17
(
7
),
805
819
.
Rickards
L.
2012
Critical Breaking Point: The Effects of Climate Variability, Change and Other Pressures on Farm Households
.
Report for the Birchip Cropping Group and the Sustainable Agriculture Initiative Platform Australia. www.bcg.org.au/cb_pages/publications.php
.
Sarewitz
D.
Pielke
R.
Jr
1999
.
Technol. Soc.
21
,
121
133
.
Shaw
A.
Sheppard
S.
Burch
S.
Flanders
D.
Wiek
A.
Carmichael
J.
Robinson
J.
Cohen
S.
2009
.
Glob. Environ. Change
19
(
4
),
447
463
.
Stainforth
D. A.
Allen
M. R.
Tredger
E. R.
Smith
L. A.
2007
.
Phil. Trans. R. Soc. A
365
,
2145
2161
.
Stephens
E. M.
Edwards
T. L.
Demeritt
D.
2012
.
WIREs Clim. Change 2012
3
,
409
426
.
van den Hove
S.
2007
.
Futures
39
(
7
),
807
826
.
van Enst
W. I.
Driessen
P. P. J.
Runhaar
H. A. C.
2014
.
J. Environ. Assess. Policy Manage.
16
(
1
). Doi: 10.1142/S1464333214500070.
Vance
T. R.
Roberts
J. L.
Plummer
C. T.
Kiem
A. S.
van Ommen
T. D.
2015
.
Geophys. Res. Lett.
42
(
1
),
129
137
. Doi:10.1002/2014GL062447.
Verdon-Kidd
D. C.
Kiem
A. S.
2010
.
J. Hydrometeorol.
11
(
4
),
1019
1031
.
Verdon-Kidd
D. C.
Kiem
A. S.
2014
.
Hydrol. Earth Syst. Sci.
18
,
2257
2264
.
Verdon-Kidd
D. C.
Kiem
A. S.
Moran
R.
2014
.
Hydrol. Earth Syst. Sci.
18
,
2235
2256
.
Wågsæther
K.
Ziervogel
G.
2011
.
Environ. Sci. Policy Sustain. Dev.
53
,
32
44
.
Wilby
R. L.
2011
.
Nat. Clim. Chang.
1
,
302
303
.
Ziervogel
G.
Downing
T.
2004
.
Clim. Change
65
,
73
101
.
Ziervogel
G.
Johnston
P.
Matthew
M.
Mukheibir
P.
2010
.
Clim. Change
103
,
537
554
.