Water resources data, models and decisions: international expert opinion on knowledge management for an uncertain but resilient future

Global water resources' systems practice, management and governance are confronted by serious challenges. Climate change, growing populations, degradation of ecosystems, competition among various users, as well as land use change, are noted as having major impacts on precipitation, evapotranspiration, subsurface water, surface water and basin geometry and, consequently, current and future water resources planning (Simonovic 2008). Since the 19th century, hydrometric monitoring and networks have been used to collect data on hydrologic variables such as rainfall, river levels and flows, evaporation, groundwater levels and other meteorological variables (Environment Agency 1999). A hydrometric network is thus defined as a group of data collection activities for different components of the hydrological cycle that are designed and operated to address a single objective or a set of compatible objectives (World Meteorological Organization (WMO) 1994; Mishra & Coulibaly 2009). Users of hydrometric data (hydrologists, climatologists, hydrogeologists, water resources managers, planners and researchers) utilise such data for pollution control, flood forecasting, irrigation and drainage issues, operation of dams and reservoirs, catchment, regional and transboundary water resources management, climate change research, environmental/economic/social impact assessments, fisheries and forestry management, hydro-electric power generation and infrastructure planning and design (Watershed Science Centre 2004; Mishra & Coulibaly 2009). River flow measurement occupies a central position in the practice of hydrometry, representing an integrated output of all the hydrological processes acting upon a particular river basin or catchment area. Such measurements can, therefore, be used to make informed decisions on a variety of issues by a range of stakeholders across a number of contexts (Dixon 2010; Hewett et al. 2010; Abdullaev & Rakhmatullaev 2014).

At the interface of the collection of hydrometric data and informed decision making are situated water resources models, which are sophisticated tools for analysing, evaluating, assessing, problem solving and decision support (Giustolisi & Savic 2006). Such models aim to facilitate easy understanding and projection of the possible outcomes of a project and the preferred alternatives in terms of management, planning, or policy-level activities. The demand for hydrometric data, analysis and modelling is on the increase due to wide ranging utility, coupled with escalating analytical capabilities and information distribution methods, driven by the need to make policy decisions across every sector – social, economic and environmental (Dixon 2010; Hannaford et al. 2013). The hydrometric information lifecycle thus comprises data/databases, models and decision-making processes and is summarised in Figure 1. The effectiveness of the system depends on whether the design and structure allows for continuous feedback (Marsh 2002). Additionally, the field of hydroinformatics, which combines all these elements, is a sociotechnical venture with the decision support systems it creates being subject to the needs and characteristics of social actors and arrangements (Abbott 2000; Abbott et al. 2001). Technology development and data availability expand the possible applications of hydroinformatics, but data must still be made into information, knowledge produced and managed; and then decisions made through either systems' automation or, where such automatic programming is not possible, with professional judgement (Babovic 2009; Gourbesville 2009).

For instance in the UK, the National Hydrometric Information Service serves as a conduit for the development of hydrometric information as its core function while providing feedback loops between data users and the institutions responsible for collecting hydrometric data (Centre for Ecology and Hydrology 2013). While well established and generally well maintained, there have been recent questions over the future of hydrology as a profession (Scott 2011) and changes to how the Environment Agencies, particularly for England and Wales, manage their hydrometric data and databases. For example, since April 2013 all duties and records have been transferred to separate teams and systems (National Archives 2012). These issues are not only seen in developed countries; in developing countries, a lack of institutional capacity coupled with a limited ability to gather essential hydrometric data due to a shortage of trained professionals and cuts to maintenance budgets (resulting in neglected or abandoned sites and networks) hinders water resources planning and decision making (Houghton-Carr & Fry 2006). In Central Asia, despite huge efforts to systematically address the data and knowledge gap, there remain technical, human and financial deficits, especially where the complexities involved are multi-dimensional (Abdullaev & Rakhmatullaev 2014).

In the case of Ghana, a number of governmental organisations have the responsibility for monitoring hydrological and hydrometric information, despite the Water Resources Commission (WRC) of Ghana having the mandate to manage the water resources of Ghana. Such organisations include the Directorate General for the Inventory of Hydraulic Resources or National Hydrological Services of Burkina Faso (DGIRH), the Hydrological Services Division (HSD, part of the Ministry of Works and Housing, collecting daily river flow data for ∼60 stations), SONABEL (Société Nationale d’électricité du Burkina), which monitors Bagré and Kompienga reservoirs (the inflow, the outflow, the evaporation and the inflow into the Bagré irrigation project) and the Volta River Authority (VRA, Ghana, which has information on the water releases from the Akosombo Dam).

With regard to Jamaica, the Water Resources Authority (WRA) is the body mandated to ensure the sustainability of Jamaica's water resources, through, among other responsibilities, the continual assessment and proper management of Jamaica's water resources (WRA 2011). The WRA established its data collection and resource management unit to provide hydrological data to guide and stimulate processes leading to decision making in relation to water resource allocation, conservation and protection, as well as to mitigate the impact of disaster conditions and, as such, monitors five flood warning systems across the island (WRA 2005). According to the Second National Communication of Jamaica to the United Nations Framework Convention on Climate Change (2011), there is a need for improving and rationalising the hydrometric network. More river gauges and more automatic weather stations are needed to aid in data collection and planning to reduce vulnerability, as well as to input to additional flood warning systems. Also, training and utilisation in proprietary software such as WaterWare (www.ess.co.at/WATERWARE/), RiverWare (www.riverware.org) and Mikebasin (now MIKE HYDRO Basin) (www.mikepoweredbydhi.com) is required to aid in the improvement of water management in Jamaica. A report on integrated water resources management in the Caribbean echoes these concerns asserting that:

and:

Building, creating or co-creating mathematical models and decision support systems, whether white, black or grey-box (Giustolisi & Savic 2006), requires acknowledging the need for stakeholder participation to advance not only technology creation and application, but also social justice in the water sector under increasingly uncertain scenarios (Abbott & Vojinovic 2010). Integrated water resources management requires tools and approaches that enable greater transparency in water management and governance and public awareness (Abdullaev & Rakhmatullaev 2014; Chen & Han 2016). This widens the focus of hydroinformatics, hydrometry, modelling and decision-making to include system qualities and quantities, properties and performance and big data (Abbott & Vojinovic 2010; Butler et al. 2016; Chen & Han 2016), necessitating interdisciplinary perspectives and academic-practitioner exchange (Hewett et al. 2010). In the context of the rise of resilience thinking and its application in the water sector (Hashimoto et al. 1982; Hamilton 2009; Blockley et al. 2012; Butler et al. 2016), these features potentially represent gaps in capability and provision across different locations, cultures and institutions, raising the issue of whether water resources systems can recover or ‘bounce back’ from or minimise consequences resulting from failures to meet levels of service due to the impacts of threats such as climate change, population growth or land use change. Additionally, are the issues, their impacts and the consequences perceived similarly across different countries, depending on the experiences of individual water resources managers (e.g., is there consensus)? Do differences in gaps, needs and priorities exist? Where big data does not exist, how can ‘basic’ data be used most wisely?

In order to answer these questions and contribute to providing clarity on such issues across the water resources arena, this paper summarises the results of a Delphi survey conducted with an expert panel focusing on hydrometric data, models and decision making. We believe this is the first time such an approach has been used across the countries of the UK, Ghana and Jamaica. The paper proceeds as follows. The Method section provides a detailed account of the background to the Delphi method and the purpose, participants and process adopted in this research. The Results and Discussion section summarises the main findings, recontextualising them in the literature and wider water resources picture. The Conclusion reiterates the main points and suggests recommendations for future focus and areas for reprioritisation.

The Delphi method has been used extensively to evaluate and progress theory in water management studies (i.e., De Loe 1995; Nagels et al. 2001; Taylor & Ryder 2003). The Delphi method can be defined as ‘a method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem’ (Linstone & Turoff 1975, p. 3). Three different Delphi techniques exist (Hiltz & Turoff 1993; Stitt-Gohdes & Crews 2004): the Policy Model, Trend Model and Structural Model. The Policy Model aims to discover the ‘for’ and ‘against’ arguments concerning differing resolutions for specific issues. The outcomes of such efforts are collaborative findings of consensus and divergence on particular topics (Hiltz & Turoff 1993). The Trend Model aims to identify the trends concerning the group and participants predict where they believe the trend will be in the future (Turoff 1970). Participants are asked to brainstorm potential assumptions and uncertainties to reflect on the process and ensure critical thinking. Finally, Structural Modelling encourages individuals to express judgements and uses them, independently, to create consensus on issues of interest (Stitt-Gohdes & Crews 2004). Needham & De Loe (1990) find the Policy Method to be well suited to discovering consensus and divergence in water management-related issues. Within this research, we follow a hybrid combination of the policy and trend models, as we examine the trends concerning the group into the future as well as expressing judgements to create consensus. Debate continues over the use of the Delphi method, which is beyond the remit of this paper; however, we respond to some of the criticisms by first providing a detailed explanation of the purpose, participants and process, as well as defining consensus, setting a level through which to quantify whether consensus was reached and detailing the data analysis approach undertaken.

The aim of the Delphi was to examine expert opinion on the limitations to practice of current capacities in and connections between hydrometric data/databases, water resources models and decision making and gain consensus on future priorities. We examined opinions on (1) current water management abilities, (2) importance of, access to and impact of access to different data types, models/tools and processes, (3) preference for trying new data collection techniques, models or decision-making processes, (4) types of threat faced by water resources systems, (5) priorities for future issues and (6) data/model/process-related interventions to address priorities (the topic titles are provided in Table 1). Funder requirements meant that developing countries required representation and therefore the authors used a snowballing technique with their existing contacts in the field of water resources management to identify participants in the UK, Ghana and Jamaica (four from each). The project timescale (six months) and the location of these experts necessitated the use of an online survey (anonymised to maintain a degree of objectivity) based on a consensus-building premise, for which the Delphi is perfectly suited. Methods such as focus groups or interviews may not have provided the preferred outcome of consensus/divergence building required to address the research aim.

The experts assumed several professional roles, roughly categorised as: academics in water management; professional consultants or employees at water companies; and government employees working directly with water management issues. Further details of the roles are omitted here due to anonymity and confidentiality. These individuals were selected based on their expertise and ability to contribute meaningful content to this stage of the research. Following the suggestions of Gibbs et al. (2001), experts were selected if they: published papers on the subject in the past five years in academic journals; taught university level courses on the subject; or, it was a primary function of their professional career. Some of these individuals assumed both professional and academic roles as defined above. However, every effort was made to ensure each country had a mix of these roles to reduce bias potentially caused by profession. While these precautions were made and no significant difference based on profession was observed between or within country, due to the sample size of this study, replication of the research with a similar mix of professions is recommended to support findings.

Members of the group were anonymous during the event and email was the main form of communication. To provide a clear account of the Delphi process followed, this research used the 14 recommendations of Garrod & Fyall (2005), which provide a clear process and were created through a review of the literature (i.e., not just one example). The process is summarised in Figure 2, where each step leads to the next, building towards a judgement to solve the issues being addressed. This was accomplished through three stages of sending questionnaires which were constantly updated to incorporate previous responses. A summary of responses was sent to each member between rounds and the questionnaires were stopped once consensus on the issue, or multiple issues, were found and further rounds would have yielded diminishing returns of insight.

A small pilot survey (n = 5) was conducted with professionals and academics working in this area of study. Feedback was collected to ensure questions were relevant and clearly stated the researchers' objectives. Survey questions for the first round of the pilot Delphi were created through review of the literature and the authors' water management experience. Less than 12 questions were presented to ensure a high level of participation. Additionally, participants were provided the ability to comment on each question to further express their opinion and collect qualitative data to support quantitative findings. In rounds two and three, participants were asked if they still agreed with their responses from the previous round after viewing the responses of their peers. Additional questions were added which were meant to address emerging trends from previous rounds.

A primary questionnaire was sent to and completed by 12 participating panel members, none of whom were included in the pilot. The pilot participants were not included, as they were already familiar with the nature of the Delphi, which would have biased results. Results were analysed and a second questionnaire was created. The second survey aimed to confirm findings from the first round and also realise potential solutions to issues raised in round one though examination of potential interventions to strengthen water resource management systems. Results from the first round, including all comments, mean scores, percentage agree, averages distinct to each country and the individuals' responses were shared with each participant. The second questionnaire was then sent and completed by all 12 members of the panel. Results from round two, similar to round one, were then sent to each participant. A final questionnaire, consisting solely of questions relating to the project and generated by panel members themselves was sent. This was done to engage participants in the co-creation of material.

Diamond et al. (2014) reported no agreement within the literature on the procedure for determining consensus or divergence in a Delphi study, although identify that percent agree is the most common measurement. In this research, percentage agreed was also used with a threshold of 75%. To further ensure no large discrepancies existed, all data were checked for a normal distribution as bimodal responses would represent distinct groups of disagreement, indicating a lack of consensus (Diamond et al. 2014). Additionally, qualitative data (participant comments) were collected for each section and the general nature of comments (positive, negative or mixed) for each question aided in determining the direction of consensus. This application of qualitative analysis in a Delphi study is similar to work by Holey et al. (2007). While some researchers (i.e., Johnson et al. 2009) have exclusively used quantitative data to determine consensus in similar research, the triangulation of both quantitative and qualitative data applied in this research was determined to provide a more encompassing understanding of panel members' responses. As some questions were not measured on a scale of agree or disagree, a different metric was needed. Instead, stability of scores between rounds was used to determine consensus. Ranking initiatives for priority was measured using a rank score corresponding to each individual participant's response. A score was calculated by assigning a value to a ranking (e.g., 10 for ranking an item first, 9 for ranking an item second, etc., depending on the number of options) for each individual ranking event and then adding those sums to give the item a score. Then, all scores were compared to determine which were ranked highest, most often. Stability was not a predetermined measure and, instead, only used after no change in prioritisation was seen between all three rounds, as recommended by Borden et al. (2017). Again, qualitative data were collected to confirm quantitative findings. All data were analysed using standard spreadsheet software (Excel).

To explore the areas of consensus and divergence in opinion among the panel, results were examined at a country level and then between rounds. After reporting the round one (R1) results on preferences for data, model and decision making types, the focus of the results is primarily directed towards round two (R2) findings, as it was at this point in the Delphi that consensus or divergence began to consolidate. We include examples of resulting graphics, pertinent to any significant findings, but primarily report the results using text due to the rich nature of the data collected during a Delphi. A summary of the country-level results is provided in Table 1, which highlights some of the consensus and divergence in opinion across the different contexts. For example, Ghanaian participants expressed their strengths were in modelling, but their weaknesses were in data/databases and decision making, whereas Jamaican participants felt weaker in modelling and UK participants felt strong in databases and modelling, but weak in data collection and decision making.

Ghana reported the lowest or equally lowest ability to manage all areas of the water management system (basic data, databases, modelling, planning and decision making) while the UK reported the highest or equally highest for every area. The following quotes from different participants summarise overall opinions in relation to each country's abilities to manage different areas of the water management process:

All countries agreed that every data type was important and within each country there was consensus on this issue. River flow data were consistently the highest scored type of data. However, each country then varied in their scoring of the importance of other data types. This may indicate that there was variation in the importance of different types of data for each country, as to be expected with varying geology and water availability. All countries reported the lowest level of access to infiltration data followed by groundwater/borehole. A high level of divergence was reported with only the UK showing consistent consensus on what was consistently reported as the highest level of access to all data types from the three groups. Outside of three options (in-house, GIS and regionalization), limited consensus was seen for models/tools/processes. There was a high amount of ‘I don't know enough … ’ responses for most models/tools, signalling that there are different processes/tools being used at different locations both within countries and between countries. With regard to trying new data collection techniques, models and decision-making processes, there was some consensus within countries, but limited consensus between countries.

A comment from a Jamaican participant reinforced the statements given in the Global Water Partnership's report (2014):

Future issues (capacity and planning under uncertainty) received the highest mean scores from the Delphi panel with policy tied for second highest. Ghana reported the highest scores for every category with Jamaica second in most but not all. The UK reported the lowest levels for each category except future systems capacity and planning for uncertainty. Why the UK reported moderately high numbers in these two categories is not resolved but could be due to older infrastructure or a ‘hyper’ concentration on preparing for the future compared to other countries. High levels of consensus were observed with a slight emphasis (highest mean scores for the total Delphi panel) on future capacity and planning for uncertainty but also policy and conflict resolution. Regarding threats faced by water resources systems, overall, climate change scored highest with population growth and political will tied for second. However, only moderate levels of consensus were seen, except for climate change and population growth where, again, high scores were consistently reported. Consequently, questions on trying new techniques, abilities and threats were carried over into R2 and questions on potential interventions related to data, models and decision making to address such issues were introduced.

GIS-based tools/processes were the most common response selected between and within all three nations, as highlighted in Figure 3. Ghana also reported a common use of multiple criteria analysis (MCA), although this was not reported in other countries. In Jamaica and the UK, limited similarities could be seen with regard to commonly used tools/processes. Comments revealed these results may be due to specific tools/processes depending on profession and availability within countries. Additionally, the use of Excel was also consistent across the panel. The following quotes from participants highlight these findings:

Regarding the trialling of new data collection, modelling or decision-making approaches, results between R1 and R2 revealed a shift to an overall desire to try new models instead of the previously desired new data collection techniques. However, interestingly, within countries no changes were seen. Ghana and the UK still preferred new modelling while Jamaica preferred new data collection techniques. Comments indicate differences between countries were mostly due to localised limitations, acknowledging that connections between data, models and decision making are crucial, but frustrated by issues beyond their control as to the robustness of each area:

In relation to future issues, resilience and sustainability, greater consensus was reported in R2 than in R1. Again, future capacity and planning for uncertainty received high scores. However, current capacity was also scored highly, indicating that the panel was not only highlighting concerns over future issues: current capacity ranked second in total mean and a high level of consensus was reported. As a result, understanding water resource capacity, current and future, were ranked first and second indicating a needed focus on these issues:

This echoes between-round results observed by Taylor & Ryder (2003), where convergence of opinion on information relating to critical reservoir fisheries enabled the identification of the most relevant areas to pursue. Similarly, in the present Delphi, these participant's comments enabled pin-pointing of the most pertinent topics to elaborate in relation to consideration of interventions.

Following on from this, five potential interventions were suggested: (i) standardisation of education and training provision across the water management sector, (ii) comprehensive review of the role/capacity of your country's water management organisations, (iii) development of new/review of existing policy relating to hydrometry/modelling/decision making and water management, (iv) increased collaborative research between nations/academics/practitioners/governments and finally (v) standardisation of tools/processes, where feasible, to enable greater within and between country sharing. Of these five interventions the highest scored were (iii) development of new/review of existing policy and (iv) increased collaborative research (Figure 4). Standardisation of tools (v) received the lowest support with one panel member identifying that standardisation can also stifle innovation:

In terms of uncertainty, this reluctance to standardise tools may be entirely appropriate as resilience requires flexibility of systems (however defined), as well as the capacity to adapt and learn from responses to threats or stresses (Butler et al. 2016). This is also in line with the antifragility concept proposed by Taleb (2012), where a system gains from responding to disorder rather than rigidity. Taken further by Babovic et al. (2017) specifically in relation to urban water infrastructure, antifragility is shown to support a management paradigm capable of delivering water systems that can respond to uncertain futures. Buurman & Babovic (2016) also echo similar points in their discussion of adaption pathways and adaptive policy making to enable flexibility and response to changing social preferences, as originally highlighted by Abbott et al. (2001). Additionally, Deng et al. (2013) provide evidence of the case-by-case value of flexibility in the design of urban water management systems through the application of a value-under-uncertainty framework (using the example of sustainable drainage systems in a catchment in Singapore). This finding potentially suggests panel members are implicitly considering resilience in their approaches to model application and consequent decision making. Considering this in the context of our other findings and the potential relevance of antifragility and adaptation pathways, it could be asserted that decision makers require support in exploring how to apply resilience/antifragility thinking and adaptive policy making in the real world of water resources management.

Consensus on supporting interventions were reported for all items except standardisation of tools (within the UK) and an additional intervention presented by the panel was an increased identification of funding sources for interventions. This was reported by several panel members from Jamaica and Ghana, further identifying a need for increased funding in these nations for water management, which was previously identified in R1:

Additionally, relating to the previously identified lack of political will, there were comments made that questioned the ability of current policy and decision makers to consistently align different policies on water resources management:

The final question of R2 asked the panel to pose a question that they would like the other participants to answer in R3. The types of question are succinctly summarised by the following question, which highlights many of the consensus themes identified throughout this Delphi:

Very few of the questions posed in R3 were directly responded to, instead generating further questions, and therefore at that point the Delphi was concluded. Overall, panel members reinforced findings by Dixon (2010) and Hannaford et al. (2013) highlighting an increasing trend in a need and desire for more data to make better informed decisions. Further, findings supported observations reported by Houghton-Carr & Fry (2006), which showed limited institutional capacity and funding leads to lower planning and decision making ability.

Aligning data, model and decision-making requirements taking into consideration uncertainty, resilience and sustainability, based on sociotechnical requirements is a complex task. To examine international expert opinion on the connections between these areas, we conducted a Delphi survey with an expert panel (n = 12). The research presented in this paper offers a strategic view on knowledge management to align connections among data, models and decision-making through identification of consensus areas for future focus. These areas predominantly include data types deemed important and that limited access to which impacts abilities to assess current and future capacities and plan under uncertainty. Recommended interventions to assist in relation to these priority areas included reviewing and developing existing and new policy and collaborative research. Additionally, dissensus areas for reprioritisation include the standardisation of tools and processes used throughout the water resources management process, highlighting that there is strength in diversity and no driver (within the context of this study) to reduce the variety of models and tools used within this field.

We would like to acknowledge the Delphi participants who gave freely of their time to provide their thoughts. We also thank the UK Engineering and Physical Sciences Research Council for funding the project through the University of Exeter's institutional Global Challenges Research Fund.