Reproducing different types of changes in hydrological indicators with rainfall-runoff models

Hydrological indicators support analyses about the impact of climate and anthropogenic changes on riverine ecosystems. As these studies often rely on hydrological models for estimating the future value of the indicators, it is important to investigate how well, and under which conditions, we can replicate changes in the indicators. This study looks at these questions by investigating the performance that can be achieved depending on the objective function for calibrating the model, the direction of the change in the indicator, the magnitude of this change and the properties of the catchments. The results indicate that, in general, indicators describing the magnitude of discharge (monthly and annual) can be adequately estimated with hydrological models, but that there are difficulties when estimating the characteristics of flow pulses, flow reversals and timing variables. For some of these indicators, it is not even possible to correctly estimate the direction of large changes. The analysis showed further that these problems cannot be resolved by adjusting the calibrated parameters, but that the model structure is unsuitable for modelling these indicators. This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying, adaptation and redistribution, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/). doi: 10.2166/nh.2020.073 om http://iwaponline.com/hr/article-pdf/51/2/238/682115/nh0510238.pdf er 2021 Carolina Massmann Institute for Hydrology and Water Management (HyWa), University of Natural Resources and Life Sciences, 1190 Vienna, Austria and Department of Civil Engineering, University of Bristol, Bristol BS8 1TR, UK E-mail: carolina.massmann@boku.ac.at


INTRODUCTION
The expected economic and population growth rates, coupled with the effect of climate change, raise fears that the number and severity of water-related conflicts might increase in the future. When dealing with competing uses of water, it is necessary that societies agree on the water quality and quantity allocated to fulfilling different needs and that they define, based on these agreements, clear targets for managing this resource (Chopra et al. ).
The defined targets will depend, among other factors, on the current state and level of use of the aquatic resources as the closer the systems are to their natural state, the more likely that they will be managed for preserving ecosystems and their biodiversity (Acreman et al. a). Accurate assessments of the current state of river ecosystems are thus important for supporting the definition of management objectives. Furthermore, such assessments constitute the basis for estimating how ecosystems could be affected by future interventions or changes (Acreman et al. a), playing a prominent role in preparing for the future.
One important aspect these assessments need to consider is the health of river communities (i.e., plants, invertebrates and fish). However, due to the lack of available ecological data, such analyses are difficult to carry out (Acreman et al. b). This is remedied by resorting to indicators describing the flow regime and using them as proxies for the ecological state by assuming that these indicators are the main determinants of the ecosystem properties (Poff et al. ). This allows for comparing the values of the indicators, for instance, at two points in time, and concluding that there is no impact on the river ecosystem if these values do not vary significantly (Acreman et al. b).
While it is recognized that this approach leaves out important factors influencing riverine communities (e.g., water temperature, sediment load), it is not unreasonable to do so as these factors tend to be correlated with river discharge (Colby ; Sinokrot & Gulliver ; Acreman et al. b), which is regarded as the most important factor.
While the hydrological indicators can be directly calculated from discharge data, it is necessary to rely on databased statistical approaches or hydrological models when discharge data are unavailable. Data-based approaches have been widely used for estimating long-term hydrological indicators by establishing links between catchment properties (e.g., soil properties or average precipitation) and the hydrological indicators using, for instance, regional  Murphy et al. ), hydrological models tend to be better suited for carrying out more detailed analyses (e.g., annual variability) at more local scales (see Olsen et al. ), while statistical models might be preferred for obtaining a general picture or a first approximation of the long-term averages at more regional scales (see Carlisle et al. ). The present study ultimately aims at improving our abilities for estimating changes in the hydrological properties of streams and using this information for adapting to climate change. As this requires detailed local analyses, the study focuses on model-based rather than on data-based indicator estimates.
Previous studies investigating the factors influencing the quality of estimated hydrological indicators obtained with modelled discharge time series have focused primarily on the impact of different model calibration strategies and objective functions. This is not surprising since this is a practical question practitioners and researchers are often confronted with when working with more than one indicator as it has been observed that models tend to replicate the characteristics to which they are calibrated well, while An aspect that has not been thoroughly considered in this context is our ability for replicating changes in hydrologic indicators. This is somewhat surprising, given that the assessment of changes in river ecosystems is one of the main applications of hydrologic indicators and there is a vast body of literature looking at the performance of rainfall-runoff models in periods with different climate properties to those observed during the calibration period (Coron et  • Investigating if the performance for estimating changes in the hydrologic indicators varies with the direction and magnitude of the considered changes.
• Identifying possible links between catchment properties and our ability for modelling changes in the indicators.
A strength of the analysis presented here is that it considers a large number of catchments exhibiting considerable variations in climate and catchment properties for obtaining more robust and representative results.

METHODOLOGY Datasets and catchments
This study analyses 560 catchments located across the  (Table S1).

Indices of hydrological alteration
For keeping the study manageable, it was necessary to select a subset of hydrological indicators. One option would have been to build on a recent study identifying the indicators cor- indicators which attempt to cover a wide range of flow conditions (average, high, very high, low, very low) and all five regime descriptors (Table 1). While the information provided in Table 1 suffices for calculating most indicators, additional information might be necessary for understanding how the number of high flow pulses (HFPs) and low flow pulses (LFPs) was estimated. The approach used in this study consisted of taking the discharge time series for the 24 years considered in the analysis and estimating the 25th and the 75th flow percentiles. A pulse is then defined as an uninterrupted period during which the discharge is above the 75th percentile (HFP) or below the 25th percentile (LFP).

Estimation of changes in the hydrological indices
The steps that need to be followed for calculating the changes in the IHAs are ( Figure 1): • Calculate the annual value of the indicator using the observed discharge (IHA o ).
• Calculate the annual value of the indicator (IHA sim ) and the annual value of the Nash-Sutcliffe Efficiency (NSE) for each of the 40,000 simulated discharge timeseries. • For all combination of the four groups (g1, g2, g3, g4) calculate the observed and simulated changes in the indicators. This results in 12 cases (i.e., g1 to g2, g1 to g3, g1 to g4, g2 to g1 and so on). The observed changes in IHAs are calculated as the difference in the observed IHA between both considered periods.
For the modelled timeseries, two sets of changes are calculated. The first is based on the optimal Monte Carlo run with respect to the NSE, whereas the other set considers the optimal parameter set with respect to the IHA of interest.

Evaluation metrics
An essential aspect when assessing the performance of a model for reproducing changes is its ability to correctly reproduce the direction of change. This is assessed with the 'agreement in direction' (AiD) metric (Equation (1)): where IHA sim,g1 and IHA o,g2 stand for the simulated and observed IHA value in the groups specified by the subscripts g1 and g2. If the simulated and observed changes agree in their direction, the product will be positive and a will equal one. If the direction of change differs between the observed and simulated IHA, their product will be negative and AiD will equal zero.
It is further important to assess the magnitude of the  (2) and (3), respectively: where IHA o,md stands for the mean value of the observed IHA values in both considered groups.

Data analysis
It is expected that both the ability for estimating the direction and the magnitude of changes might vary depending on the magnitude and direction of the considered change.
For instance, it might be more difficult to get the direction of change right, the smaller the change is. The ability for correctly reproducing the magnitude of change might also vary depending on the type of change (i.e., if it corresponds to a wet-to-dry or a dry-to-wet situation and to a large or small change). The analysis was therefore carried out for different percentiles of observed IHA changes. While the plots on the right side agree better with the observed IHAs than the plots calibrated with the NSE, there is a con-      There is a tendency for the objective function that is best for modelling the direction of change to have also the best performance for modelling the magnitude of change. It is further seen that for the direction of change there is a tendency of having values closer to 50 in the central columns and more extreme values (either closer to zero or closer to 100) towards the edges. This indicates that the differences between the two objective functions are less pronounced for smaller than for larger changes. This relationship is, however, less evident for the magnitude of change.
Changes in the IHAs describing flow magnitudes (i.e., the first five indicators in Table 1)  The impact of the objective function used for defining the best parameter set is observed in Figure 4, which shows the results when using the IHA as objective function.
The most striking difference between these plots at the ones presented in Figure 3 is their asymmetry, which indicates that the results vary depending on the type of change (i.e., from high to low indicator values or vice versa), something which is not observed when the models are calibrated with the NSE.
How well can we estimate the magnitude of IHA changes?
The first four rows of plots in Figure 5 show Besides an analysis of the NE, it is important to look at the relative error as it might be easier to grasp the relevance of an error when comparing its magnitude, for example, to the mean value of the indicator. Another advantage of using the relative error is that it facilitates comparisons of model performance between indicators. The last three rows of plots in Figure 5 show the relative error for some indicators. The relative error for the indicators describing  Table 3 indicate that the errors for negative changes are larger than the errors for positive changes, whereas this is the other way around when the values in Table 3 are smaller than one. Since negative changes are encountered when the indicator goes from high to lower values, values above one in Table 3 indicate that changes can be better modelled when they are from low to high indicator values. Besides analysing the impact of the direction of change on the mean error (Table 3), it was investigated how the    The results were estimated for five different change percentiles and for the NSE and IHA calibrated models.   • It was found that our ability for modelling changes in hydrological indicators is correlated with our ability for modelling the corresponding indicators.

SUMMARY AND CONCLUSIONS
• The quality of the estimates of IHA changes depends strongly on the considered indicator. The hydrological model was able to provide reliable estimates of the direction of the changes for the IHAs describing the discharge.  • An analysis of differences in the quality of the IHA estimates depending on the direction of the changes showed that changes from low to high indicator values could be better modelled than changes from high to low values. This effect was more pronounced when the models were calibrated with respect to the IHAs. This has some implications when modelling the impacts of climate change as we will calibrate our models for the current climate and the use for estimating the discharge for dryer conditions. Since it was found that these changes (wet to dry) are more difficult to get right, it is important to avoid overestimating our ability for modelling dryer periods.
• An exception is found for the agreement in direction for the indicators describing HFPs and calibrated with the IHA, where the direction of changes could be best predicted for changes from high to low indicator values.
• An important strength of this study is that it relies on many catchments and that it considers different types of changes (with respect to the magnitude and direction of the changes). Such a stratification of the changes allows for additional insight. For example, we found that models are able to model adequately low IHA values of the indicators describing the discharge dynamics, but have problems in reproducing high values.

DATA AVAILABILITY STATEMENT
The primary data used in the study are publicly available from U.S. organizations. More detailed information about how it can be accessed is found in the text. Processed data are available from the author upon request.