Do direct and inverse uncertainty assessment methods present the same results ?

Hydrological models are simplified imitations of natural and man-made water systems, and because of this simplification, always deal with inherent uncertainty. To develop more rigorous modeling procedures and to provide more reliable results, it is inevitable to consider and estimate this uncertainty. Although there are different approaches in the literature to assess the parametric uncertainty of hydrological models, their structures and results have rarely been compared systematically. In this research, two different approaches to analyze parametric uncertainty, namely direct and inverse methods are compared and contrasted. While the direct method employs a sampling simulation procedure to generate posterior distributions of parameters, the inverse method utilizes an optimization-based approach to optimize parameter sets of an interval-based hydrological model. Two different hydrological models and case studies are employed, and the models are set by two distinct mathematical operations of interval mathematics. Findings of this research show that while the choice of the interval mathematic method can affect the final results, generally, the inverse method cannot be counted on as a reliable tool to analyze the parametric uncertainty of hydrological models, and the direct method provides more accurate results. This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying, adaptation and redistribution, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/). doi: 10.2166/hydro.2020.190 om http://iwaponline.com/jh/article-pdf/22/4/842/844265/jh0220842.pdf 021 Arman Ahmadi Department of Biological and Agricultural Engineering, University of California, Davis, CA, USA Mohsen Nasseri (corresponding author) School of Civil Engineering, College of Engineering, University of Tehran, Tehran, Iran and Department of Water Management, Faculty of Civil Engineering and Geosciences, Delft University of Technology, Delft, The Netherlands E-mail: mnasseri@ut.ac.ir


INTRODUCTION
Hydrological modeling, like any other physical and empirical process-based representation of a natural phenomenon, is a simplification of real-world systems; therefore, it always deals with inevitable uncertainty (Solomatine et al.  According to the equifinality thesis, proposed by Beven (), when assessing the uncertainty associated with prediction, rather than one correct answer, there are many acceptable representations of the system's reality that cannot be easily rejected and should be taken into account. In Trying to fill this gap, in the current article, two different interval mathematic formulations which come from fuzzy mathematics, namely SIM and modified interval mathematics (MIM) are applied to develop interval-based hydrological models. Then, by the means of GA, the best bounds of the models' parameters are calibrated as the output of the inverse models. In addition to the inverse method, GLUE is used to analyze the parametric uncertainty of the same models and to evaluate the inverse method's results. So, two specific objectives have been followed in the current article: to analyze the parametric uncertainty of two conceptual hydrologic models, each assigned to an appropriate case study, using direct and inverse methods; and to compare and contrast the results of these two methods and to examine the dissimilarities in their results and discuss the reasons for them.
In the following parts of the paper, first, the implemented direct and inverse methods are presented, followed by an introduction to the case studies and hydrological models. Then, the comparative results are presented and subsequently discussed and concluded. Figure 1 shows the flowchart depicting the methodology presented in the current research. The steps of the methodology are discussed in the following sections. The first step is the selection of the hydrological models and case studies that are presented in the following subsections (Step 1: Preprocessing).

Hydrological models
Two monthly water balance models are used in this research. The first model, which is referred to as the Guo model, was originally developed by Guo et al. (); it is a water balance model appropriate to model wet climate areas. The model consists of a surface water storage, without groundwater and subsurface flow. Precipitation and evapotranspiration are its input variables, and its output is total streamflow. The model has three parameters to calibrate, namely evapotranspiration coefficient (C ), initial soil water content (S1), and soil moisture scale factor (SC). The model structure is depicted in Figure 2(a). The second hydrological model, which is referred to as the enhanced Guo model with snowpack, is another monthly water balance model, first proposed by Guo et al.
() (without considering snowpack) and developed further by Nasseri et al. (b, ). Like the first model, this hydrological model employs water continuity and mass balance equations, but it also includes portions of snowmelt, soil water content, and groundwater on simulated streamflow. The model has ten parameters to calibrate, namely precipitation scale factor (SF), initial snowpack (SP(1)), minimum and maximum temperature criteria (T s and T m , respectively), surface runoff coefficient (K s ), snowmelt coefficients (K sn and K sn1 ), base flow coefficient (K g ), soil moisture scale factor (S max ), and evapotranspiration parameter (P 1 ). The model structure is illustrated in Figure 2

Case studies
The first case study of the paper is the Adour-Garrone basin    Table 1 reports some hydrological information on the case studies.

MODELING PROCEDURE
As depicted in Figure 1, the first step for implementing both direct and inverse methods is setting the range of model parameters (Step 2: Uncertainty Assessment). It is important to choose the same range for each parameter in different methods, which is indispensable for an impartial comparison of the approaches' results. The range of the parameters of each hydrological model is chosen based on the physical and mathematical characteristics and also the parameters' nature. The models' parameters and their ranges are presented in Table 2. In the following sections, direct and inverse uncertainty assessment methods are described.

Direct uncertainty assessment method
The selected direct method in this study is GLUE. After setting the range of the parameters, the GLUE algorithm runs a sampling simulation procedure to generate accepted samples according to an acceptance criterion. The prior distribution of all parameters is uniform distribution; therefore, it is assumed that there is not any prior knowledge about the parameters' intervals except for their lower and upper limits. The acceptance criterion is the Nash-Sutcliffe (NS) efficiency criterion, which is a very popular statistical metric in hydrological studies (Nash & Sutcliffe ): where Q obs i and Q cal i are the observed and calculated streamflow at the ith time step, respectively, m Q obs is the long-term average of observed streamflow, and N is the number of simulation's time steps. In the direct method, three different acceptance rates (NS ¼ 60%, 65%, and 70%) are considered; and for each, the GLUE algorithm is run to generate 50,000 accepted samples. Basically, an accepted sample consists of a set of parameters that are used in the hydrological model under simulation and generates the time series of streamflow that has an NS value more than the acceptance rate. All 50,000 accepted parameter samples along with their computed streamflow time series will be processed to achieve their posterior distributions.

Inverse uncertainty assessment method
The results of the inverse method to analyze parametric uncertainty are explained in an interval form for each parameter. In other words, it provides the optimized upper and lower bounds of the parameters instead of their distributions. In this research, a real-coded GA is used to optimize the hydrological models compatible with the interval mathematics. For all optimizations, the ranges of parameters are the same and equal to the direct method.
Moreover, GA parameters are the same for all optimizations, namely mutation probability, crossover probability, population size, and maximum number of generations which are 0.09, 0.8, 100, and 2,500, respectively. By setting a penalty for the fitness function, the GA optimization is conditioned to generate results with specific characteristics.
where Q þ i and Q À i are the upper and lower limits of streamflow, which are computed by the optimized hydrological model in the ith time step by applying the optimized set of parameters to the hydrological models, and Q obs i is the observed streamflow in that time step. Again, N is the number of observations.
The fitness function of the GA is normalized uncertainty efficiency (NUE), another metric to evaluate the performance of an uncertainty simulator (Nasseri et al. , a). In the formulation of NUE, POC along with average relative interval length (ARIL) is used. The formulations of ARIL and NUE are as follows: where A À and B À represent the lower values and A þ and B þ represent the upper values of the interval numbersÃ andB (as the fuzzy interval number), respectively.
The second concept is MIM which is similar to modified fuzzy mathematics (MFM), first proposed by Nasseri et al.
(, a). For example, the formulation of addition operation (þ) in MIM is as follows: Eventually, the final result is an interval and equals

RESULTS OF THE PARAMETRIC UNCERTAINTY ASSESSMENTS
As mentioned earlier, in both direct and inverse methods, all parameters are selected from a pre-specified range, which is the same for both methods. The parameters and their allocated ranges are presented in Table 2.
In Figures 5 and 6 Therefore, it can be inferred from Figures 5 and 6 that the inverse method is not able to identify not only parameters' distributions but also parameters' limits. On the other hand, the direct method can accurately provide the meaningful distributions of the model's parameters, causing a better understanding of the model's parametric uncertainty.
It can be seen in the figures that in some cases (e.g. parameter S1 in the first hydrological model, Figure 5(c)), the inverse method does not even calculate two limits for the parameter set, but it results in just one number. In other words, the lower and upper limits are the same. This happens for both standard and modified operations (Figures 5   and 6).
To investigate the uncertain parameter bounds resulted from direct and inverse methods and to compare them more accurately, Table 3 shows the POC. This amount represents the fraction of samples generated by the direct method that falls into the bound calculated by the inverse method ( Figures 5 and 6). As can be seen in Table 3, there is not any specific and concrete correlation between the bounds calculated by the inverse method and the histograms or parameters' distributions resulting from the direct method.
In Table 4, the results of implementing parameter sets optimized by the inverse method are presented. Three different metrics, namely ARIL, POC, and NUE, are employed to examine the fitness and accuracy of uncertainty bounds generated by the inverse method.
As mentioned in the methodology section, the inverse method implements GA to optimize the set of parameters with a specific threshold of POC (0.6, 0.65, or 0.7) as its optimization constraint; while its fitness function is NUE. Therefore, the algorithm tries to find the best set of parameters that maximizes the amount of NUE, while its POC metric is greater than or equal to the threshold. Table 4 shows that for both hydrological models reformulated with SIM, the GA is not able to find a suitable set of parameters and to satisfy the models' POC constraint. For the simple Guo, as the amounts of ARIL show, GA results in a very narrow bound, which cannot satisfy the constraint. The low value of ARIL and narrow bound can mathematically provide a relatively high amount of NUE (Equation (4)), but the result is not reliable at all because the lower and upper limits of streamflow uncertainty bound are equal or their difference is very close to zero.
In the second hydrological model reformulated with SIM, GA is able to satisfy the constraints, but the amount of ARIL is very high. Actually, the lower limit of streamflow bound is equal to zero (Figure 7(a)). Therefore, it can be inferred that in the case of SIM, the inverse method cannot provide an accurate and reliable uncertainty bound.
Looking at the results of both hydrological models reformulated by MIM, it can be understood that they can be counted on as reliable uncertainty bounds. In all cases, optimization constraint is satisfied and the amount of NUE is maximized with the best set of parameters (Figure 7(b)).

DISCUSSION AND CONCLUSION
In the current paper, to fill a research gap of comparative studies in the realm of uncertainty assessment methods, two frameworks of parametric uncertainty simulation, namely direct and inverse methods, have been described and evaluated.
To conduct this comparison, two hydrological models and two case studies have been employed. In addition, the GLUE uncertainty assessment method has been selected as the direct framework because of its feedforward information flow and its selection of the behavioral parameter sets considering the pre-specified acceptance rates.
To assess the parametric uncertainty using the inverse framework, an optimization procedure reformulated by interval mathematics has been used. To achieve the The current inverse framework resulting optimized pair of the lower/upper limits of parameter sets transferred the uncertainty assessment method from a probabilistic/possibilistic topic to a single-objective optimization problem. So, based on the results and statistical metrics, it can be concluded that the direct method can generally provide more reliable insight in different hydrological models and with different datasets than the current inverse uncertainty assessment framework.
As one possible future research idea, it seems that using multi-objective optimization to create a set of