Abstract

Different hydrological models provide diverse perspectives of the system being modeled, and inevitably, are imperfect representations of reality. Irrespective of the choice of models, the major source of error in any hydrological modeling is the uncertainty in the determination of model parameters, owing to the mismatch between model complexity and available data. Sensitivity analysis (SA) methods help to identify the parameters that have a strong impact on the model outputs and hence influence the model response. In addition, SA assists in analyzing the interaction between parameters, its preferable range and its spatial variability, which in turn influence the model outcomes. Various methods are available to perform SA and the perturbation technique varies widely. This study attempts to categorize the SA methods depending on the assumptions and methodologies involved in various methods. The pros and cons associated with each SA method are discussed. The sensitivity pertaining to the impact of space and time resolutions on model results is highlighted. The applicability of different SA approaches for various purposes is understood. This study further elaborates the objectives behind selection and application of SA approaches in hydrological modeling, hence providing valuable insights on the limitations, knowledge gaps, and future research directions.

INTRODUCTION

The reckless and continuous anthropogenic activities of continuous nature adversely affect the quantity and quality of water resources, thereby altering various hydrological fluxes and feedback processes (Pielke et al. 1999; Rose & Peters 2001; Sivapalan et al. 2003; Jiang et al. 2012). Hydrological modeling is a mathematical representation of these hydrological processes which influence primarily the energy and water balance of a watershed (McCuen 1973). In general, models attempt to represent the complex spatially distributed interactions of water, energy, and vegetation by means of mathematical equations. Broadly, two types of models exist: conceptual and physically based hydrological models (HMs) (Beven & O'Connell 1982; Beven 1989; Madsen 2000). Regardless of the modeling approach, inevitably, all models are imperfect representations of reality, with each providing different perspectives of the system and approximating various physical processes with some constants and parameters. In most of the approaches, though, many parameters would be observable (e.g., basin area, slope, elevation, vegetation type), and a few would be unobservable conceptualizations of basin characteristics. Irrespective of the choice of models, the primary problem in any hydrological modeling is the uncertainty in the determination of these model parameters. Further complications arise owing to the mismatch between model complexity and the data which are available to parameterize, initialize, and calibrate such models (Troch et al. 2003; Ye et al. 2008; Gupta et al. 2012; Zhang et al. 2012; Song et al. 2015).

A sensitivity analysis (SA) to determine the possible values to be assigned to the parameters and the qualitative and or quantitative variations in the output of an associated model, should be an integral part of any hydrological modeling study (Vemuri et al. 1969; Saltelli et al. 2000). Despite SA being an unavoidable step in modeling, no unique definition exists; rather, there are different definitions depending upon the fields of application (Razavi & Gupta 2015; Song et al. 2015). In hydrological modeling, SA can be simply defined as the change in the output responses due to the change in one or more model inputs or parameters. However, considering the fact that the sensitivity of model parameters is only one of the objectives of SA, this definition is incomplete. One probable reason behind the absence of any generalized definition is the limited practice of SA methods as a tool to design and study the hydrologic models (McCuen 1973).

Critical examination of the relations between model inputs and outputs is essential, which may: (i) help to identify any potential deficiencies in model structure and formulation; (ii) provide guidance for model order reduction and parameterization; and (iii) analyze the information content of available observations (Castaings et al. 2007). SA methods, hence, help to identify the parameters that have a strong impact on the model outputs, thereby influencing the efficacy of any model. Dawdy & O'Donnell (1965) stated that ‘In general, the greater the sensitivity of the model response to a parameter, the closer and sooner will that parameter be optimized’ and concluded that the insensitive parameters often did not approach the true value. It is worthwhile to mention here that SA takes into account the effect of parameters as well as the uncertainties in model forcings (D'Agnese et al. 1999; Hill & Tiedeman 2006). Traditionally, SA has been carried out through manual calibration, which is very tedious and time-consuming (Boyle et al. 2000; Madsen et al. 2002). SA based on automatic calibration procedures can be classified into local and global search strategies (Sorooshian & Gupta 1995). While local approaches deal with assessing the effect of parameters on the output by varying each parameter, one at a time around any base case, global approaches assess the change in output by varying all the parameters simultaneously over the entire feasible range.

Past studies provide insight into the application of SA in various fields such as economics (Coyle et al. 2003), food safety (Frey & Patil 2002; Mokhtari & Frey 2005), groundwater (Mishra et al. 2009), chemical (Saltelli et al. 2005; 2012) and environmental modeling (Hamby 1994), etc. The application of SA methods in hydrological modeling, although very limited (Blasone et al. 2007), has been gaining attention in the recent past. Initially, SA in HMs was carried out using local, deterministic, gradient-based methodologies (Dawdy & O'Donnell 1965; Nash & Sutcliffe 1970; Johnston & Pilgrim 1976). Later, robust and effective global search procedures were evolved (Wang 1991; Duan & Gupta 1992; Sumner et al. 1997). In addition, recent works have highlighted hybrids of local and global approaches in which derivative-based methods are employed to obtain the distribution of parameter sensitivity across the entire feasible parameter space (Rakovec et al. 2014); the definition of feasible ranges has a distinct subjective component, however. Apart from analyzing the sensitivity of various parameters, the sensitivity of HM responses to spatial resolution (Bloschl & Sivapalan 1995; Bruneau et al. 1995; Finnerty et al. 1997; Merz et al. 2009) and to the uncertainties in the spatial rainfall patterns were also explored in the past (Obled et al. 1994; Arnaud et al. 2011).

A critical review of different SA and sampling methods is absolutely necessary, considering the facts below:

  1. First, most of the past reviews concentrate on global analyses approaches (Iooss & Lematre 2015; Song et al. 2015). Despite the advancements in these global approaches, local approaches are found to be beneficial in regional modeling studies, especially in semi-distributed/distributed hydrological modeling (Liang 1994; Rakovec et al. 2014). Thus, in addition to global approaches, local approaches need to be given more focus.

  2. Second, a rigorous classification of SA approaches by considering different physical aspects of modeling studies and the suitability of SA methods, simultaneously need to be studied. Since perturbations in each SA approach may vary depending on the distribution of model parameters (whether it is lumped, semi-distributed, or distributed), a critical review of each SA method taking into account the choice of HMs is of the utmost importance (Saltelli et al. 2008).

  3. Third, a review of SA and sampling techniques based on its application in different parts of the world (regional characteristics) has not been attempted by any studies so far. We found a strong affinity between SA techniques deployed and regions modeled (Spruill et al. 2000; Tucker & Whipple 2002). Such revelation will provide a platform for the modeler to adopt an appropriate SA method depending on the study area being modeled. This is particularly significant, considering the fact that the objectives, assumptions, and methodology to be taken up for a suitable SA technique are largely influenced by the hydrological and physical characteristics of the region to be modeled. This information may help in developing a more realistic representation of hydrologic linkages of the region through models and will help in saving time.

  4. Finally, a comprehensive review of different SA objectives and goals to be considered in hydrologic modeling, highlighting the importance of conducting such an analysis, is necessary to aid the modelers’ community.

Hence, the purpose of this paper is to critically review the existing local and global approaches of SA methods deployed in hydrological modeling. Emphasis is also placed on the categorization of the SA methods, which will facilitate an effective selection of suitable SA method. The importance of SA is highlighted in the section ‘Need for SA’. Various SA approaches are detailed in the section ‘Classification of sampling methods, SA approaches, and their pros and cons’ by binning the approaches based on the methodology, number of parameters, etc. The pros and cons of different approaches are also elaborated upon. The spatio-temporal dependency of HMs is discussed in the section ‘HMs and SA application’. Different key factors which should be considered in SA in hydrological modeling are highlighted in the section ‘Key factors to be considered in SA’. The regional dependence of SA and sampling approaches are detailed in the section ‘Regional dependence of SA methods and sampling strategies’. Finally, possible challenges which may be encountered while applying SA approaches are discussed in the section ‘Discussion and conclusions’.

NEED FOR SA

The study of sensitivity or SA serves a range of purposes depending upon the nature of application, of which a few are summarized below:

Determination of response surface

Response surface is a region or space that represents the possible changes in model response in accordance with the changes in model input parameters. In any modeling study, identification of the most sensitive area among the entire factor space is the key requirement from SA, in order to avoid any possible perfunctory analysis. The determination of response surface helps to reduce the parameter space by identifying the most influential parameters, and thus reducing the complexity of models. Various approaches to assess the response surface developed by past studies (Sorooshian & Arfi 1982; Faravelli 1990; Bauer et al. 1999; Isukapalli et al. 2000) primarily focused on the reduction of the total number of model runs to speed up the simulations and also on the interaction between the fluxes in order to closely replicate the physical system under consideration.

Determination of the most influential uncertain parameters

Ranking the influential parameters in the order of their relative contribution to the variability to the model responses, termed as factor prioritization, is one major objective of SA. Identification and ranking of the input parameters are significant in driving the model simulations close to the true value. Proper understanding about the effect of even small variations in input factors on model responses and its feedback effects on model processes will help to minimize the overall uncertainty (Kuczera & Parent 1998; Crosetto et al. 2000; Razavi & Gupta 2015; Song et al. 2015).

Determination of interaction between the parameters

Assessment of interactions between various parameters involved in model processes is necessary to reveal the nature of processes and the possible influence of one variable on any other variables (dependent or independent) (Norton et al. 2004). However, computation of interaction between parameters is often complex and limited because of the inherent non-linear nature of most of the HMs (e.g., Frey & Patil 2002; Saltelli et al. 2004; Nossent et al. 2011).

Concordance between the model and the physical system

Another aspect of SA is to understand the concordance between the model structure and the assumptions on which it is based. This will help to theoretically evaluate the processes of the relevant system and to assess the extent to which the realistic representation of the real world has been achieved in the model (Reiter 1987; Boyle et al. 2000).

Model simplification and identification of non-influential parameters

Another objective of SA is the efficient and optimum use of the model by removing the redundant parameters, thereby reducing the computation time and errors. The concept of over-parameterization, which is usually neglected in many modeling exercises, is critically addressed by a few studies (Spear & Hornberger 1980; Beven 1989; van Griensven et al. 2006). These studies highlighted the significance of the removal of many non-influential parameters in the model while parameterization/calibration improved model efficiency.

CLASSIFICATION OF SAMPLING METHODS, SA APPROACHES, AND THEIR PROS AND CONS

Approaches adopted for SA depend on the type of HMs to a great extent. HMs may be classified as lumped, semi-distributed, and distributed (Chow et al. 1988), with the distributed model allowing more variability of parameters over the entire feasible space. Lumped models are approximate representations of the system and are mostly inefficient in accurately replicating the real system since the parameters involved are lumped, i.e., area weighted averages, and do not represent the exact physical features of hydrologic processes (Beven 1989; Hughes & Beater 1989; Arnaud et al. 2011). Although distributed modeling provides a platform which allows the variability of all the parameters, it is highly complex due to the associated uncertainty in the determination of the parameters. SA will help to reduce the complexity and uncertainty and help to gain better understanding of the concordance between the output and the inputs in any hydrological modeling study.

The approaches commonly adopted to perform SA can be classified into two: local and global (Saltelli et al. 2004; van Griensven et al. 2006). However, this classification is crude and vague. The decision on the type of approach, i.e., whether global or local, and the requirement of sampling methods to screen the number of simulations primarily depends on the range of parameter space (King 2009). The classification of SA methods is a challenging task in its own way, mainly because of the contradictory nature of inherent assumptions in practical applications. Saltelli et al. (2004) highlights four important points which need to be implemented during SA: (i) ordering, (ii) fixing, (iii) mapping of parameters, and (iv) reducing the variability of output. Keeping all the above points under consideration, this study adopts a more meaningful classification of SA methods with an intent to provide a simple guideline for selection of the SA method in hydrological modeling exercises. Hence, in this paper, SA methods are placed in the following categories: (1) scale dependent and scale independent methods (Morris 1991; Saltelli 1999); (2) function dependent and function independent methods (Razavi & Gupta 2015); (3) derivative-based and non-derivative-based methods (Saltelli et al. 2004); and (4) single parameter and multiple parameter methods (Hamby 1994; Saltelli et al. 2008). Broad classifications with examples are given in Table 1. A brief description of each method is further discussed. Some SA approaches demand high computational requirements, hence selection of the appropriate sampling strategy can help in subsidizing this demand. Therefore, along with SA approaches, sampling methods are also described.

Table 1

Classification of SA methods and sampling strategies

S. no. Sub-class Method Description Common techniques (e.g.) 
1.1 Scale dependent Depends on the selection of time step or the amount by which the factor or parameter is varied EE method. OAT sampling 
1.2 Scale independent Independent of the step size Variance-based method. Monte Carlo sampling 
2.1 Function dependent Depends on the assumed functional form or model OAT sampling. FF sampling 
2.2 Function independent Devoid of any assumptions regarding function or model Variance based method. EE method 
3.1 Derivative based All local approaches are derivative based. Less computation time All local approaches: Monte Carlo. OAT sampling 
3.2 Non-derivative based All global approaches which allow interaction between parameters All global approaches: LH method 
4.1 Single parameter Only one parameter is involved Random values. Stratified sampling. Mean and variance estimates 
4.2 Multiple parameter More than one parameter is involved in model formulation OAT sampling. Multivariate stratified sampling 
S. no. Sub-class Method Description Common techniques (e.g.) 
1.1 Scale dependent Depends on the selection of time step or the amount by which the factor or parameter is varied EE method. OAT sampling 
1.2 Scale independent Independent of the step size Variance-based method. Monte Carlo sampling 
2.1 Function dependent Depends on the assumed functional form or model OAT sampling. FF sampling 
2.2 Function independent Devoid of any assumptions regarding function or model Variance based method. EE method 
3.1 Derivative based All local approaches are derivative based. Less computation time All local approaches: Monte Carlo. OAT sampling 
3.2 Non-derivative based All global approaches which allow interaction between parameters All global approaches: LH method 
4.1 Single parameter Only one parameter is involved Random values. Stratified sampling. Mean and variance estimates 
4.2 Multiple parameter More than one parameter is involved in model formulation OAT sampling. Multivariate stratified sampling 

Random value sampling

In random value sampling, also referred to as non-uniform sampling, the value of a parameter is selected randomly within its range (say, varying from X1 to XN). This may create different densities of values in different intervals (Saltelli et al. 2008), with the possibility of too many values falling in a short interval and too few values in other regions. However, given sufficient sample size, random sampling generates unbiased estimates of mean and variance, since, as the sample size increases, the cluster of values taken by a parameter balance out and spread out in the entire range.

Random sampling has found its application in various fields. Price & Dixon (1987) developed two versions of controlled random search procedures for computer-aided design workstations, which is applicable in other fields also. Brazil & Krajewski (1987) compared a uniform random search and adaptive random search (ARS) procedure for the modeling of Sacramento Soil Moisture Accounting (SAC-SMA) model and concluded that the ARS method is an improvement over the uniform random search approach. A major disadvantage of random search procedure is that it does not consider the nature of response surface, hence making the search more irrational.

Stratified sampling

Stratified sampling reduces the computation time by rapidly converging the calculated values close to the observed value (McKay et al. 1979; Thomas & Lewis 1995; Christiaens & Feyen 2002). In stratified sampling, screening is done systematically in such a manner that each subinterval of equal length contains the same number of sample points. As compared to random sampling, stratified sampling is more ‘process-based’ and is easier to implement.

Hydrological response unit (HRU)-based and index-based are types of stratified sampling method. HRU-based sampling provides effective stratification of a watershed by forming heterogeneous units each having common characteristics with respect to their hydrological dynamics (Flügel 1995). As HRU-based stratification does not consider any topographical information, it is found to be effective only in sampling parameters like soil moisture and soil water (Ali & Roy 2009). In the index-based method, a topographic index is calculated to illustrate the characteristics of different source points. Index-based methods are usually not preferred in spatially high resolution studies, since the estimation of index is highly susceptible to the resolution of data (Zhang & Montgomery 1994; Beven 1997). The topographic index (λi) is given by:  
formula
(1)
where is the upslope-drained area to a point i, and is the local slope angle at that point (Beven & Kirkby 1979).

Monte Carlo sampling

In Monte Carlo sampling, screening of input parameters are done on the basis of their distribution function. Monte Carlo estimate of function f(x) is equal to the average value of f(x) over n points selected at random from the entire space. Since long records of data can be generated, the results from Monte Carlo sampling are statistically valid (Krajewski et al. 1991). Monte Carlo sampling is flexible and easy to put into practice, which is evident from its wide range of applications in various areas, e.g., study of hydrologic effects of spatial variability of rainfall (Smith & Hebbert 1979), ecological modeling (Annan 2001), distributed catchment modeling (Binley et al. 1989a, 1989b; Krajewski et al. 1991), and subsurface studies (Sharma et al. 1987). Despite its ease of application, one of the drawbacks of Monte Carlo is that as the number of parameters increases, a large number of simulations are required to get precise results (Boyle et al. 1997). Another difficulty in the implementation of Monte Carlo sampling is the assignment of input ranges and derivation of linked probability distribution functions (PDFs). Many times, model parameters are either not directly measurable or it would be cost prohibitive to collect adequate random samples of input to derive the true PDFs and ranges (Annan 2001; Muleta & Nicklow 2005).

Latin hypercube sampling

As described above, Monte Carlo and derived approaches are ineffective when modeling involves a large number of input parameters. The concept of the Latin hypercube (LH) simulation (McKay et al. 1979; Iman & Conover 2007) has been developed to obviate this limitation. In LH sampling, the entire factor space is divided into P equal intervals with probability of occurrence equal to 1/P using stratified sampling. Random values of the parameters are generated such that sampling is done only once for each parameter and for each interval. This approach results in P independent simulations, with model run of P times (van Griensven et al. 2006). This method simultaneously allows for the sampling of all parameters and gives direct measure of the distribution of output.

The diverse areas where LH sampling is employed are: water quality modeling (Melching & Anmangandla 1993), watershed modeling (Melching 1995), and distributed hydrological modeling (Christiaens & Feyen 2002). Many studies have also compared LH sampling with different methods such as differential analysis, factorial design (McKay et al. 1979; Iman & Helton 1985; Kleijnen 2005) and claimed that LH sampling produces reliable results. Numerous studies have also employed LH-one-at-a-time (LH-OAT) method in hydrological modeling studies using the Soil and Water Assessment Tool (SWAT), in which the OAT method is repeated for each point sampled using LH sampling (Holvoet et al. 2005; van Griensven et al. 2006; Mulungu & Munishi 2007; Ndomba et al. 2008). In particular, LH sampling is preferred in distributional HMs due to its efficiency in dealing with the curse of dimensionality.

Log-odds ratio estimates

The association between model simulation and real observation can be easily studied by estimating the log-odds ratio (LOR). LOR aids the modeler in factor prioritization and is adopted when the output is expressed in probabilistic terms (Frey & Patil 2002). The log of ratio of the probability of the occurrence of an event to the probability of the non-occurrence of an event is called LOR. Due to the ease of understanding, LOR is considered as the ideal conversion of probability (Christensen 2006). The outputs can be examined by calculating ΔLOR, which is given by:  
formula
(2)
where is the probability of the occurrence and is the non-occurrence of an event when the input parameter changes by some amount. is the probability of the occurrence and is the non-occurrence of an event when there is no change in the input parameters.

LOR gives the measure of dependency of the output on the input. Positive LOR indicates higher probability of occurrence of an event while negative LOR indicates very less probability of occurrence. Stiber et al. (1999) states that the effect of a particular input on output is proportional to the magnitude of LOR estimate. LOR estimates have been widely used in hydrology (e.g., (1) rainfall analysis (Hughes et al. 1999; Mackay et al. 2001); (2) forecast studies (Stephenson 2000)). LOR considers the cross-correlation between dependent (e.g., daily rainfall depths, stream-flow patterns, etc.) and independent (e.g., rainfall occurrences, precipitation, soil depths, etc.) variables. Despite its wide application, this method does not give any information related to the interaction between the inputs and is therefore difficult to implement in non-linear models.

OAT method

In this method, as the name itself implies, the effect on response or output is judged by varying one input parameter at a time from the pool of sensitive parameters, while keeping the other parameters constant. OAT design is suggested in studies where spatial variations are more prominent than temporal variations. Although OAT sampling is efficient in generating trusthworthy results for factor prioritization and is endorsed by many past studies, its applicability is questioned by a few studies. For modeling exercises with a large number of parameters and if only a few are influential parameters, then OAT sampling is prone to produce uncertain results (Saltelli et al. 2008). The reliabilty of sensitivity results for a particular input variable is largely dependent on the value chosen for other model parameters. In addition, sensitivity results in OAT sampling largely depend on the scale by which the parameter is varied (Razavi & Gupta 2015). The interaction between different inputs is largely ignored (Saltelli 1999) in OAT sampling. Saltelli & Annoni (2010), in their article titled ‘How to avoid a perfunctory SA’ illustrated the inadequacy of OAT through a novel geometric proof, thus providing a realistic and state-of-the-art argument against OAT to the modeling community. Methods like Monte Carlo, elementary-effect, linear regression, and factorial design are improvements over OAT sampling (Saltelli 1999; Crosetto et al. 2000; Campolongo et al. 2007; Dimov & Georgieva 2010; Saltelli et al. 2010; Shin et al. 2013).

Fractional factorial design

Fractional factorial (FF) design (Fisher 1942) considers a fraction of total number of possible combinations of input factors in order to reduce the number of experimental runs. The overall effect and also the effect of possible interactions are estimated by grouping the entire factor space into different ‘levels’. The indices are calculated for each combination of levels. Hence, a full factorial design having k number of factors has nk runs, where n is the number of levels. Therefore, the FF method suffers from the curse of dimensionality and is efficient only when the number of factors is small. When sample size is moderate or large, the three-way and higher order interactions are proved to be insignificant (Kleijnen 2005). FF design is proposed for moderate and higher dimensional factor space with nk−p runs where 1/np is the size of the fraction. FF design has been employed in many past studies (Henderson-Sellers 1993; Cryer & Havens 1999; Xu & Gertner 2007; Meng & Quiring 2008). For detailed information of FF sampling, readers are requested to refer to Saltelli et al. (1995) and Kleijnen (2005).

Variance-based approach

Variance-based technique is a modified sampling-based technique, in which quantification of the partial conditional variance of the model outputs is carried out. The model output is reduced based on a fixed set of parameters. For example, consider a HM expressed in the form where the vector Y denotes the m number of outputs and vector X denotes the n inputs to the model and is the vector of k model parameters. Then the total variance of the model output can be expressed as:  
formula
(3)
where denotes the conditional variances expressed as , and so on.

The first term Vi represents the main effect of βi, i.e., the variance of the expected value of the output, conditioned on the input parameter βi. The second term represents the interaction between βi and βi, i.e., the variance of the expected value of the output, conditioned on the input parameter βi.

Now, first order sensitivity of ith parameter is determined by computing the variation in the model output by keeping the ith parameter constant and resampling the other input parameters. The first order sensitivity index of order k is hence defined as:  
formula
(4)
The presence of the interaction or the interdependence between the model parameters can also be judged by calculating the total effect index, which is the combination of first order sensitivity of the ith factor and its interaction with the other sensitive parameters. The total effect of the ith parameter is measured by estimating the variation of the model output by resampling the ith parameter while keeping the other sensitive parameters constant. Total effect sensitivity index is expressed as:  
formula
(5)
Computation of each term in Equation (5) is a cumbersome task. In order to overcome this, another expression of total effect index (Saltelli et al. 2008) is:  
formula
(6)
Variance-based methods consider the entire range of parameter space and also their interactions. However, for semi-distributed models, such an approach may lead to complexity because of the existence of a large number of parameters and their interactions. Therefore, prior investigation is needed to decide the applicability of this method for SA in semi-distributed HMs.

Saltelli et al. (2008) states that total effect index is always greater than or equal to the first order index of a particular factor. However, it is contradicted by the fact that in the case of correlated factors, total effect index may take a value smaller than the first order sensitivity index (Pappenberger et al. 2008). Many times, variance-based method is preferred over other methods since it is devoid of any model function and can simulate interaction effects efficiently. Variance-based method has been adopted in various hydrological modeling studies, such as distributed modeling, rainfall–runoff modeling, etc. (Lilburne et al. 2006; Sobol & Kucherenko 2009; Saltelli et al. 2010; Zhan et al. 2013; Rakovec et al. 2014). An extension of this approach is also adopted in multi-objective SA (Gupta et al. 1999; Bastidas et al. 2006; Pappenberger et al. 2008; van Werkhoven et al. 2008a; Wagener et al. 2009). The major drawback of the variance-based method is that it requires considerable computation time (model runs), which in turn, demands a viable, efficient, and robust sampling technique for its implementation.

Elementary effect method

Elementary effect (EE) method (Morris 1991) is known for its ease of application and computational benefits making it suitable for computationally demanding modeling exercises (Shin et al. 2013). Unlike derivative-based method, it allows larger variability of the inputs and follows the concept of local variation around base point. A sensitivity measure is ultimately derived, which is an average of many local measures over the entire input space. This method is found to be efficient when the number of input parameters are large (Saltelli et al. 2008). The EE method was first introduced to examine the significance of each factor individually by ranking them in the order of their importance. This method is model independent and gives a qualitative measure of sensitivity.

Consider a function F(X) having n independent input parameters . The entire factor or input space is discretized into k levels. The EE of the ith input factor with Δ change, is given by:  
formula
(7)
where Δ takes a value in .
The two sensitivity indices for EE method, viz., the mean and standard deviation, is given by:  
formula
(8)
 
formula
(9)
where r is the number of EEs estimated for each input factor.
The mean μ accounts for the total effect on the output responses and standard deviation σ measures the interaction among different input parameters. Campolongo et al. (2007) substituted the measure μ with μ* to neutralize the effects of opposite signs, especially when the model is non-monotonic. μ* is able to simulate the total sensitivity measure by working with a group of factors, i.e., the case when more factors are moved at the same time, factors are first considered in groups. The modified mean μ* is defined as:  
formula
(10)

Higher indicates that the parameter i significantly influences the model response, whereas higher values indicate that the interaction of i with other parameters is significant. Campolongo et al. (2007) further states that μ* is a good alternative of the total sensitivity index ST. Sobol & Kucherenko (2009) suggest that the assessment of global sensitivity for the individual factors or groups of such factors can be improved by estimating mean of squares.

Each sensitive parameter is resampled once to compute the model response. The EE of each parameter is then computed by deducting the model response from the responses generated due to change in the ith parameter and normalizing by the amount with which the ith parameter is changed. These EEs are calculated for each sensitive parameter and sensitivity indices are estimated. EE method has found wide applications in the field of hydrology, such as studies related to runoff generation using Distributed Time Variant-Gain Model (DTVGM) (Zhan et al. 2013), Xinanjiang model (Song et al. 2012; 2013), and MIKE/NAM rainfall–runoff model (Liu & Sun 2010). Also, in water quality modeling studies, the EE method proved its efficacy in finding out the optimum repetition number of the EEs (r) of the method (Ruano et al. 2011). Further, Rakovec et al. (2014) combined the EE method effectively with the variance-based and regional SA methods, termed as distributed evaluation of local sensitivity analysis (DELSA) method and demonstrated its application to a simple non-linear reservoir model to find the most influencing parameters. Despite its numerous advantages such as lower computational cost, ease of implementation, etc., it may lead to false conclusions due to the scaling issues (Razavi & Gupta 2015) and due to the ranking method based on only μ, μ*, and σ (Sobol & Kucherenko 2009) which may be ineffective for highly non-linear functions.

HMs AND SA APPLICATION

Available conceptual as well as distributed HMs namely, Variable Infiltration Capacity model, SWAT, TOPMODEL, MIKE-SHE, etc., demands the screening of sensitive model parameters. Studies in the past have incorporated a wide application of SA approaches in various HMs. Table 2 summarizes the previous studies of local and global SA, carried out in the various hydrological modeling studies.

Table 2

Review of local and global SA approaches

SA methods Type of SA approach HMs Number of parameters Number of runs Reference 
DELSA Hybrid of local and global Five Framework for Understanding Structural Errors (FUSE) model structure (FUSE-016, FUSE-014, FUSE-160, FUSE-072, and FUSE-170) 14 300 Rakovec et al. (2014)  
Central difference Local TOPKAPI 35 71 Foglia et al. (2009)  
Monte-Carlo sampling based Local VIC 10 59,049 Demaria et al. (2007)  
Combination of Monte-Carlo sampling and likelihood estimates. Generalized Likelihood Uncertainty Estimation (GLUE) Global VIC 6, 7 – Bao et al. (2011), He & Pang (2015)  
MIKE-SHE >400 Blasone et al. (2008)  
LH sampling with regression analysis Global SWAT 35 – Muleta & Nicklow (2005)  
LH sampling with probability distribution of outputs Global MIKE-SHE 25 Christiaens & Feyen (2002)  
LH-OAT Global SWAT 27,41 >100 Holvoet et al. (2005), van Griensven et al. (2006)  
Elementary-effect (EE)/Morris method Global SAC-SMA 14 280 Gan et al. (2014)  
HL-RDHM 78 × 14 20,000 Herman et al. (2013a, 2013c)  
REALM 14 18,000 King & Perera (2013)  
MIKE/NAM >6,000 Liu & Sun (2010)  
Variance-based (VB)/Sobol method Global SAC-SMA 14 1,050 Gan et al. (2014), van Werkhoven et al. (2008a, 2009), Wagener et al. (2009)  
HL-RDHM 78 × 14 >6 million Herman et al. (2013a)  
HBV 12 10,000 Herman et al. (2013b)  
SWAT 26 336,000, 60,000 Nossent et al. (2011), Zhang et al. (2013)  
TOPMODEL 5,632 Reusser et al. (2011)  
 Five Framework for Understanding Structural Errors (FUSE) model structure (FUSE-016, FUSE-014, FUSE-160, FUSE-072, and FUSE-170) 14 2,000,000 Rakovec et al. (2014)  
Combination of EE and VB method Global Xin'anjiang model 10 100,000 Song et al. (2013)  
DTVGM 14 600, 4,000 Zhan et al. (2013)  
SA methods Type of SA approach HMs Number of parameters Number of runs Reference 
DELSA Hybrid of local and global Five Framework for Understanding Structural Errors (FUSE) model structure (FUSE-016, FUSE-014, FUSE-160, FUSE-072, and FUSE-170) 14 300 Rakovec et al. (2014)  
Central difference Local TOPKAPI 35 71 Foglia et al. (2009)  
Monte-Carlo sampling based Local VIC 10 59,049 Demaria et al. (2007)  
Combination of Monte-Carlo sampling and likelihood estimates. Generalized Likelihood Uncertainty Estimation (GLUE) Global VIC 6, 7 – Bao et al. (2011), He & Pang (2015)  
MIKE-SHE >400 Blasone et al. (2008)  
LH sampling with regression analysis Global SWAT 35 – Muleta & Nicklow (2005)  
LH sampling with probability distribution of outputs Global MIKE-SHE 25 Christiaens & Feyen (2002)  
LH-OAT Global SWAT 27,41 >100 Holvoet et al. (2005), van Griensven et al. (2006)  
Elementary-effect (EE)/Morris method Global SAC-SMA 14 280 Gan et al. (2014)  
HL-RDHM 78 × 14 20,000 Herman et al. (2013a, 2013c)  
REALM 14 18,000 King & Perera (2013)  
MIKE/NAM >6,000 Liu & Sun (2010)  
Variance-based (VB)/Sobol method Global SAC-SMA 14 1,050 Gan et al. (2014), van Werkhoven et al. (2008a, 2009), Wagener et al. (2009)  
HL-RDHM 78 × 14 >6 million Herman et al. (2013a)  
HBV 12 10,000 Herman et al. (2013b)  
SWAT 26 336,000, 60,000 Nossent et al. (2011), Zhang et al. (2013)  
TOPMODEL 5,632 Reusser et al. (2011)  
 Five Framework for Understanding Structural Errors (FUSE) model structure (FUSE-016, FUSE-014, FUSE-160, FUSE-072, and FUSE-170) 14 2,000,000 Rakovec et al. (2014)  
Combination of EE and VB method Global Xin'anjiang model 10 100,000 Song et al. (2013)  
DTVGM 14 600, 4,000 Zhan et al. (2013)  

FUSE: Framework for Understanding Structural Errors; TOPKAPI: Topographic Kinematic Approximation and Integration; VIC: Variable Infiltration Capacity Model; MIKE-SHE, MIKE/NAM: rainfall–runoff model; SWAT: Soil and Water Assessment Tool; SAC-SMA: Sacramento Soil Moisture Accounting; HL-RDHM: Hydrology Laboratory – Research Distributed Hydrologic Model; RELAM: Resource Allocation Model; HBV: Hydrologiska Byrans Vattenbalansavdelning Model; TOPMODEL: Topography-based Hydrological Model; DTVGM: Distributed Time Variant Gain Model.

As mentioned earlier, in addition to global approaches, local SA methods play an important role in SA of various HMs, especially while dealing with the major issue of computational requirement. Rakovec et al. (2014) proposed a methodology of SA by combining local and global approaches (EE and VB methods, repectively). A comparison of the DELSA approach with traditional Sobol method on two models, i.e., a simple non-linear reservoir model and a complex conceptual HM framework (FUSE) had demonstrated the potential utility (lower computational cost) of the DELSA approach (Rakovec et al. 2014). Furthermore, SA using a combination of LH sampling and probability distribution in the MIKE-SHE model required only 25 runs (Christiaens & Feyen 2002), while the same exercise when implemented using GLUE approach required more than 400 runs (Blasone et al. 2008). Similarly, when SA of SWAT model is carried out with LH-OAT design, it requires less model simulations as compared to traditional Sobol method (Holvoet et al. 2005; van Griensven et al. 2006; Nossent et al. 2011; Zhang et al. 2013). Gan et al. (2014) skillfully comprehended the evaluation of various SA approaches using the SAC-SMA model and demonstrated that the Morris method requires lesser model evaluations when compared with the Sobol method. It was also shown that the traditional methods like regression and correlation analysis are not efficient in dealing with real-time problems, which are non-linear or non-monotonic in nature. The efficacy of the Morris method over Sobol method is also demonstrated in Herman et al. (2013a). Further, SA of 78 × 14 parameters of HL-RDHM model was carried out by both Morris (20,000 model runs) and Sobol method (>6 million model runs) (Herman et al. 2013a, 2013c).

SPATIO-TEMPORAL DEPENDENCE OF SA

SA of dynamic hydrologic systems is predominantly dependent on both space and time (McCuen 1973; Finnerty et al. 1997; Bastola & Murphy 2013). The physical significance of the input factors can easily be assessed by considering the sensitivity of the input factors to space and time resolutions. The dependence of SA on space and time is attributed to the availability of input data from distinct real data sources, at different spatial and temporal scales. In fully distributed HMs, spatial variability of input parameters are considered to capture catchment heterogeneity in model response. The prevalent use of these complex distributed models necessitates a proper understanding of its physical basis both spatially and temporally (O'Loughlin et al. 2013).

Finnerty et al. (1997) explored the space–time scale sensitivity of the Sacramento model and found that a decrease in spatial resolution brings down the evapotranspiration estimates, which in turn, increases the total channel runoff. Increase in surface runoff is also witnessed with a decrease in time scale (for constant spatial scale). In addition to the Sacramento model, the responses from TOPMODEL also indicate sensitivity towards space and time (Bruneau et al. 1995). The importance of considering various temporal scales in SA, to gain adequate knowledge about the underlying model processes, is highlighted in various past studies (Wagener et al. 2003, 2009; Kavetski & Clark 2010; Reusser & Zehe 2011; Reusser et al. 2011; Herman et al. 2013a, 2013b, 2013c). However, most of the related studies assessing the temporal variability had ignored many of the important parameters in an attempt to consider longer simulation periods. A possible reason could be that many processes depend on definite climatic conditions, which occur at regular time intervals and are confined to small areas, and hence, are identifiable at short time scales. For example, when a conceptual rainfall–runoff model is operated at daily time scales, almost all input parameters are significant; whereas as time scale is increased to yearly, only a few factors relating to soil storage and recession constants show high sensitivities (Massmann & Holzmann 2012). Bruneau et al. (1995) state that the efficiency of model is more sensitive to the choice of time scales when compared to that of the size of the grid, and demonstrated this by simulating surface runoff with larger spatial resolution and medium temporal resolution. Likewise, in hydrological modeling, the intrinsic linkage between the input parameters and the space–time scale has been established by many studies (Obled et al. 1994; Bruneau et al. 1995; Finnerty et al. 1997; Annan 2001).

KEY FACTORS TO BE CONSIDERED IN SA

Objective of SA and associated definition

The first and foremost step before undertaking SA is to obtain adequate information on the objective behind it. As discussed earlier, there are various objectives of SA. Limited knowledge of the purpose of the study and lack of proper definition of sensitivity will produce inaccurate or even conflicting results regarding the sensitivities of the associated parameters. Sensitivity results differ as objective functions vary (van Werkhoven et al. 2008b; Wagener et al. 2009; Liu & Sun 2010; Shin et al. 2013; Song et al. 2013; Zhan et al. 2013). However, this can be addressed through an intuitive selection of single weighted objective functions that incorporate the essence of different objective functions, in the form of weights (Hill & Tiedeman 2006; Foglia et al. 2009).

Computational budget

SA approaches differ in terms of the application or computational cost. Computational budget depends on the number of model runs required to simulate realistic model responses. Since any modeler would demand more realistic results, which in turn require complex models, excessive computational funding is needed for most of the modeling exercises. To circumvent this problem, Razavi et al. (2012a, 2012b) proposed a surrogate modeling approach in which complex models that require more computational resources in terms of time and cost, are replaced by proxy models which are relatively fast, thereby reducing the cost demands to a great extent.

Selection of appropriate method of SA for desired model

Proper selection of SA method is the key step in making any analysis successful. Selection of SA approaches for different modeling exercises depends on a number of factors like physical basis of model, which is employed, time to carry out the analysis, number of input factors, etc. Selection of a best-suited SA approach ensures robust and realistic results.

Range of the parameter and the number of samples

In any sampling method, the number of samples to be considered is one of the crucial factors responsible for the model outcome. Determining the range of a parameter depends on the physical basis of that parameter and the simulated results from the past studies. The same parameter may exhibit a wide range for one study region and may take a constant value for any other region (Shin et al. 2013; Zhan et al. 2013). It is to be noted that, any inaccuracy in the determination of parameter ranges may result in considering an insensitive parameter to be sensitive and vice versa (Shin et al. 2013). The number of samples is also found to significantly affect the model responses (Tong 2006).

Distinguishing SA and uncertainty analysis

As stated before, the uncertainty in the input parameters enhances the factor of uncertainty in the model responses, which in turn, significantly influences the manipulation of model outputs by decision-makers regarding the sensitivity studies. While the objectives of SA and uncertainty analyses (UA) are different, they often complement each other. Saltelli & Annoni (2010) state that, while UA gives the measure of the certainty or uncertainty of the inputs or the outcome, SA complements UA by finding the source of uncertainty. Although these two analyses have different objectives, often they are paired (Yapo et al. 1996; Gupta et al. 1999; Crosetto et al. 2000; Madsen 2000).

Correlation between input parameters

The majority of SA methods neglect the strong correlation between the inputs and consider input parameters to be independent. The inter-relations between various inputs significantly affect the model outcomes and the associated uncertainties (Lemke et al. 2004; Xu & Gertner 2007; Manache & Melching 2008; Pan et al. 2011). Therefore, proper understanding about the dependencies between input factors and their influence on model results is important in any modeling study.

REGIONAL DEPENDENCE OF SA METHODS AND SAMPLING STRATEGIES

Although pool of methods are available to perform SA, since all the approaches widely differ in the distribution pattern of parameters, mathematical representations, pertinence, and the sensitivity indices, one must be very conscientious while selecting a befitting SA strategy. In order to generate significant and convincing results, the appropriate SA strategy should be selected by considering the primary objective of the study and also the computational time needed. Adoption of a particular SA and sampling strategy, to some extent, depends on the modeler's technical familiarity of the region (or basin) being modeled and its hydrological characteristics. Our review brought up a strong affinity between SA/sampling techniques deployed and regions modeled. A systematic literature review with different SA and sampling methods as keywords, i.e., ‘Random sampling’, ‘Stratified sampling’, ‘LH sampling’, ‘Monte-Carlo method’, ‘EE method’, ‘Variance based method’ along with the keyword ‘hydrology’ was carried out to explore the regional preference of SA methods in hydrological modeling. An approximate analysis on the worldwide usage of various approaches thus derived is illustrated in Figure 1. Although different regions employ different sampling and sensitivity techniques, the most popular techniques in the Asian region are variance-based (18%) and Monte Carlo (18%) methods. EE (33%) and LH (31%) methods are widely used in North America. LH (35%) and Monte Carlo (33%) methods are popular in Central and South America. Random sampling and stratified sampling are widely used in Africa, Europe (including the Middle East), and Australia, with the percentage usage being 19%, 13%, and 24% respectively, in each region. Interestingly, LH, Monte Carlo, EE and variance-based techniques and their extensions have very little exposure in Africa and European countries including the Middle East.

Figure 1

Relative percentage of usage of various SA approaches in different regions across the world.

Figure 1

Relative percentage of usage of various SA approaches in different regions across the world.

DISCUSSION AND CONCLUSIONS

SA, the most important component of hydrologic modeling, helps to simplify the complexities and understand the physical processes of complex hydrologic systems in a comprehensive way. The principal aim of SA is to assess the variability of response surface with respect to significant changes in input factors and to prioritize these factors by finding the non-influential factors. This would simplify the complexity of the model either by omitting a few trivial input parameters or by assigning a constant value to them. Various SA methods exist which differ in terms of mathematical approaches, assumptions, availability, cost of application, and applicability. The employment of any SA approach depends on the field of application and the definition. In this paper, different SA techniques are reviewed and categorized in terms of the number of parameters, scale and function dependency, derivative and non-derivative basis. It is widely accepted that the ambiguous definition of SA produces different or even contradictory outcomes (Hamby 1994). Hence, the first and foremost aim in performing SA is defining the problem domain according to the nature of application, to produce worthwhile results.

Most of the existing methods have their own pros and cons and therefore may not be considered as superior or inferior when compared to each other. Many of the studies also questioned the reliability or dependency of any one sensitivity technique, because as the complexity of model increases, the task to find out the correlated parameters and interaction effects becomes more challenging and time-consuming. Also, when modeling is done with an intention to depict the physical processes more realistically or, in other words, when the model becomes more distributed, the number of parameters or the factors contributing to modeling studies also increases. This further increases the uncertainty in the model outcomes primarily due to the uncertain model inputs. Hence, multi-objective SA find its application in many studies (Gupta et al. 1998; Bastidas et al. 2006). A few studies even highlighted the applicability of single best objective function comprising all the features of different objectives (Blasone et al. 2007; Foglia et al. 2009).

For low-dimensional systems, various approaches like OAT, stratified sampling and LOR estimates may simulate realistic results. However, with increase in the number of input factors, the analysis suffers from the curse of dimensionality and becomes cumbersome. EE method, preferred in the case of complex models, also gives erroneous results and mistakes non-influential parameters as influential ones in the modeling. EE method is highly dependent on the step size or scale. In any complex system, where interaction between parameters plays an important role, variance-based method is adopted, which does not depend on the formulation of objective function. Variance-based methods facilitate spatial-temporal dependency of the input factors. The only drawback of the variance-based technique is the high cost of analysis, which can be reduced through proper screening of samples and prior implementation. Variance-based method together with sampling methods like LH sampling and EE method may eliminate many drawbacks such as computational cost, interaction effects between variables and spatial-temporal dependency of the outcome on the input factors. It is evident that almost all the SA techniques presented are susceptible to one or other limitation. The majority of the methods assumes independency among the inputs, which may not replicate real-world problems. Therefore, considering all these factors, modelers should adopt subjective judgments while carrying out SA in hydrologic modeling studies, which in turn demand thorough knowledge about the inherent characteristics of the system and the interactions between the various variables involved. These facts, further, reinforce the need to conduct and develop a comprehensive SA method, which would aid in automating the steps in hydrologic modeling, thus helping decision-makers in adequate management of system components.

ACKNOWLEDGEMENTS

The authors would like to thank the Department of Science and Technology (India) for partly supporting this study through the grant no: SB/FTP/ETA-0110/2013 to Dr Dhanya C.T. The authors express sincere thanks to the Indian Institute of Technology, Delhi for supporting this work.

REFERENCES

REFERENCES
Annan
,
J. D.
2001
Modelling under uncertainly: Monte Carlo methods for temporally varying parameters
.
Ecol. Modell.
136
,
297
302
.
doi:10.1016/S0304-3800(00)00413-0
.
Arnaud
,
P.
,
Lavabre
,
J.
,
Fouchier
,
C.
,
Diss
,
S.
&
Javelle
,
P.
2011
Sensitivity of hydrological models to uncertainty in rainfall input
.
Hydrol. Sci. J.
56
,
397
410
.
doi:10.1080/02626667.2011.563742
.
Bao
,
Z.
,
Liu
,
J.
,
Zhang
,
J.
,
Fu
,
G.
,
Wang
,
G.
,
Yan
,
X.
,
Zhang
,
A.
,
Xu
,
Q.
&
Shang
,
M.
2011
Estimation of baseflow parameters of variable infiltration capacity model with soil and topography properties for predictions in ungauged basins
.
Hydrol. Earth Syst. Sci. Discuss
8
,
7017
7053
.
doi:10.5194/hessd-8-7017-2011
.
Bastidas
,
L. A.
,
Hogue
,
T. S.
,
Sorooshian
,
S.
,
Gupta
,
H. V.
&
Shuttleworth
,
W. J.
2006
Parameter sensitivity analysis for different complexity land surface models using multicriteria methods
.
J. Geophys. Res.
111
,
1
19
.
doi:10.1029/2005JD006377
.
Bauer
,
K. W.
,
Parnell
,
G. S.
&
Meyers
,
D. A.
1999
Response surface methodology as a sensitivity analysis tool in decision analysis
.
J. Multi-Criteria Decis. Anal.
8
,
162
180
.
doi:10.1002/(SICI)1099-1360(199905)8:3 %3C 162::AID-MCDA241 > 3.0.CO;2-X
.
Beven
,
K.
1997
TOPMODEL: a critique
.
Hydrol. Process.
11
,
1069
1085
.
Beven
,
K.
&
O'Connell
,
P. E.
1982
On the Role of Physically-Based Distributed Modelling in Hydrology
.
Institute of Hydrology
,
Wallingford
,
UK
.
Beven
,
K. J.
&
Kirkby
,
M. J.
1979
Un modèle à base physique de zone d'appel variable de l'hydrologie du bassin versant
.
[A physically based, variable contributing area model of basin hydrology]
Hydrol. Sci. Bull.
24
,
43
69
.
doi:10.1080/02626667909491834
.
Binley
,
A.
,
Beven
,
K.
&
Elgy
,
J.
1989a
A physically based model of heterogeneous hillslopes: 2. Effective hydraulic conductivities
.
Water Resour. Res.
25
,
1227
.
doi:10.1029/WR025i006p01227
.
Binley
,
A.
,
Elgy
,
J.
&
Beven
,
K.
1989b
A physically based model of heterogeneous hillslopes: 1. Runoff production
.
Water Resour. Res.
25
,
1219
1226
.
doi:10.1029/WR025i006p01219
.
Blasone
,
R. S.
,
Madsen
,
H.
&
Rosbjerg
,
D.
2007
Parameter estimation in distributed hydrological modelling: comparison of global and local optimisation techniques
.
Nord. Hydrol.
38
,
451
.
doi:10.2166/nh.2007.024
.
Blasone
,
R.-S.
,
Madsen
,
H.
&
Rosbjerg
,
D.
2008
Uncertainty assessment of integrated distributed hydrological models using GLUE with Markov chain Monte Carlo sampling
.
J. Hydrol.
353
,
18
32
.
doi:10.1016/j.jhydrol.2007.12.026
.
Bloschl
,
G.
&
Sivapalan
,
M.
1995
Scale issues in hydrological modelling: a review
.
Hydrol. Process.
9
,
251
290
.
Boyle
,
P.
,
Broadie
,
M.
&
Glasserman
,
P.
1997
Monte Carlo methods for security pricing
.
J. Econ. Dyn. Control
21
,
1267
1321
.
doi:10.1016/S0165-1889(97)00028-6
.
Boyle
,
D. P.
,
Gupta
,
H. V.
&
Sorooshian
,
S.
2000
Toward improved calibration of hydrologic models: combining the strengths of manual and automatic methods
.
Water Resour. Res.
36
,
3663
3674
.
doi:10.1029/2000WR900207
.
Brazil
,
L. E.
&
Krajewski
,
W. F.
1987
Optimization of complex hydrologic models using random search methods
.
Eng. Hydrol. Proceedings
ASCE,
Williamsburg, VA,
August
3
7
, 1987.
Bruneau
,
P.
,
Gascuel-Odoux
,
C.
,
Robin
,
P.
,
Merot
,
P.
&
Beven
,
K.
1995
Sensitivity to space and time resolution of a hydrological model using digital elevation data
.
Hydrol. Process.
9
,
69
81
.
doi:10.1002/hyp.3360090107
.
Campolongo
,
F.
,
Cariboni
,
J.
&
Saltelli
,
A.
2007
An effective screening design for sensitivity analysis of large models
.
Environ. Model. Softw.
22
,
1509
1518
.
doi:10.1016/j.envsoft.2006.10.004
.
Castaings
,
W.
,
Dartus
,
D.
,
Dimet
,
L.
&
Sen
,
G. S.
2007
Sensitivity analysis and parameter estimation for the distributed modeling of infiltration excess overland flow
.
Hydrol. Earth Syst. Sci. Discuss.
4
,
363
405
.
Chow
,
V. T.
,
Maidment
,
D. R.
&
Mays
,
L. W.
1988
Applied Hydrology
.
McGraw-Hill Series in Water Resources and Environmental Engineering, McGraw-Hill
,
New York
,
USA
.
Christensen
,
R.
2006
Log-Linear Models and Logistic Regression
.
Springer Texts in Statistics. Springer Science & Business Media, New York
.
Coyle
,
D.
,
Buxton
,
M. J.
&
O'Brien
,
B. J.
2003
Measures of importance for economic analysis based on decision modeling
.
J. Clin. Epidemiol.
56
,
989
997
.
doi:10.1016/S0895-4356(03)00176-8
.
Crosetto
,
M.
,
Tarantola
,
S.
&
Saltelli
,
A.
2000
Sensitivity and uncertainty analysis in spatial modelling based on GIS
.
Agric. Ecosyst. Environ.
81
,
71
79
.
doi:10.1016/S0167-8809(00)00169-9
.
D'Agnese
,
F. A.
,
Faunt
,
C. C.
,
Hill
,
M. C.
&
Keith Turner
,
A.
1999
Death Valley regional ground-water flow model calibration using optimal parameter estimation methods and geoscientific information systems
.
Adv. Water Resour.
22
,
777
790
.
doi:10.1016/S0309-1708(98)00053-0
.
Dawdy
,
D. R.
&
O'Donnell
,
T.
1965
Mathematical models of catchment behavior
.
J. Hydraul. Div.
91
,
123
137
.
Demaria
,
E. M.
,
Nijssen
,
B.
&
Wagener
,
T.
2007
Monte Carlo sensitivity analysis of land surface parameters using the variable infiltration capacity model
.
J. Geophys. Res.
112
,
1
15
.
doi:10.1029/2006JD007534
.
Dimov
,
I.
&
Georgieva
,
R.
2010
Monte Carlo algorithms for evaluating Sobol’ sensitivity indices
.
Math. Comput. Simul.
81
,
506
514
.
doi:10.1016/j.matcom.2009.09.005
.
Finnerty
,
B. D.
,
Smith
,
M. B.
,
Seo
,
D.-J.
,
Koren
,
V.
&
Moglen
,
G. E.
1997
Space-time scale sensitivity of the Sacramento model to radar-gage precipitation inputs
.
J. Hydrol.
203
,
21
38
.
doi:10.1016/S0022-1694(97)00083-8
.
Fisher
,
R. A.
1942
The Design of Experiments
,
3rd edn.
Oliver and Boyd
,
London
,
UK
.
doi:10.2307/2300364
.
Foglia
,
L.
,
Hill
,
M. C.
,
Mehl
,
S. W.
&
Burlando
,
P.
2009
Sensitivity analysis, calibration, and testing of a distributed hydrological model using error-based weighting and one objective function
.
Water Resour. Res.
45
,
1
18
.
doi:10.1029/2008WR007255
.
Frey
,
C. H.
&
Patil
,
S. R.
2002
Indentification and review of sensitivity analysis methods
.
Risk Anal.
22
,
553
578
.
doi:10.1111/0272-4332.00039
.
Gan
,
Y.
,
Duan
,
Q.
,
Gong
,
W.
,
Tong
,
C.
,
Sun
,
Y.
,
Chu
,
W.
,
Ye
,
A.
,
Miao
,
C.
&
Di
,
Z.
2014
A comprehensive evaluation of various sensitivity analysis methods: a case study with a hydrological model
.
Environ. Model. Softw.
51
,
269
285
.
doi:10.1016/j.envsoft.2013.09.031
.
Gupta
,
H. V.
,
Sorooshian
,
S.
&
Yapo
,
P. O.
1998
Toward improved calibration of hydrologic models: multiple and noncommensurable measures of information
.
Water Resour. Res.
34
,
751
.
doi:10.1029/97WR03495
.
Gupta
,
H. V.
,
Bastidas
,
L. A.
,
Sorooshian
,
S.
,
Shuttleworth
,
W. J.
&
Yang
,
Z. L.
1999
Parameter estimation of a land surface scheme using multicriteria methods
.
J. Geophys. Res.
104
,
19491
19503
.
doi:10.1029/1999JD900154
.
Gupta
,
H. V.
,
Clark
,
M. P.
,
Vrugt
,
J. A.
,
Abramowitz
,
G.
&
Ye
,
M.
2012
Towards a comprehensive assessment of model structural adequacy
.
Water Resour. Res.
48
,
1
16
.
doi:10.1029/2011WR011044
.
He
,
R.
&
Pang
,
B.
2015
Sensitivity and uncertainty analysis of the variable infiltration capacity model in the upstream of Heihe River basin
.
Proc. Int. Assoc. Hydrolog. Sci.
368
,
312
316
.
doi:10.5194/piahs-368-312-2015
.
Henderson-Sellers
,
A.
1993
A factorial assessment of the sensitivity of the BATS land-surface parameterization scheme
.
Am. Meteorol. Soc.
6
,
227
247
.
Herman
,
J. D.
,
Kollat
,
J. B.
,
Reed
,
P. M.
&
Wagener
,
T.
2013a
From maps to movies: high-resolution time-varying sensitivity analysis for spatially distributed watershed models
.
Hydrol. Earth Syst. Sci.
17
,
5109
5125
.
doi:10.5194/hess-17-5109-2013
.
Herman
,
J. D.
,
Kollat
,
J. B.
,
Reed
,
P. M.
&
Wagener
,
T.
2013b
Method of Morris effectively reduces the computational demands of global sensitivity analysis for distributed watershed models
.
Hydrol. Earth Syst. Sci.
17
,
2893
2903
.
doi:10.5194/hess-17-2893-2013
.
Herman
,
J. D.
,
Reed
,
P. M.
&
Wagener
,
T.
2013c
Time-varying sensitivity analysis clarifies the effects of watershed model formulation on model behavior
.
Water Resour. Res.
49
,
1400
1414
.
doi:10.1002/wrcr.20124
.
Hill
,
M.
&
Tiedeman
,
C.
2006
Effective Groundwater Model Calibration: With Analysis of Data, Sensitivities, Predictions, and Uncertainty
.
John Wiley & Sons
,
Hoboken, NJ
,
USA
.
Holvoet
,
K.
,
van Griensven
,
A.
,
Seuntjens
,
P.
&
Vanrolleghem
,
P. A.
2005
Sensitivity analysis for hydrology and pesticide supply towards the river in SWAT
.
Phys. Chem. Earth
30
,
518
526
.
doi:10.1016/j.pce.2005.07.006
.
Hughes
,
D. A.
&
Beater
,
A. B.
1989
The applicability of two single event models to catchments with different physical characteristics
.
Hydrol. Sci. J.
34
,
63
78
.
doi:10.1080/02626668909491309
.
Hughes
,
J. P.
,
Guttorp
,
P.
&
Charles
,
S. P.
1999
A non-homogeneous hidden Markov model for precipitation occurrence
.
J. R. Stat. Soc. Ser. C Appl. Statist.
48
,
15
30
.
Iman
,
R. L.
&
Conover
,
W. J.
2007
Small sample sensitivity analysis techniques for computer models.with an application to risk assessment
.
Commun. Stat. - Theory Methods
9
,
1749
1842
.
doi:10.1080/03610928008827996
.
Iman
,
R. L.
&
Helton
,
J. C.
1985
Comparison of Uncertainty and Sensitivity Analysis Techniques for Computer Models
.
Report No. NUREG/CR-3904; SAND-84-1461
.
Sandia National Labs.
,
Albuquerque, NM
,
USA
.
Iooss
,
B.
,
Lematre
,
P.
2015
A review on global sensitivity analysis methods
. In:
Uncertainty Management in Simulation-Optimization in Complex Systems
(
Dellino
,
G.
&
Meloni
,
C.
, eds).
Springer Science + Business Media
,
New York
,
USA
.
Jiang
,
S.
,
Ren
,
L.
,
Yong
,
B.
,
Fu
,
C.
&
Yang
,
X.
2012
Analyzing the effects of climate variability and human activities on runoff from the Laohahe basin in northern China
.
Hydrol. Res.
43
,
3
.
doi:10.2166/nh.2011.133
.
Johnston
,
P. R.
&
Pilgrim
,
D. H.
1976
Parameter optimization for watershed models
.
Water Resour. Res.
12
,
477
486
.
doi:10.1029/WR012i003p00477
.
King
,
D. M.
2009
On the Importance of Input Variables and Climate Variability to the Yield of Urban Water Supply Systems
.
PhD Thesis
,
Victoria University
,
Australia
.
Kleijnen
,
J. P. C.
2005
An overview of the design and analysis of simulation experiments for sensitivity analysis
.
Eur. J. Oper. Res.
164
,
287
300
.
doi:10.1016/j.ejor.2004.02.005
.
Krajewski
,
W. F.
,
Lakshmi
,
V.
,
Georgakakos
,
K. P.
&
Jain
,
S. C.
1991
A Monte Carlo study of rainfall sampling effect on a distributed catchment model
.
Water Resour. Res.
27
,
119
128
.
doi:10.1029/90WR01977
.
Kuczera
,
G.
&
Parent
,
E.
1998
Monte Carlo assessment of parameter uncertainty in conceptual catchment models: the Metropolis algorithm
.
J. Hydrol.
211
,
69
85
.
doi:10.1016/S0022-1694(98)00198-X
.
Lemke
,
L. D.
,
Abriola
,
L. M.
&
Lang
,
J. R.
2004
Influence of hydraulic property correlation on predicted dense nonaqueous phase liquid source zone architecture, mass recovery and contaminant flux
.
Water Resour. Res.
40
,
1
18
.
doi:10.1029/2004WR003061
.
Liang
,
X.
1994
A Two-Layer Variable Infiltration Capacity Land Surface Representation for General Circulation Models
.
PhD Dissertation
,
University of Washington
,
USA
.
Lilburne
,
L.
,
Gatelli
,
D.
&
Tarantola
,
S.
2006
Sensitivity analysis on spatial models : a new approach
.
7th International Symposium on Spatial Accuracy Assessment in Natural Resourources and Environmental Sciences (ACCURACY 2006)
,
Lisbon
,
Portugal
,
5–7 July, 2006
, pp.
329
338
.
Liu
,
Y.
&
Sun
,
F.
2010
Sensitivity analysis and automatic calibration of a rainfall–runoff model using multi-objectives
.
Ecol. Inform.
5
,
304
310
.
doi:10.1016/j.ecoinf.2010.04.006
.
Mackay
,
N. G.
,
Chandler
,
R. E.
,
Onof
,
C.
&
Wheater
,
H. S.
2001
Disaggregation of spatial rainfall fields for hydrological modelling
.
Hydrol. Earth Syst. Sci. Discuss.
5
,
165
173
.
McCuen
,
R. H.
1973
The role of sensitivity analysis in hydrologic modeling
.
J. Hydrol.
18
,
37
53
.
doi:10.1016/0022-1694(73)90024-3
.
McKay
,
M. D.
,
Beckman
,
R. J.
&
Conover
,
W. J.
1979
Comparison of three methods for selecting values of input variables in the analysis of output from a computer code
.
Technometrics
21
,
239
245
.
doi:10.1080/00401706.1979.10489755
.
Melching
,
C. S.
1995
Reliability estimation
. In:
Computer Models of Watershed Hydrology
(
Singh
,
V. P.
ed.).
Water Resources Publications
,
Highlands Ranch, CO
,
USA
.
Melching
,
C. S.
&
Anmangandla
,
S.
1993
Improved first-order uncertainty method for water-quality modeling
.
J. Environ. Eng.
118
,
791
805
.
Meng
,
L.
&
Quiring
,
S. M.
2008
A comparison of soil moisture models using soil climate analysis network observations
.
Am. Meteorol. Soc.
9
,
641
659
.
doi:10.1175/2008JHM916.1
.
Merz
,
R.
,
Parajka
,
J.
&
Blöschl
,
G.
2009
Scale effects in conceptual hydrological modeling
.
Water Resour. Res.
45
.
doi:10.1029/2009WR007872
.
Mishra
,
S.
,
Deeds
,
N.
&
Ruskauff
,
G.
2009
Global sensitivity analysis techniques for probabilistic ground water modeling
.
Ground Water
47
,
730
747
.
doi:10.1111/j.1745-6584.2009.00604.x
.
Mokhtari
,
A.
&
Frey
,
H. C.
2005
Recommended practice regarding selection of sensitivity analysis methods applied to microbial food safety process risk models
.
Hum. Ecol. Risk Assess. An Int. J.
11
,
591
605
.
doi:10.1080/10807030590949672
.
Muleta
,
M. K.
&
Nicklow
,
J. W.
2005
Sensitivity and uncertainty analysis coupled with automatic calibration for a distributed watershed model
.
J. Hydrol.
306
,
127
145
.
doi:10.1016/j.jhydrol.2004.09.005
.
Mulungu
,
D. M. M.
&
Munishi
,
S. E.
2007
Simiyu River catchment parameterization using SWAT model
.
Phys. Chem. Earth
32
,
1032
1039
.
doi:10.1016/j.pce.2007.07.053
.
Nash
,
J. E.
&
Sutcliffe
,
J. V.
1970
River flow forecasting through conceptual models part I – a discussion of principles
.
J. Hydrol.
10
,
282
290
.
doi:10.1016/0022-1694(70)90255-6
.
Ndomba
,
P.
,
Mtalo
,
F.
&
Killingtveit
,
A.
2008
SWAT model application in a data scarce tropical complex catchment in Tanzania
.
Phys. Chem. Earth
33
,
626
632
.
doi:10.1016/j.pce.2008.06.013
.
Norton
,
E. C.
,
Wang
,
H.
&
Ai
,
C.
2004
Computing interaction effects and standards errors in logit and probit models
.
Stata J.
4
,
154
167
.
Nossent
,
J.
,
Elsen
,
P.
&
Bauwens
,
W.
2011
Sobol’ sensitivity analysis of a complex environmental model
.
Environ. Model. Softw.
26
,
1515
1525
.
doi:10.1016/j.envsoft.2011.08.010
.
Obled
,
C.
,
Wendling
,
J.
&
Beven
,
K.
1994
The sensitivity of hydrological models to spatial rainfall patterns: an evaluation using observed data
.
J. Hydrol.
159
,
305
333
.
doi:10.1016/0022-1694(94)90263-1
.
O'Loughlin
,
F.
,
Bruen
,
M.
&
Wagener
,
T.
2013
Parameter sensitivity of a watershed-scale flood forecasting model as a function of modelling time-step
.
Hydrol. Res.
44
,
334
.
doi:10.2166/nh.2012.157
.
Pan
,
F.
,
Zhu
,
J.
,
Ye
,
M.
,
Pachepsky
,
Y. A.
&
Wu
,
Y. S.
2011
Sensitivity analysis of unsaturated flow and contaminant transport with correlated parameters
.
J. Hydrol.
397
,
238
249
.
doi:10.1016/j.jhydrol.2010.11.045
.
Pappenberger
,
F.
,
Beven
,
K. J.
,
Ratto
,
M.
&
Matgen
,
P.
2008
Multi-method global sensitivity analysis of flood inundation models
.
Adv. Water Resour.
31
,
1
14
.
doi:10.1016/j.advwatres.2007.04.009
.
Pielke
,
R. A.
,
Walko
,
R. L.
,
Steyaert
,
L. T.
,
Vidale
,
P. L.
,
Liston
,
G. E.
,
Lyons
,
W. A.
&
Chase
,
T. N.
1999
The influence of anthropogenic landscape changes on weather in South Florida
.
Mon. Weather Rev.
127
,
1663
1673
.
doi:10.1175/1520-0493(1999)127 < 1663:TIOALC > 2.0.CO;2
.
Price
,
W. L.
&
Dixon
,
C. L. C. W.
1987
Global optimization algorithms for a CAD workstation
.
J. Optim. Theory Appl.
55
,
133
146
.
Rakovec
,
O.
,
Hill
,
M. C.
,
Clark
,
M. P.
,
Weerts
,
A. H.
,
Teuling
,
A. J.
&
Uijlenhoet
,
R.
2014
Distributed evaluation of local sensitivity analysis (DELSA), with application to hydrologic models
.
Water Resour. Res.
50
,
409
426
.
doi:10.1002/2013WR014063
.
Razavi
,
S.
,
Tolson
,
B. A.
&
Burn
,
D. H.
2012a
Review of surrogate modeling in water resources
.
Water Resour. Res.
48
.
doi:10.1029/2011WR011527
.
Razavi
,
S.
,
Tolson
,
B. A.
&
Burn
,
D. H.
2012b
Numerical assessment of metamodelling strategies in computationally intensive optimization
.
Environ. Model. Softw.
34
,
67
86
.
doi:10.1016/j.envsoft.2011.09.010
.
Reiter
,
R.
1987
A theory of diagnosis from first principles
.
Artif. Intell.
32
,
57
95
.
doi:10.1016/0004-3702(87)90062-2
.
Reusser
,
D. E.
,
Buytaert
,
W.
&
Zehe
,
E.
2011
Temporal dynamics of model parameter sensitivity for computationally expensive models with the Fourier amplitude sensitivity test
.
Water Resour. Res.
47
.
doi:10.1029/2010WR009947
.
Ruano
,
M. V.
,
Ribes
,
J.
,
Ferrer
,
J.
&
Sin
,
G.
2011
Application of the Morris method for screening the influential parameters of fuzzy controllers applied to wastewater treatment plants
.
Water Sci. Technol.
63
(
10
),
2199
2206
.
doi:10.2166/wst.2011.442
.
Saltelli
,
A.
1999
Sensitivity analysis: could better methods be used?
J. Geophys. Res.
104
,
3789
3793
.
doi:10.1029/1998JD100042
.
Saltelli
,
A.
&
Annoni
,
P.
2010
How to avoid a perfunctory sensitivity analysis
.
Environ. Model. Softw.
25
,
1508
1517
.
doi:10.1016/j.envsoft.2010.04.012
.
Saltelli
,
A.
,
Andres
,
T. H.
&
Homma
,
T.
1995
Sensitivity analysis of model output. Performance of the iterated fractional factorial design method
.
Comput. Stat. Data Anal.
20
,
387
407
.
doi:10.1016/0167-9473(95)92843-M
.
Saltelli
,
A.
,
Chan
,
K.
&
Scott
,
E. M.
2000
Sensitivity Analysis
.
John Wiley & Sons Ltd.
,
New York
,
USA
.
Saltelli
,
A.
,
Tarantola
,
S.
,
Campolongo
,
F.
&
Ratto
,
M.
2004
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
.
John Wiley & Sons
,
New York
,
USA
.
Saltelli
,
A.
,
Ratto
,
M.
,
Tarantola
,
S.
&
Campolongo
,
F.
2005
Sensitivity analysis for chemical models
.
Chem. Rev.
105
,
2811
2828
.
doi:10.1021/cr040659d
.
Saltelli
,
A.
,
Ratto
,
M.
,
Andres
,
T.
,
Campolongo
,
F.
,
Cariboni
,
J.
,
Gatelli
,
D.
,
Saisana
,
M.
&
Tarantola
,
S.
2008
Global Sensitivity Analysis: The Primer
.
John Wiley & Sons
,
New York
,
USA
.
Saltelli
,
A.
,
Annoni
,
P.
,
Azzini
,
I.
,
Campolongo
,
F.
,
Ratto
,
M.
&
Tarantola
,
S.
2010
Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index
.
Comput. Phys. Commun.
181
,
259
270
.
doi:10.1016/j.cpc.2009.09.018
.
Saltelli
,
A.
,
Ratto
,
M.
,
Tarantola
,
S.
&
Campolongo
,
F.
2012
Update 1 of: sensitivity analysis for chemical models
.
Chem. Rev.
112
,
1
21
.
Sharma
,
M. L.
,
Luxmoore
,
R. J.
,
DeAngelis
,
R.
,
Ward
,
R. C.
&
Yeh
,
G. T.
1987
Subsurface water flow simulated for hillslopes with spatially dependent soil hydraulic characteristics
.
Water Resour. Res.
23
,
1523
1530
.
doi:10.1029/WR023i008p01523
.
Shin
,
M. J.
,
Guillaume
,
J. H.
,
Croke
,
B. F. W.
&
Jakeman
,
A. J.
2013
Addressing ten questions about conceptual rainfall-runoff models with global sensitivity analyses in R
.
J. Hydrol.
503
,
135
152
.
doi:10.1016/j.jhydrol.2013.08.047
.
Sivapalan
,
M.
,
Takeuchi
,
K.
,
Franks
,
S. W.
,
Gupta
,
V. K.
,
Karambiri
,
H.
,
Lakshmi
,
V.
,
Liang
,
X.
,
McDonnell
,
J. J.
,
Mendiondo
,
E. M.
,
O'Connell
,
P. E.
,
Oki
,
T.
,
Pomeroy
,
J. W.
,
Schertzer
,
D.
,
Uhlenbrook
,
S.
&
Zehe
,
E.
2003
IAHS decade on predictions in ungauged basins (PUB), 2003–2012: shaping an exciting future for the hydrological sciences
.
Hydrol. Sci. J.
48
,
857
880
.
doi:10.1623/hysj.48.6.857.51421
.
Sobol
,
I. M.
&
Kucherenko
,
S.
2009
Derivative based global sensitivity measures and their link with global sensitivity indices
.
Math. Comput. Simul.
79
,
3009
3017
.
doi:10.1016/j.matcom.2009.01.023
.
Song
,
X.
,
Kong
,
F.
,
Zhan
,
C.
&
Han
,
J.
2012
Sensitivity analysis of hydrological model parameters using a statistical theory approach
.
Adv. Water Sci.
23
,
642
649
.
Song
,
X.
,
Kong
,
F.
,
Zhan
,
C.
&
Han
,
J.
2013
Parameter identification and global sensitivity analysis of Xin'anjiang model using meta-modeling approach
.
Water Sci. Eng.
6
,
1
17
.
doi:10.3882/j.issn.1674-2370.2013.01.001
.
Song
,
X.
,
Zhang
,
J.
,
Zhan
,
C.
,
Xuan
,
Y.
,
Ye
,
M.
&
Xu
,
C.
2015
Global sensitivity analysis in hydrological modeling: review of concepts, methods, theoretical framework, and applications
.
J. Hydrol.
523
,
739
757
.
doi:10.1016/j.jhydrol.2015.02.013
.
Sorooshian
,
S.
&
Arfi
,
F.
1982
Response surface parameter sensitivity analysis methods for postcalibration studies
.
Water Resour. Res.
18
,
1531
1538
.
doi:10.1029/WR018i005p01531
.
Sorooshian
,
S.
&
Gupta
,
V.
1995
Model calibration
. In:
Computer Models of Watershed Hydrology
(
Singh
,
V. P.
ed.).
Water Resources Publications
,
Fort Collins, CO
,
USA
, pp.
23
68
.
Spear
,
R. C.
&
Hornberger
,
G. M.
1980
Eutrophication in peel inlet. 2. Identification of critical uncertainties via generalized analysis
.
Water Res.
14
,
43
49
.
doi:10.1016/0043-1354(80)90040-8
.
Spruill
,
C. A.
,
Workman
,
S.
&
Taraba
,
J. L.
2000
Simulation of daily and monthly stream discharge from small watersheds using the swat model
.
Trans. ASAE
43
,
1431
1439
.
doi:10.13031/2013.3041
.
Stephenson
,
D.
2000
Use of the ‘odds ratio’ for diagnosing forecast skill
.
Weather Forecast.
15
,
221
232
.
doi:10.1175/1520-0434(2000)015 < 0221:UOTORF > 2.0.CO;2
.
Stiber
,
N. A.
,
Pantazidou
,
M.
&
Small
,
M. J.
1999
Expert system methodology for evaluating reductive dechlorination at TCE sites
.
Environ. Sci. Technol.
33
,
3012
3020
.
doi:10.1021/es981216s
.
Thomas
,
R. B.
&
Lewis
,
J.
1995
An evaluation of flow-stratified sampling for estimating suspended sediment loads
.
J. Hydrol.
170
,
27
45
.
doi:10.1016/0022-1694(95)02699-P
.
Tong
,
C.
2006
Refinement strategies for stratified sampling methods
.
Reliab. Eng. Syst. Saf.
91
,
1257
1265
.
doi:10.1016/j.ress.2005.11.027
.
Troch
,
P.
,
Paniconi
,
C.
&
McLaughlin
,
D.
2003
Catchment-scale hydrological modeling and data assimilation
.
Adv. Water Resour.
26
,
131
135
.
Tucker
,
G. E.
&
Whipple
,
K. X.
2002
Topographic outcomes predicted by stream erosion models: sensitivity analysis and intermodel comparison
.
J. Geophys. Res. Solid Earth
107
,
ETG 1–1
ETG 1–16
.
doi:10.1029/2001JB000162
.
van Griensven
,
A.
,
Meixner
,
T.
,
Grunwald
,
S.
,
Bishop
,
T.
,
Diluzio
,
M.
&
Srinivasan
,
R.
2006
A global sensitivity analysis tool for the parameters of multi-variable catchment models
.
J. Hydrol.
324
,
10
23
.
doi:10.1016/j.jhydrol.2005.09.008
.
van Werkhoven
,
K.
,
Wagener
,
T.
,
Reed
,
P.
&
Tang
,
Y.
2008a
Characterization of watershed model behavior across a hydroclimatic gradient
.
Water Resour. Res.
44
,
1
16
.
doi:10.1029/2007WR006271
.
van Werkhoven
,
K.
,
Wagener
,
T.
,
Reed
,
P.
&
Tang
,
Y.
2008b
Rainfall characteristics define the value of streamflow observations for distributed watershed model identification
.
Geophys. Res. Lett.
35
,
1
6
.
doi:10.1029/2008GL034162
.
van Werkhoven
,
K.
,
Wagener
,
T.
,
Reed
,
P.
&
Tang
,
Y.
2009
Sensitivity-guided reduction of parametric dimensionality for multi-objective calibration of watershed models
.
Adv. Water Resour.
32
,
1154
1169
.
doi:10.1016/j.advwatres.2009.03.002
.
Vemuri
,
V.
,
Dracup
,
J. A.
,
Erdmann
,
R. C.
&
Vemuri
,
N.
1969
Sensitivity analysis method of system identification and its potential in hydrologic research
.
Water Resour. Res.
5
,
341
349
.
doi:10.1029/WR005i002p00341
.
Wagener
,
T.
,
McIntyre
,
N.
,
Lees
,
M. J.
,
Wheater
,
H. S.
&
Gupta
,
H. V.
2003
Towards reduced uncertainty in conceptual rainfall-runoff modelling: dynamic identifiability analysis
.
Hydrol. Process.
17
,
455
476
.
doi:10.1002/hyp.1135
.
Wagener
,
T.
,
Van Werkhoven
,
K.
,
Reed
,
P.
&
Tang
,
Y.
2009
Multiobjective sensitivity analysis to understand the information content in streamflow observations for distributed watershed modeling
.
Water Resour. Res.
45
,
1
5
.
doi:10.1029/2008WR007347
.
Xu
,
C.
&
Gertner
,
G.
2007
Extending a global sensitivity analysis technique to models with correlated parameters
.
Comput. Stat. Data Anal.
51
,
5579
5590
.
doi:10.1016/j.csda.2007.04.003
.
Yapo
,
P. O.
,
Gupta
,
H. V.
&
Sorooshian
,
S.
1996
Automatic calibration of conceptual rainfall-runoff models: sensitivity to calibration data
.
J. Hydrol.
181
,
23
48
.
doi:10.1016/0022-1694(95)02918-4
.
Ye
,
M.
,
Meyer
,
P. D.
&
Neuman
,
S. P.
2008
On model selection criteria in multimodel analysis
.
Water Resour. Res.
44
.
doi:10.1029/2008WR006803
.
Zhan
,
C.
,
Song
,
X.
,
Xia
,
J.
&
Tong
,
C.
2013
An efficient integrated approach for global sensitivity analysis of hydrological model parameters
.
Environ. Model. Softw.
41
,
39
52
.
doi:10.1016/j.envsoft.2012.10.009
.
Zhang
,
C.
,
Chu
,
J.
&
Fu
,
G.
2013
Sobol's sensitivity analysis for a distributed hydrological model of Yichun River Basin, China
.
J. Hydrol.
480
,
58
68
.
doi:10.1016/j.jhydrol.2012.12.005
.