## Abstract

Distributed watershed models should pass through a careful sensitivity analysis and calibration procedure before they are utilized as a decision making aid in the planning and management of water resources. Although manual approaches are still frequently used for sensitivity and calibration, they are tedious, time consuming, and require experienced personnel. This paper describes two typical and effective automatic approaches for sensitivity analysis and calibration for the Soil and Water Assessment Tool (SWAT). These included the Sequential Uncertainty Fitting (SUFI-2) algorithm and Shuffled Complex Evolution (SCE-UA) algorithm. The results show the following. (1) The main factor that influences the simulated accuracy of the Heihe River basin runoff is the Soil Conservation Service (SCS) runoff curve parameters. (2) SWAT performed very well in the Heihe River basin. According to the observed runoff data from 2005 to 2013, the determination coefficient *R*^{2} of the simulation and the efficiency coefficient (*Ens*) of the model was higher than 0.8. (3) Compared with the Shuffled Complex Evolution, the SUFI-2 algorithm provides almost the same overall ranking of the sensitive parameters, but it is found to require less time with higher accuracy. The SUFI-2 provides a practical and flexible tool to attain reliable deterministic simulation and uncertainty analysis of SWAT, it can lead to a better understanding and to better estimated values and thus reduced uncertainty.

## INTRODUCTION

Hydrologic models are useful tools in that they enable us to investigate many practical issues that arise during planning, design, and operation of water resources systems (Brush *et al.* 2009; Kuhta *et al.* 2012; Thirel *et al.* 2015). They undergo some aspect of conceptualization, and their results are only as realistic as the model assumptions, quality of inputs, and parameter values that are considered for a particular system. The purpose of using a hydrologic model is to establish baseline characteristics whenever data are not available. Furthermore, there are long-term impacts that are difficult to calculate (Huo *et al.* 2011; Huo & Li 2013). For most hydrologic models, it is imperative that a mechanism that improves accuracy of model estimates, based on the observed information available to the modeler, should be implemented before using models for their intended purposes. Due to spatial variability, budget constraints or access difficulties, model input parameters always contain uncertainty to some extent. However, a model user has to assign values to each parameter. Parameters that are not well understood may be left unchanged even though they are sensitive or are adjusted to implausible values. Not knowing the sensitivity of parameters can also result in time being uselessly spent on non-sensitive ones. Focus on sensitive parameters can lead to a better understanding and to better estimated values and thus reduced uncertainty. Therefore sensitivity analysis as an instrument for the assessment of the input parameters with respect to their impact on model output is useful not only for model development, but also for model validation and reduction of uncertainty (Hamby 1995; Slišković 1995; Huo *et al*. 2016a, 2016b). There are many different methods of sensitivity analysis. Yet, do they yield equivalent results? Are the same parameter sensitivities identified, regardless of the chosen method? Semi-distributed, long-term, continuous simulation models, such as the Soil and Water Assessment Tool (SWAT), which are capable of describing spatial and temporal variability, should generally be used for complex watershed simulation problems. The SWAT model has more than 200 parameters, most of which have clear physical meaning, and can be determined by measured data (Wang *et al.* 2007; Luo *et al.* 2018). However, in practice there are still a number of parameters that need calibration. For example, calibration is needed for computing a variety of sensitive parameters for runoff generation and convergence parameter estimation. The determination of reasonable parameter values plays an important role in improving the accuracy of the model (Fritz *et al.* 1993; Li *et al.* 2010).

At present there are the genetic algorithm, Bayesian method, regionalized sensitivity analysis and other methods to calibrate the model parameters in SWAT. Ren *et al.* (2011) presented a sensitivity analysis of the SWAT model for the Zengjiang basin in China. Ju *et al.* (2011) also report a sensitivity analysis of the model that they conducted on the Hongmenchuan basin in China. Although these applications mentioned above improved model accuracy, a set of integrated analyses still need to be developed, as model parameter selection often depends on the modeler's experience and the level of understanding of the model. Thus, there are many different methods of sensitivity analysis (Hamby 1995; Beven & Freer 2001; Peng *et al.* 2017). Daily river stage-discharge relationship was predicted using different modeling scenarios. The ensemble empirical mode decomposition (EEMD) algorithm and wavelet transform (WT) were used as a hybrid pre-processing approach. The decomposed sub-series was further broken down into intrinsic mode functions via EEMD to obtain features with higher stationary properties (Roushangar & Alizadeh 2019). How to obtain better accuracy to process sensitivity analysis in the typical region is significant.

Considering the aforementioned issues, the present study compared two different approaches of sensitivity analysis in Xi'an Heihe River basin by using the SWAT model. The main work included: (1) factors such as SCS that influenced the simulated accuracy were analyzed; (2) the sensitivity, accuracy and speed of the simulation between the Sequential Uncertainty Fitting (SUFI-2) algorithm and Shuffled Complex Evolution (SCE-UA) algorithm (Duan *et al.* 2001; Luo *et al.* 2015b) were compared; and (3) the optimal method to improve the operation accuracy and speed was established. This work aims at improving the accuracy of the model simulation and improving the computing speed of the sensitivity analysis for the parameters.

## MATERIALS AND METHODS

### Study area and used data

The Heihe River basin in northwest China consists of an arid to semi-arid climate with a variety of soil types, land uses, and vegetation; there are also significant differences across the basin's microclimates, especially in regard to precipitation and temperature distributions throughout the basin. The catchment area is located in the northern Qinling range district, where the terrain environment is complex and tilts from southwest to northeast. The upstream area consists of the Qinling Mountains, the middle area is defined by the Qinling Mountain diluvial and the downstream area is the Weihe terrace. In recent years, following the development of a city water supply and irrigation project that withdrew water from a reservoir, many development and environmental problems have begun to emerge in the original geographical region benefiting from the Heihe River basin.

The Heihe River in Shaanxi Province is one of the largest tributaries of the Weihe River. It is located between 107°43′–108°24′ E and 33°42′–34°13′ N, and is a second-level tributary of the Yellow River. The Heihe River originates in the southern side of the Qinling Taibai Mountain range and flows from southwest to northeast. The watershed has an area of 2,258 km^{2}. The mountains in this area have steep slopes that enhance quick runoff and soil erosion. Agriculture occurs in the terraces and low-slope valleys. In the agricultural areas, environmental destruction by human activities is obvious (Figure 1).

The Heihe River basin supports approximately 3 million people near the city of Xi'an, Shaanxi province, China. The water resources in the basin are widely utilized for drinking water, navigation, agricultural irrigation, and hydropower, and provide the basis for local livelihoods. For example, approximately 400,000 m^{3} of water are supplied to Xi'an city every day, 370,000 acres of farmland are irrigated each year, and 73,080,000 kWh are generated annually (Li *et al.* 2010; Luo *et al.* 2015a). In recent years, there has been increasing concern that the water resources in the river system may be vulnerable to global climate change, which could have considerable implications for the people living in this region.

The Heihe River basin terrain attribute data were obtained by 1:1,000,000 DEM extractions in the study area, using the Heihe River basin land use map and soil type map, establishing the watershed land use and soil using attribute space database with the support of ArcGIS. Meteorological data consist of measured daily precipitation, maximum and minimum temperature, wind speed and relative humidity from 2005 to 2013. There are 15 Meteorological sites in Zhouzhi County Shaanxi province (Figure 1). Measured runoff data are from the Chenhe observatory for the period of record from 2005 to 2013.

### The SWAT model

SWAT is used worldwide and has been chosen by the Environmental Protection Agency to be one of their Better Assessment Science Integrating point and Nonpoint Sources (BASINS) models (Biondić *et al.* 1997; Whittemore 2000). Thus, it is increasingly used by regulating environmental agencies.

In this paper, the results of two different, relatively simple approaches were compared with the example of the hydrological model SWAT. The SWAT model, developed by the Agricultural Research Service (ARS) of the US Department of Agriculture (USDA), is a distributed watershed hydrological model which is based on physical processes. It can simulate different land-use and various agricultural management measures that influence the migration of waters, sediment, nutrients. A variety of sensitive parameters need calibration in SWAT to ensure proper parameter convergence and runoff calculation. The reasons for calibration include the uncertainty of the model, the basic assumptions of the model, the deviation of the input data and the deviation of the model parameter calculation from the true value resulting from the resolution. Therefore, computation of reasonable values of the model parameters, plays an important role in improving the accuracy of the model (Li *et al.* 2010).

### Model sensitivity analysis

Sensitivity is expressed by a dimensionless index, which is calculated as the ratio between the relative change of model output and the relative change of a parameter. The two investigated approaches differ in the way that the ranges of parameter variation are defined. An alternative approach to define the parameter variation is considered in which the parameters are not varied by a fixed percentage of the initial value but by a fixed percentage of the valid parameter range.

SUFI-2 algorithm is a comprehensive optimization and gradient search method developed by Abbaspour (2007) in 2007. It can not only calibrate multiple parameters at the same time, but also includes a global search function. The algorithm also takes into account the uncertainty of the input data, model structure, parameters and measured data (Tišljar & Velić 1993; Abbaspour 2007; Reicher *et al.* 2008).

The SUFI-2 algorithm assumes a relatively large space that is used to fill parameter values at the onset of the search process, so that the measured data can be included in the 95% prediction uncertainty (95PPU) range. The range of uncertainty intervals is then narrowed gradually, while taking into account the change of the *P*-factor (*P*-factor is the indicator of a credible degree) and *R*-factor (*R*-factor is the indicator of measured variables linear correlation degree). Every change of parameters will result in a recalculation of the sensitivity matrix and covariance matrx. This in turn leads to an update of the parameters at the start the next simulation, resulting in a closer match between the analogy and measured values.

Computation of a *P*-factor value close to 1 or *R*-factor close to 0 determines the better calibration results. Computed values of *R ^{2}* and

*Ens*will in turn serve as the objective function for the model (Liu 2012). The structure of the SUFI-2 process is shown in Figure 2.

The SUFI-2 algorithm comprises the following seven steps: (1) determine the objective function; (2) determine the physical meaning of the parameters and scope of the interval; (3) depending on the selected objective function, perform multiple simulations for each parameter; (4) after the parameter range is determined, start the Latin hypercube sampling; (5) after Latin hypercube sampling, perform simulations with a variety of parameter combinations; (6) evaluate, simulate and calculate for the first step; (7) analyze the uncertainty of the parameters.

Sensitivity analysis is conducted to assess input parameters of the model and to determine the degree of influence, which is the premise of calibrating model parameters. The purpose of this study is to conduct parameter sensitivity analysis by multivariate regression model.

In this study, the Heihe River basin was selected for the sensitivity analysis, employing 11 parameters in SWAT to analyze uncertainty. The physical meaning and initial range of these parameters are shown in Table 1, and an initial parameter range can be determined according to SUFI-2 sensitivity analysis results.

Parameter name . | Meaning . | Initial value . | Final value . | Represent^{a}_(value range)
. |
---|---|---|---|---|

CN2.mgt | SCS runoff curve parameter | 79 | 84 | r_(1.5,14.0) |

ALPHA_BF.gw | Base flow subsided parameter | 0.05 | 0.31 | v_(0.0,0.45) |

SOL_K(1).sol | Saturated hydraulic conductivity parameter | 90 | 58.32 | r_(−0.8,0.8) |

GW_DELAY.gw | Hysteresis parameters of groundwater | 31 | 157.60 | v_(122.0,374.0) |

SOL_BD(1).sol | Moist bulk density | 1.36 | 1.29 | r_(1.1,2.5) |

ESCO.hru | Soil evaporation compensation parameter | 0.95 | 0.90 | v_(0.8,0.9) |

CH_N2.rte | Manning coefficient of main river bed | 0.01 | 0.24 | v_(0.2,0.3) |

SFTMP.bsn | Snowfall temperature parameter | 0 | −1.40 | v_(−4.2,1.0) |

SOL_AWC(1).sol | Available soil water parameter | 0.13 | 0.10 | r_(0,0.5) |

GWQMN.gw | Depth threshold of base flow produced by shallow aquifer | 31 | 0.84 | v_(0.3,1.2) |

CH_k2.rte | Effective hydraulic conductivity in main channel alluvium | 1 | 2.85 | v_(0,45.6) |

Parameter name . | Meaning . | Initial value . | Final value . | Represent^{a}_(value range)
. |
---|---|---|---|---|

CN2.mgt | SCS runoff curve parameter | 79 | 84 | r_(1.5,14.0) |

ALPHA_BF.gw | Base flow subsided parameter | 0.05 | 0.31 | v_(0.0,0.45) |

SOL_K(1).sol | Saturated hydraulic conductivity parameter | 90 | 58.32 | r_(−0.8,0.8) |

GW_DELAY.gw | Hysteresis parameters of groundwater | 31 | 157.60 | v_(122.0,374.0) |

SOL_BD(1).sol | Moist bulk density | 1.36 | 1.29 | r_(1.1,2.5) |

ESCO.hru | Soil evaporation compensation parameter | 0.95 | 0.90 | v_(0.8,0.9) |

CH_N2.rte | Manning coefficient of main river bed | 0.01 | 0.24 | v_(0.2,0.3) |

SFTMP.bsn | Snowfall temperature parameter | 0 | −1.40 | v_(−4.2,1.0) |

SOL_AWC(1).sol | Available soil water parameter | 0.13 | 0.10 | r_(0,0.5) |

GWQMN.gw | Depth threshold of base flow produced by shallow aquifer | 31 | 0.84 | v_(0.3,1.2) |

CH_k2.rte | Effective hydraulic conductivity in main channel alluvium | 1 | 2.85 | v_(0,45.6) |

^{a}v_means the existing parameter value is to be replaced by the given value, a means the given value is added to the existing parameter value, and r_means the existing parameter value is multiplied by (1 + a given value).

The SCE-UA algorithm is a global optimal solution algorithm with consistent, effective and rapid search for hydrological model parameters. It was developed by Duan *et al.* (2003) in 1992, and is combined with the simplex method, the steepest descent method, and a genetic algorithm. Thus the parameter optimization in the application of a watershed hydrological model is very extensive. The algorithm can search the feasible space of all parameters, with a successful finding rate of 100%. The SCE-UA algorithm also has multiple parameters, which are categorized as certain and uncertain. Bai & Liu (2009) recommend choosing appropriate parameters in order to obtain the global optimal solution of the algorithm.

### Calculation of the sensitivity index

*R*and Nash-Sutcliffe efficiency coefficient (

^{2}*Ens*) were selected to evaluate model performance.

*R*represents a consistent level of analog data and measured data, where smaller

^{2}*R*values indicate poor agreement.

^{2}*R*= 1 represents that the simulated and measured data coincide perfectly (Moriasi

^{2}*et al.*2009). The efficiency coefficient

*Ens*is calculated for the model: where is the measured data, is analog data, is the measured average data,

*n*is the number of observations. Values of

*Ens*that approach 1 indicate a close match between measured and simulated model output.

In this study, SWAT-CUP was used to calibrate model parameters (Slišković 1994; Abbaspour 2013); a separate data set was then used for model validation. SWAT-CUP is a program for calibration of SWAT models. The program links SUFI-2 to SWAT. Any of the procedures could be used to perform calibration and uncertainty analysis of a SWAT model. SWAT-CUP also has graphical modules to observe simulation results, uncertainty range, sensitivity graphs, watershed visualization using Bing map, and statistical reports.

## RESULTS AND DISCUSSION

### Calibration results

The monthly runoff data from the Chenhe township hydrological stations was for the period of record from 2005 to 2013. The model calibration period was from 2005 to 2009, while the validation period was from 2010 to 2013 (Figure 3).

The larger the absolute value of the parameter, the more obvious the impact to prediction results of the model. Minus value indicates that the parameter's influence on the prediction results negatively, which means increasing the parameter value will reduce the output of the model. Results of this study show that the SCS (Soil Conservation Service) runoff curve (CN2.MGT), saturated hydraulic conductivity (SOL_K(1).Sol), and effective water content (SOL_AWC) were the most sensitive parameters determined in the analysis of the Heihe River basin (Table 2).

Parameter name . | t-Stat . | P-value
. | Sensitivity . |
---|---|---|---|

CN2.mgt | 17.3 | 0 | 1 |

SOL_K(1).sol | 9.2 | 0 | 2 |

SOL_AWC(1).sol | −6.8 | 0.14 | 3 |

Alpha_BF.gw | 2.9 | 0.16 | 4 |

Esco.hru | −6.8 | 0.12 | 5 |

CH_N2.rte | −2.6 | 0.18 | 6 |

CH_K2.rte | −2.4 | 0.33 | 7 |

GW_DELAY | −1.3 | 0.42 | 8 |

SFTMP | −1.1 | 0.40 | 9 |

GWQMN | −1 | 0.36 | 10 |

SOL_BD(1).sol | −0.9 | 0.49 | 11 |

Parameter name . | t-Stat . | P-value
. | Sensitivity . |
---|---|---|---|

CN2.mgt | 17.3 | 0 | 1 |

SOL_K(1).sol | 9.2 | 0 | 2 |

SOL_AWC(1).sol | −6.8 | 0.14 | 3 |

Alpha_BF.gw | 2.9 | 0.16 | 4 |

Esco.hru | −6.8 | 0.12 | 5 |

CH_N2.rte | −2.6 | 0.18 | 6 |

CH_K2.rte | −2.4 | 0.33 | 7 |

GW_DELAY | −1.3 | 0.42 | 8 |

SFTMP | −1.1 | 0.40 | 9 |

GWQMN | −1 | 0.36 | 10 |

SOL_BD(1).sol | −0.9 | 0.49 | 11 |

As shown in Table 3, the monthly runoff correlation coefficient *R ^{2}* is 0.88 for the calibration period, model efficiency coefficient

*Ens*is 0.86, and

*P*-factor and

*R*-factor are 0.50 and 0.28 respectively. For the validation period, the correlation coefficient

*R*is 0.92, model efficiency coefficient

^{2}*Ens*is 0.92, the

*P*-factor is 0.47 and the

*R*-factor is 0.28. The coefficient and model efficiency coefficient are above 0.8, which suggests that the SWAT model of the Heihe River basin performs well. The monthly mean runoff data were used for the measurements; a hydrological model can be sufﬁciently evaluated by these statistics, and these measures are not oversensitive to extreme values. Small

*P*-factor and

*R*-factor indicate that the number of observed streamflow values that fall within the range of uncertainty is sparse, and therefore the actual uncertainty of the runoff simulation is greater. This may be related to reservoir operation and the transfer of water for agricultural purposes within the basin.

Algorithm . | Time period . | PBIAS . | R^{2}
. | Ens . | P-factor
. | R-factor
. |
---|---|---|---|---|---|---|

SUFI − 2 | Calibration period (2005–2009) | 22% | 0.87 | 0.86 | 0.50 | 0.28 |

Verification period (2010–2013) | 8% | 0.91 | 0.88 | 0.47 | 0.28 | |

SCE-UA | Calibration period (2005–2009) | 17% | 0.86 | 0.84 | 0.42 | 0.34 |

Verification period (2010–2013) | 23% | 0.77 | 0.74 | 0.39 | 0.40 |

Algorithm . | Time period . | PBIAS . | R^{2}
. | Ens . | P-factor
. | R-factor
. |
---|---|---|---|---|---|---|

SUFI − 2 | Calibration period (2005–2009) | 22% | 0.87 | 0.86 | 0.50 | 0.28 |

Verification period (2010–2013) | 8% | 0.91 | 0.88 | 0.47 | 0.28 | |

SCE-UA | Calibration period (2005–2009) | 17% | 0.86 | 0.84 | 0.42 | 0.34 |

Verification period (2010–2013) | 23% | 0.77 | 0.74 | 0.39 | 0.40 |

In this study, the modules of the model operated sensitivity analysis and parameter calibration with the SCE-UA algorithm are presented in Figure 4.

### Comparison of two different approaches of sensitivity analysis

In order to compare the performances of the two different methodologies and test statistically if the difference between two groups is significant, it is necessary to enter the calculation results of two different methodologies by groups. Finally, to test whether the difference between the means of the two groups is statistically significant, we will calculate Include Percent Bias (PBIAS) statistics. PBIAS gives the probability that the incidental difference may happen by chance. Normally, a 0.05 criterion is used as the cut-off value. If the calculated PBIAS probability is equal to or greater than 0.05, then we claim that the difference may happen, meaning that the difference is not statistically significant; if less than 0.05, we claim that the difference is not likely to happen occasionally, meaning that the difference is statistically significant. As shown in Table 3, the calculated PBIAS probability is greater than 0.05. The PBIAS, *Ens* and *R*-factor of SUFI-2 are smaller than SCE-UA in the calibration and verification period. The *R ^{2}, Ens* and

*P*-factor of SUFI-2 method are all larger than SCE-UA for calibration and verification period. Therefore SUFI-2 method is superior to SCE-UA. But we can claim that although there is a difference between the two groups, the difference is not statistically significant; that is, the difference may be caused by chance.

Through the comparison of runoff simulation for the SUFI-2 and SCE-UA algorithms (Table 3), it can be seen that SUFI-2 plays a more important role in the improvement of model accuracy. Compared to the SCE-UA algorithm, the SUFI-2 algorithm is more applicable to the model of runoff simulation in Heihe River basin.

Using the SUFI-2 algorithm, the running time is about 1 min, which compares to about 7 minutes for the SCE-UA algorithm. Thus, the SUFI-2 running rate is seven times faster than the SCE-UA algorithm for a process. Therefore, the efficiency of the SUFI-2 algorithm is improved substantially in making iterative mathematical calculations. The SUFI-2 can be popularized and applied further as a kind of efficient rate method.

Hydrological processes described in SWAT by idealized mathematical empirical formulae introduce a degree of uncertainty that is related to model parameter estimation. Model input data and the observed data used for calibration also introduce uncertainty in model predictions. The SUFI-2 method can be used to account for these uncertainties to obtain a minimum parameter uncertainty range (Schoul *et al.* 2008). It is these advantages that improve computing speed and model accuracy.

Future research on the effect of climate change on watershed ecology should be undertaken using SUFI-2 method to provide the appropriate data. Based on the results of the present comprehensive analysis, the anticipated climate change in the Heihe River basin will adversely impact the water resources in the watershed. To minimize or mitigate these impacts, various adaptation methods have been proposed for the study area. These are independent of whether future climate change will occur as predicted and include (1) more efficient land-use management, (2) better surface and ground water resource development at the local level, and (3) forestation and soil conservation programmes. The results of this research are vital to ensuring that the ever-increasing pressure on water resources can be met with a planned and efficient solution rather than by crisis management.

## CONCLUDING REMARKS

In both sensitivity analysis approaches, the overall greatest importance is attributed to the runoff curve number CN2 parameter, which exhibits very high sensitivity. The initial CN2 value is 79.0 and the final calibrated value is 84.2.

Based on model simulations of the Heihe River basin with the SUFI-2 and SCE-UA algorithms, the test results show the following:

- (1)
Calibration is an important step to improve the accuracy of the model.

- (2)
Based on criteria presented by Moriasi

*et al.*(2009),*R*and^{2}*Ens*values greater than 0.8 suggest that SWAT performed very well on the Xi'an Heihe River basin. - (3)
Compared with the SCE-UA algorithm, the SUFI-2 algorithm exhibits fast convergence, high precision calibration, and significantly improved efficiency of automatic parameter calibration. This in turn results in a substantial increase in the overall effect of the automatic calibration, especially for larger watersheds. The method converges more quickly and the results are more stable, which leads to a greater potential for successful model calibration on large, complex watersheds.

Both sensitivity analysis approaches provide approximately similar results. Although there may be different results in individual cases, the overall ranking of sensitive parameters is more or less the same. Thus an identification of the sensitive parameters is possible, chosen independently from the range in parameter variation. Similar ranking into sensitive and less sensitive parameters are obtained with both methods. Furthermore, as the results are also affected by the employed catchment, the use of simple approaches seems to have advantages, as they are easier to perform.

## ACKNOWLEDGEMENTS

The authors would be grateful to professor Xun-hong Chen for his insightful remarks and comments that improved the manuscript. The authors would be grateful to Lecturer Jucui Wang for her continuous support in the field work. The authors would also be grateful to the reviewers for their insightful remarks for enlightening the manuscript.

## FUNDING

This work was supported by the National Natural Science Foundation of China (Grant Nos. 41672255, 41877232, 41402255, and 41790444); High-tech research cultivation project (Grant Nos. 300102268202 and 300102299201).

## CONFLICTS OF INTEREST

The authors declare that there is no conflict of interest regarding the publication of this paper.