This study compared hydrological model performances under different sub-annual calibration schemes using two conceptual models, IHACRES and HYMOD. In several publications regarding sub-annual calibration, the authors showed that such an approach generally performed better than the conventional whole period method. Hence, there are advantages in dividing the data into sub-annual periods for calibration. However, little attention has been paid to the issue of how to calibrate the non-continuous sub-annual period. Unlike the conventional calibration which assumes time-invariant parameters for the calibration period, the model parameters vary in sub-annual calibration. We have explored two sub-annual calibration schemes, serial calibration scheme (SCS) and parallel calibration scheme (PCS). We assume that the relationships between the rainfall and runoff could be different for each sub-annual period and consider intra-annual variations of the system. The models are then evaluated for a different validation period to avoid over-fitting and the optimal sub-annual calibration period is explored. Overall, we have found that PCS performed slightly better than SCS and the optimal calibration periods are seasonal and bimonthly for IHACRES and biannual for HYMOD. Since there are pros and cons in both SCS and PCS, we recommend choosing the method depending on the purpose of the model usage.

INTRODUCTION

Hydrological modelling is an essential tool for understanding the hydrological behaviour of a catchment (Madsen 2000; Wagener et al. 2003), and it is a complicated task (De Vos et al. 2010). The most common method for identifying the optimised model parameters is through calibration with the use of historical observation data. Objective functions, such as Nash–Sutcliffe efficiency (Nash & Sutcliffe 1970) are used to minimise the difference between the observed and simulated flows. This calibrating scheme is widely applied (Sorooshian 1991; Gan & Biftu 1996; Gupta et al. 1998, 2009). The validation is a standard practice in hydrological modelling (Andréassian et al. 2009) to test the model with the data outside of the calibration period to evaluate the model performance.

A recognised issue in hydrological modelling is uncertainties which are attributed to the model structural errors and parameterisation errors. The uncertainty due to model structure errors is generally quantified by using several different models, and numerous methods are proposed regarding quantification of the uncertainty of parameterisation problems. The time varying parameters, which may arise from catchment change (such as land use/cover change), climate variability and climate change (such as change of evapotranspiration dynamics of vegetation due to higher or lower temperatures) may be another source of uncertainty. Such changes may be significant within a year (e.g., contrasting vegetation cover between winter and summer in many parts of the world). Recently, there have been some studies about the stability of the model performances and the effect of parameter values (Xu 1999; Li et al. 2014; Patel & Rahman 2015; Yan & Zhang 2014). The basis for time varying model parameters can be explained by several reasons (Merz et al. 2011). First, the hydrological model has structure errors and the calibrated parameters may change for different time periods in order to compensate these problems with the model structures (Wagener et al. 2003). Second, catchment characteristic changes (Brown et al. 2005) such as land use and vegetation variations (Merz & Blöschl 2009) can also lead to a change of calibrated parameters. However, the correlation between parameters is complicated (Wagener 2007), which makes it hard to understand the reason for the parameter changes in time (Wagener et al. 2010).

Therefore, most parameterisation schemes in hydrological models are based on the assumption that the parameters do not change for the entire calibration period. There has been some research about the adequate data length for calibration (Xu & Vandewiele 1994; Zeng et al. 2016). In addition, some researchers have attempted to develop more accurate models by adapting sub-annual calibration, which is based on seasonal or monthly time periods (i.e., non-continuous periods) in order to better simulate temporal variations of the varying catchment conditions within the year. In such a scheme, intra-annual variation of the data is taken into account. For example, Luo et al. (2012) examined ten different parameterisation schemes at catchments in Australia. Their results have shown that calibrating the model for each individual month separately produces better model performance than the other schemes, and this is particularly evident for the dry months when the flow is low and difficult to forecast. Levesque et al. (2008) conducted seasonal calibration, in which summer and winter seasons were calibrated separately. When the model was calibrated based on the summer (dry period) data, the model performance improved considerably. However, there was no advantage when only the winter (wet period) data were used for calibration compared with the conventional calibration method which used the entire data over the whole period. Paik et al. (2005), Kim & Lee (2014), Zhang et al. (2015) and Kim et al. (2016) also used the seasonal calibration method.

Similar to the sub-annual discrete calibration, Hartmann & Bardossy (2005) investigated the transferability of hydrological models by dividing the observation period into different climatic conditions (i.e., warm, cold, wet and dry). They conducted the calibration only for the chosen years which were discontinuous, and the model was running continuously for the entire observation period. De Vos et al. (2010) proposed clustering time series according to hydrological similarities and allowing the parameters to vary over the clusters during calibration. Seiller et al. (2012) selected five non-continuous hydrologic years for four contrasting climate conditions: dry/warm, dry/cold, humid/warm and humid/cold. Calibration was done on each period and validation was conducted on contrasting climate conditions. The model was kept running continuously on the entire time series while the optimisation was done only for the chosen years. The above-mentioned studies demonstrate that model parameter values can vary over time in accordance with seasonal variations and indicate that there are some advantages in using this scheme, although further investigation and improvements are needed.

In this study, regarding sub-annual period calibration, we focus on two issues that should be resolved, which have not been considered yet in the literature. First, guidance on calibration schemes for non-continuous time periods should be provided. Most studies have conducted calibrations based on only the chosen sub-annual period (i.e., only the selected sub-annual period data are used for parameter optimisation in the objective function) while the entire time series data were used to run the model continuously. However, the downside of such an approach is that there are discontinuities in the time series of soil moisture when the separately calibrated flows from individual sub-annual periods are combined to examine the performance of the entire time period. Another calibration scheme is to optimise the entire varying parameters simultaneously. De Vos et al. (2010) divided the data into 12 clusters and the model parameters were set differently for each of the 12 clusters, which resulted in the number of degrees of freedom equal to 12 clusters times the number of parameters. Then, these parameters are optimised simultaneously. These different methodologies point towards the need for guidance on the non-continuous sub-period calibration scheme. Second, most non-continuous sub-period calibration studies are not interested in the model performance of the total time series but that of particular time periods (e.g., dry season or wet season). Therefore the questions are, ‘How can we assess the model efficiency of the entire time series using non-continuous sub-annual period calibration? Is it reasonable to just combine the flows calibrated separately?’, ‘In this case, what about the issue of discontinuity in soil moisture simulation?’ Third, different sub-annual calibration periods may present different model efficiencies due to underfitting or overfitting issues and there may be an optimal period which can be evaluated by using cross validation. So far, there is no consensus method in the literature regarding how to calibrate non-continuous sub-annual time periods. Given such a background, this paper explores the following questions:

  • (1) What schemes are applicable for non-continuous sub-annual calibration?

  • (2) Which schemes are both logical and practical?

  • (3) What are the pros and cons of these schemes?

  • (4) What is the optimal time period for sub-annual calibration?

In this study, two sub-annual calibration schemes are employed to explore these questions, i.e., serial calibration and parallel calibration schemes (PCS). The sub-annual calibration has been performed on five different time scales: annual, biannual, seasonal, bimonthly and monthly. We assume that the system is changing due to various reasons, such as vegetation change (e.g., seasonal change of deciduous vegetation and crop rotation), soil structure change (e.g., seasonal soil compaction by farm animals or farming machines), etc. Therefore the system response, i.e., the relationships between the rainfall and runoff may be different for different time scales and there may be an optimal time period for sub-annual calibration. In other words, different sets of varying parameters are optimised to consider intra-annual variations. As the number of sub-annual calibration period increases (i.e., from annual to monthly) the model is more likely to fit to the observations since it has more flexibility to cope with the change of the system. On the other hand, it is more likely to fit to the noise which is the well-known trade-off between the bias and variance in mathematical modelling. Therefore, the models are evaluated for different validation periods to overcome the overfitting (or over-parameterisation) issue and to explore the best sub-annual model.

Both calibration schemes are applied to one catchment located in the southwest of England. Since the main aim of this study is to introduce the concept and the logic of non-continuous sub-period calibration scheme, we believe only one catchment is sufficient to prove the concept. It is hoped that a wide application of the proposed methodology in other catchments under different conditions will help the hydrological community to find useful patterns on this important issue.

CASE STUDY AREA AND THE HYDROLOGICAL MODELS

Study area and data

The Thorverton catchment is used in this study. It has an area of 606 km2 and is a sub-catchment of the Exe catchment. The Exe catchment is located in the southwest of England with an area of 1,530 km2 and an average annual rainfall of 1,088 mm. Figure 1 shows the overview of the Thorverton catchment area. Daily time series of the observed precipitation, potential evapotranspiration and flow data (1961–1990) over the Thorverton catchment was obtained from the UK Met Office and daily temperature data downloaded from the UKCP09 gridded observation data sets.
Figure 1

Location of the Thorverton catchment (the thick highlighted line) in the UK.

Figure 1

Location of the Thorverton catchment (the thick highlighted line) in the UK.

Hydrological models

IHACRES

The first model used in this paper is the conceptual rainfall–runoff model IHACRES (Jakeman & Hornberger 1993) which has eight parameters. This model has been widely applied to a variety of catchments for hydrological analysis and climate impact studies (Jakeman et al. 1993; Littlewood 1999; Letcher et al. 2001; Kim & Lee 2014). The model is composed of a non-linear module and a linear module, as shown in Figure 2, and model parameters are listed in Table 1. A non-linear module converts rainfall to effective rainfall which is calculated from the following equations: 
formula
1
where rk is the observed rainfall, C is the mass balance, l is the soil moisture index threshold and p is the power on soil moisture, respectively. The soil moisture is calculated from: 
formula
2
where is the drying rate given by: 
formula
3
where is the drying rate at reference temperature, f is the temperature modulation, is the reference temperature, and is the observed temperature. A linear module assumes that there is a linear relationship between the effective rainfall and flow. Two components in this module, quick flow and slow flow, can be connected in parallel or in series. In this study, two parallel storages in the linear module are used because they reflect the catchment conditions, and the streamflow at time step k is defined by the following equations: 
formula
4
 
formula
5
 
formula
6
where and are quick flow and slow flow, respectively, and and β are recession rate and peak response, respectively. The relative volumes of quick flow and slow flow can be calculated from: 
formula
7
Therefore, one parameter is determined if the other three parameters are known among the four linear module parameters (αq, αs, βq, βs).
Table 1

Parameters in the IHACRES model

Module Parameter Description Range 
Non-linear c Mass balance 0 to 0.04 
τw Reference drying rate 0 to 50 
f Temperature modulation of drying rate 0 to 4 
l Soil moisture index threshold 0 to 50 
p Power on soil moisture 0 to 3 
Linear αq, αs Quick and slow flow recession rate −1 to 0 
βq, βs Fractions of effective rainfall for peak response − 1 to 0 
Module Parameter Description Range 
Non-linear c Mass balance 0 to 0.04 
τw Reference drying rate 0 to 50 
f Temperature modulation of drying rate 0 to 4 
l Soil moisture index threshold 0 to 50 
p Power on soil moisture 0 to 3 
Linear αq, αs Quick and slow flow recession rate −1 to 0 
βq, βs Fractions of effective rainfall for peak response − 1 to 0 
Figure 2

Structure of the IHACRES model.

Figure 2

Structure of the IHACRES model.

HYMOD

Another conceptual rainfall–runoff model used in this study is HYMOD, which has five parameters. The model consists of a simple rainfall excess model based on the probability distributed principle (Moore 1985) and has been applied by several recent studies (Boyle 2001; Wagener et al. 2001; Vrugt et al. 2003; De Vos et al. 2010). The model parameters are described in Table 2 and the model structure is illustrated in Figure 3. The cumulative distribution function of the water storage capacity C is in the following form: 
formula
8
where is the maximum soil moisture storage capacity in the catchment and controls the degree of spatial variability of the soil moisture capacity. The excess rainfall is treated as the runoff which is divided into quick flow and slow flow based on the partitioning factor a. The runoffs are routed through three identical quick flow tanks and a parallel slow flow tank. The flow rates are determined by the recession coefficient for quick flow tank and slow flow tank
Table 2

HYMOD model parameters

Parameter Unit Range Description 
Cmax mm 1–500 Maximum soil moisture storage capacity 
bexp – 0.01–1.99 Spatial variability of soil moisture capacity 
α – 0.01–0.99 Quick/slow flow distribution factor 
Rs day 0.01–0.99 Recession coefficient for slow flow tank 
Rq day 0.01–0.99 Recession coefficient for quick flow tank 
Parameter Unit Range Description 
Cmax mm 1–500 Maximum soil moisture storage capacity 
bexp – 0.01–1.99 Spatial variability of soil moisture capacity 
α – 0.01–0.99 Quick/slow flow distribution factor 
Rs day 0.01–0.99 Recession coefficient for slow flow tank 
Rq day 0.01–0.99 Recession coefficient for quick flow tank 
Figure 3

Structure of the HYMOD model (adopted from Vrugt et al. (2003)).

Figure 3

Structure of the HYMOD model (adopted from Vrugt et al. (2003)).

METHODOLOGY

Optimisation method

For the optimisation algorithm, we used dynamically dimensioned search (DDS) (Tolson & Shoemaker 2007) which is a simple, single objective, heuristic global search algorithm. DDS searches the parameter space globally and incrementally localises as the number of iterations reaches the maximum allowable number of simulations. The procedure from global to local scales is done by probabilistically decreasing the number of model parameters in the neighbourhood. New search avoids poor local optima and parameter values are updated by perturbing the current solution values in the randomly selected dimensions with perturbation magnitudes randomly sampled from a normal distribution with a zero mean. More details can be found in Tolson & Shoemaker (2007).

Model parameterisation schemes

The calibration has been conducted with the use of root mean square error (RMSE) to minimise the difference between the observed and simulated flow and the optimised parameters have been tested in the validation period: 
formula
9
where and are the simulated and observed runoff, respectively. i is the ith day, and N is the number of days in the calibration period. In this study, only RMSE is used as the objective function. Exploration of the effect of different objective functions and their combinations in calibrating rainfall–runoff models can be another research topic to extend the current study (e.g., Jie et al. 2015).
We conducted three different calibration schemes, i.e., the conventional whole period calibration scheme and two dynamic sub-annual calibration schemes. The common assumption of the conventional calibration scheme is to use time-invariant parameters. The model parameters do not change during the entire calibration period. On the other hand, the dynamic calibration schemes are based on the idea that the optimised parameter values can vary according to different climatic and catchment conditions. In this study, we have assumed sub-annually (annual, biannual, seasonal, bimonthly and monthly) varying parameters. With regard to a monthly model, for instance, the model parameters are the same for each month for all years but differ between the months. This scheme can be conducted in two different ways: serial calibration scheme (SCS) and PCS. Among five different time scales for sub-annual calibration, illustrations of the monthly calibration schemes are presented in Figure 4. SCS optimises the whole parameter sets simultaneously, i.e., 12 monthly models are calibrated at the same time. For example, 96 parameters (eight parameters times 12 sets) in IHACRES and 60 parameters (five parameters times 12 sets) in HYMOD are optimised simultaneously. On the other hand, PCS optimises 12 different monthly models separately. Only the data from individual months are used in the objective function while the model is running for the whole period. The calibrated flow data from each separate monthly model are then combined together to compare with the observed flow.
Figure 4

Illustration of the conventional calibration, SCS and PCS for the calibration period from 1961 to 1970.

Figure 4

Illustration of the conventional calibration, SCS and PCS for the calibration period from 1961 to 1970.

Calibration has been done for three time periods: 1960s, 1970s and 1980s. Normally a warm-up period (e.g., 1 or 2 years) is used during calibration to reduce the influence of the initial values of state variables. In this study, instead of using a warm-up period, we set the initial value of soil moisture as a parameter to be optimised to avoid the warm-up problem (this is fine for calibration, but not suitable for validation in which a warm-up period of 1 year is still needed because the initial value of soil moisture is not a model parameter).

Evaluation of the parameterisation schemes and the optimal calibration period

To explore the optimal time period (i.e., optimal number of groups) of sub-annual calibration, which is based on a balance between the bias and the variance (Figure 5), the model performance is evaluated in the validation period with the use of RMSE. If the sub-annual calibration period is too long it may not be flexible enough to capture the change of the system and may lose the temporal information due to underfitting (high bias and low variance). On the other hand, if the period is too short, even the noise of the system will be matched due to overfitting (low bias and high variance). Hence, it is possible that there could be an optimal sub-annual calibration period; thus far, there are no reported studies regarding this issue.
Figure 5

Trade-off between the bias and the variance to explain the model overfitting and underfitting (Han 2011).

Figure 5

Trade-off between the bias and the variance to explain the model overfitting and underfitting (Han 2011).

The three-fold cross validation is applied to evaluate the performance of the model and to find the optimal sub-annual calibration period (Figure 6). The RMSE values are averaged for all six validation periods.
Figure 6

Three-fold cross validation scheme (θ is optimised parameter set).

Figure 6

Three-fold cross validation scheme (θ is optimised parameter set).

Sub-annually optimised parameters from SCS and PCS can be applied in two ways for validation: serial run and parallel run. For a monthly model, for instance, the serial run is to apply the monthly varying parameters in series at one model run, while the parallel run is to apply 12 sets of parameters individually (i.e., run from January parameters to December parameters separately). Therefore, the parameters from SCS can be applied in series and the parameters from PCS can be applied in parallel. The parameters optimised from SCS and PCS are expressed as follows: 
formula
10
where y is the calibration period, c is the type of calibration scheme and m is the number of groups (e.g., 12 for monthly calibration).
For example, 1960s monthly parameter sets from SCS are applied to the validation period in which the monthly parameters are switched every month while the model runs continuously ((1) in Figure 7). One hydrograph is produced in this way. On the other hand, applying monthly parameter sets from PCS in parallel runs result in 12 hydrographs. Each monthly parameter set is applied separately while the model is run for the whole validation period ((2) in Figure 7). Note that in this figure, the hydrographs are represented only for the corresponding month for illustration purposes, although the hydrographs are continuous in reality. The flow data from each month period are picked up and combined to build the entire flow data for comparison with the observed flow.
Figure 7

Illustration of the evaluation of 1960s monthly calibrated parameters. The flows are simulated by applying the parameters to 1970s (or 1980s) rainfall.

Figure 7

Illustration of the evaluation of 1960s monthly calibrated parameters. The flows are simulated by applying the parameters to 1970s (or 1980s) rainfall.

As previously noted, the initial value of the soil moisture is set as if it is a ‘model parameter’ and is optimised during calibration. However, in reality, the soil moisture is not a hydrological model parameter but a state variable, hence, the optimised initial soil moisture value estimated from the calibration period should not be used in the validation period. If we know the correlation between the soil moisture and the flow, the initial soil moisture value in the validation period could be estimated from the initial flow value. However, since this is not the main point of the study, we just assume that the initial soil moisture value evolves to an appropriate value after 1 year instead of building the soil moisture and flow relationship. Therefore, the first 1 year of the validation period has not been taken into account in evaluating the model efficiency.

RESULTS OF MODEL EVALUATION

Figure 8 presents the model's performance for SCS and PCS. The calibration and validation results show the typical trend of bias trade-off (Figure 5). The more calibration groups we divide the data into the more flexible the model becomes and the smaller the error becomes for the calibration scheme. However, the improvement in the calibration period comes with the cost of worsened validation results due to overfitting (or over-parameterisation). For the study catchment, the optimal sub-annual calibration periods, where the RMSE of the validation result is the smallest, are bimonthly and seasonal for SCS and PCS, respectively, for the IHACRES. For the HYMOD, biannual is the optimal period for both SCS and PCS. The implication of this result is that it is reasonable to assume that the catchment response is gradually changing within the year, and to capture the change of the system intra-annual variation should be considered for calibrating a hydrological model.
Figure 8

Calibration and validation results for SCS and PCS for IHACRES and HYMOD.

Figure 8

Calibration and validation results for SCS and PCS for IHACRES and HYMOD.

Figure 9 compares the SCS and PCS performance of the validation results. It is reasonable to suppose that the SCS will be better than the PCS since the SCS has a sound logic by continuously running the model like the real system from the first to the last in series, while the PCS considers only the chosen time period for estimating the model efficiency although each sub-annual model runs continuously. However, both IHACRES and HYMOD show that the PCS is better than SCS. This might be because the parameters in PCS are adjusted to provide the best fit to one specific period without being affected by other periods, where the optimal parameter values are different, while the SCS has an issue of optimisation in high dimensional spaces and the interdependency of numerous parameters.
Figure 9

Comparison of SCS and PCS performance for IHACRES and HYMOD.

Figure 9

Comparison of SCS and PCS performance for IHACRES and HYMOD.

DISCUSSION

Sub-annual calibration and over-parameterisation

There is a possible concern that the sub-annual calibration scheme has an over-parameterisation (over-fitting) problem. For example, the monthly calibration scheme in IHACRES has 96 parameters (eight parameters times 12 sets), which could be recognised as 12 monthly models of eight parameters equivalent to one model of 96 parameters. However, this argument is not exactly right due to misinterpretation of multiplying the model parameter numbers by the number of models. Figure 10 explains why the two concepts are not equivalent. Suppose we have a linear model which is calibrated on a monthly basis. Each monthly model has one parameter (a1, a2…a12). However, when these 12 separately calibrated models are combined (Figure 10(a)) to evaluate the annual performance, it is apparent that this combined model is different to a single model with 12 parameters (Figure 10(b)) since the model structure between the two is different. Therefore the two models, a combined model and one model with 12 parameters are not equivalent and should not be confused.
Figure 10

Schematic of 12 monthly models of one parameter and one model of 12 parameters.

Figure 10

Schematic of 12 monthly models of one parameter and one model of 12 parameters.

We have done an experiment with a synthetic flow to prove that these two models are not equivalent. It is assumed that the system does not change in order to simplify the problem and SCS is applied for five different time scales: annual, biannual, seasonal, bimonthly and monthly. To make a stationary system, we assumed the 1960s parameters calibrated in the conventional method as a ‘true parameter set’ which are used to generate the ‘true flow’. A random error (noise) is added to the ‘true flow’ to produce real flow data (Equation (11)): 
formula
11
where is the random error added flow, is the ‘true flow’, is a random error intensity coefficient, which is 0.2 in this study, and μ is a random sample from a Gaussian distribution. This flow series generated from a stationary system is then used to calibrate the hydrological model, IHACRES.
The hypothesis is that a simple model (m parameters) which is separately calibrated on a sub-annual basis (n groups) is equivalent to a complicated single model with m × n parameters (Equation (12)): 
formula
12
The hypothesis will be rejected if the errors are similar even though the number of sub-annual calibration groups (n) increases. This is because given that the hypothesis is true, the model becomes more complicated (i.e., one model with m × n parameters) when the number of groups n increases, which results in less error due to overfitting (Figure 11). The calibration has been performed on an annual, biannual, seasonal, bimonthly and monthly time scale based on SCS. The model performances are presented in Table 3. The RMSE values are similar among different time scales which mean that the sub-annual calibration scheme does not improve the model performance. Therefore, the hypothesis should be rejected. The reason for no improvements in the complicated model is that the model structures are still the same although the sub-annual calibration has been performed in different time scales. Hence, the sub-annually (n groups) calibrated model (m parameters) is not equivalent to a single model with m × n parameters.
Table 3

RMSE of different calibration period

Calibration period Annual Biannual Seasonal Bimonthly Monthly 
RMSE (m3/s) 4.24 4.23 4.31 4.40 4.33 
Calibration period Annual Biannual Seasonal Bimonthly Monthly 
RMSE (m3/s) 4.24 4.23 4.31 4.40 4.33 
Figure 11

Schematic of overfitting issue when the number of groups increases: (a) a model with one parameter; (b) a model with two parameters; (c) a model with four parameters.

Figure 11

Schematic of overfitting issue when the number of groups increases: (a) a model with one parameter; (b) a model with two parameters; (c) a model with four parameters.

Another possible concern is that a time-variant parameter is not a parameter any more but a state variable. This should not be a problem because such a time-variant parameter scheme has already been widely adopted in the ‘adaptive control’ (Tao 2003) which is used to control the system with varying parameters.

Calibration of nonstationary system and the optimal calibration period

As mentioned in the previous section, if the system is stationary, there is no advantage in applying the sub-annual calibration scheme. However, in real life the system is nonstationary and it is changing with time. The response of the nonstationary system may be different dependent on the catchment change (e.g., seasonal vegetation change, seasonal soil structure change, etc.) which means that different hydrological model parameters may be needed. In this changing system, the more calibration groups divided, the less the error will be since the model has more flexibility to fit to the observation. Since there is a trade-off between bias and variance, there is an optimal calibration period between annual and monthly models, as shown in Figure 8. It is understandable that IHACRES (eight parameters) has a shorter optimised calibration period than HYMOD (five parameters) since IHACRES has more parameters than HYMOD, which can cope with the change of the system better.

Comparison of soil moisture

As stated in the Introduction, the sub-annual parameterisation scheme mostly applied in hydrological model calibration is PCS. The model is kept running continuously for the entire time series while the optimisation is done only for the chosen sub-annual period (e.g., monthly or seasonal). However, this method has a drawback. Figure 12 presents a part of the time series of soil moisture from the monthly model for SCS and PCS. It is apparent that in the PCS the time series shows discontinuity at each monthly boundary, which is illogical and a downside of this method although the flow simulation performance is excellent, as presented in previous sections. On the other hand, in the SCS the time series is continuous, which is a realistic simulation. This is because in the serial calibration, the model runs continuously, while in the parallel calibration, although each monthly model runs continuously only the corresponding month data are picked up and combined together.
Figure 12

Time series of soil moisture for different calibration schemes.

Figure 12

Time series of soil moisture for different calibration schemes.

To see whether the time varying parameter is reasonable between different time periods of calibration, we investigated the state variable, soil moisture for IHACRES and HYMOD. In Figure 13, it can be seen that among sub-annual calibration schemes, the patterns of soil moisture of monthly and seasonal calibrations have been ruined compared with the conventional annual calibration method (low in summer, high in winter), while the biannual calibration result shows a similar pattern. From the flow point of view, the optimal calibration period for SCS is the bimonthly model (Figure 9(a)) while, from the soil moisture point of view, at least the biannual model should be used although the RMSE between the flows is not the least in this case. If the calibration period is divided more in detail than the biannual period, the pattern of soil moisture would not be realistic.
Figure 13

Time series of soil moisture from 1965 to 1967 in different calibration periods for IHACRES.

Figure 13

Time series of soil moisture from 1965 to 1967 in different calibration periods for IHACRES.

In contrast to the IHACRES, the HYMOD generates more reasonable soil moisture patterns (Figure 14). Although the sub-annual calibration period is divided in more detail, the pattern of soil moisture is maintained. Therefore, the biannual calibration period which is optimal for SCS (Figure 9(b)) is valid for HYMOD.
Figure 14

Time series of soil moisture from 1965 to 1967 for different calibration periods for HYMOD.

Figure 14

Time series of soil moisture from 1965 to 1967 for different calibration periods for HYMOD.

From this result, it is found that realistic flow and soil moisture simulations may not both be achieved at the optimal state. Sometimes we may get the right answers (predicting the best flow for practical purposes) for the wrong reasons (unrealistic soil moisture). That is, the flow may be optimum at the expense of soil moisture due to the drawback of the hydrological model itself. There has been relevant research regarding this issue by Zhuo & Han (2016a, 2016b).

Which calibration scheme is more reliable

We have explored three options of calibration schemes for hydrological modellers to choose from: (1) the conventional method considering inter-annual variations of the data, (2) SCS and (3) PCS considering intra-annual variations of the data. Logically, the SCS should be better than the PCS since the model should run in series by logic, but the results show that the PCS beats the SCS. A possible interpretation of this result may be in part due to the issue of optimisation in a high dimensional space and the interdependency of numerous parameters in SCS.

Then the question is which method is more reliable? Our recommendation depends on the purpose of the calibration. If one is interested in the volume of the flow water, we recommend the PCS since it represents the best flow simulation result and is easy to calibrate due to the small number of parameters, although there are discontinuities in the time series of soil moisture. On the other hand, when one is interested in soil moisture as well as flow, we recommend the SCS which is more logical and the performance is not much worse in flow but shows continuity in soil moisture simulation. However, the downside of this method is that it is not efficient and takes a longer time to calibrate than PCS since the number of degrees of freedom equals the number of sub-annual period (e.g., 12 for monthly calibration) times the number of parameters.

CONCLUSIONS

This study has compared the hydrological model performances under different sub-annual period calibration schemes using two conceptual models, IHACRES and HYMOD. There are some studies regarding sub-annual period calibration methods, but little attention has been paid to the issue of how to calibrate (e.g., in series or in parallel) the non-continuous time series and what is an optimal time period for sub-annual calibration. It is, therefore, important to investigate reliable calibration schemes for non-continuous sub-annual period calibration. In this study, we have proposed two alternative approaches to calibrate hydrological models by sub-annual calibration schemes in which unique model parameter sets are estimated for each sub-period (annual, biannual, seasonal, bimonthly and monthly). In one approach, unique parameter sets for each sub-period are calibrated simultaneously and parameter values are thus changed for each sub-period – the SCS. In the second approach, the model is calibrated n (the number of groups, e.g., four for seasonal) times but only using data from one sub-period at the time. The n models are then combined to get the result of a complete time series – the PCS. Then, three-fold cross validation is used to find the optimal sub-annual calibration period; it is possible that this optimal calibration period is related to local catchment change and the purpose of the data usage. Overall, we have found that the model calibrated on the sub-annual period schemes generally performs better than the model calibrated in the conventional way, which implies that it is worth considering intra-annual variations in calibrating hydrological models in a changing environment. For the study catchment, from the flow point of view (predicting flow only for practical purposes), the optimal calibration periods for IHACRES are bimonthly and seasonal for SCS and PCS, respectively. For HYMOD, biannual is the best sub-annual model for both SCS and PCS. However, from the soil moisture point of view, dividing the sub-annual calibration period sometimes may not produce a realistic soil moisture pattern, which indicates improvement of the hydrological model structure is needed to achieve both the simulated flow and soil moisture at the optimal state. Therefore, not only the simulated flow but also the soil moisture needs to be considered to find the optimal calibration period. Among the dynamic calibration schemes, PCS performed slightly better than SCS. Since there are pros and cons in both SCS and PCS, we recommend choosing the method depending on the purpose of the sub-period calibration. Although the study catchment is specific to southwest England, the methodology proposed in this study is generic and applicable to other catchments. Since only one catchment is explored in this investigation, it is clear that such a study has not completely solved this problem. We hope this paper will stimulate the hydrological community to explore a variety of sites with different hydrological models so that valuable experiences and knowledge can be gained to improve our understanding about such a complex model calibration issue.

ACKNOWLEDGEMENTS

The first author is grateful for the financial support from the Government of Republic of Korea in carrying out his PhD study at the University of Bristol. The data used in this study are available upon request from the corresponding author via email (kk12496@bristol.ac.uk).

REFERENCES

REFERENCES
Andréassian
V.
Perrin
C.
Berthet
L.
Le Moine
N.
Lerat
J.
Loumagne
C.
Oudin
L.
Mathevet
T.
Ramos
M.
Valéry
A.
2009
Crash tests for a standardized evaluation of hydrological models
.
Hydrology and Earth System Sciences Discussions
13
,
1757
1764
.
Boyle
D. P.
2001
Multicriteria Calibration of Hydrologic Models
.
PhD Dissertation
,
University of Arizona
,
Tucson, AZ
,
USA
.
Brown
A. E.
Zhang
L.
McMahon
T. A.
Western
A. W.
Vertessy
R. A.
2005
A review of paired catchment studies for determining changes in water yield resulting from alterations in vegetation
.
Journal of Hydrology
310
,
28
61
.
Han
D.
2011
Flood Risk Assessment and Management
.
Bentham Science Publishers
,
Sharjah
,
United Arab Emirates
.
Hartmann
G.
Bárdossy
A.
2005
Investigation of the transferability of hydrological models and a method to improve model calibration
.
Advances in Geosciences
5
,
83
87
.
Jakeman
A.
Hornberger
G.
1993
How much complexity is warranted in a rainfall-runoff model?
Water Resources Research
29
,
2637
2649
.
Jie
M. X.
Chen
H.
Xu
C. Y.
Zeng
Q.
Tao
X. E.
2015
A comparative study of different objective functions to improve the flood forecasting accuracy
.
Hydrology Research
.
doi:10.2166/nh.2015.078.
Kim
K. B.
Kwon
H.-H.
Han
D.
2016
Hydrological modelling under climate change considering nonstationarity and seasonal effects
.
Hydrology Research
47
,
260
273
.
Letcher
R.
Schreider
S. Y.
Jakeman
A.
Neal
B.
Nathan
R.
2001
Methods for the analysis of trends in streamflow response due to changes in catchment condition
.
Environmetrics
12
,
613
630
.
Levesque
E.
Anctil
F.
Van Griensven
A.
Beauchamp
N.
2008
Evaluation of streamflow simulation by SWAT model for two small watersheds under snowmelt and rainfall
.
Hydrological Sciences Journal
53
,
961
976
.
Li
H.
Beldring
S.
Xu
C.-Y.
2014
Stability of model performance and parameter values on two catchments facing changes in climatic conditions
.
Hydrological Sciences Journal
60
(
7–8
),
1317
1330
.
Paik
K.
Kim
J. H.
Kim
H. S.
Lee
D. R.
2005
A conceptual rainfall-runoff model considering seasonal variation
.
Hydrological Processes
19
,
3837
3850
.
Seiller
G.
Anctil
F.
Perrin
C.
2012
Multimodel evaluation of twenty lumped hydrological models under contrasted climate conditions
.
Hydrology and Earth System Sciences
16
,
1171
1189
.
Sorooshian
S.
1991
Parameter estimation, model identification, and model validation: conceptual-type models
. In:
Recent Advances in the Modeling of Hydrologic Systems
(
Bowles
D. S.
Enda O'Connell
P.
, eds).
Springer
,
The Netherlands
, pp.
443
467
.
Tao
G.
2003
Adaptive Control Design and Analysis
.
John Wiley & Sons
,
Hoboken, NJ
,
USA
.
Vrugt
J. A.
Gupta
H. V.
Bouten
W.
Sorooshian
S.
2003
A Shuffled Complex Evolution Metropolis algorithm for optimization and uncertainty assessment of hydrologic model parameters
.
Water Resources Research
39
(
8
),
1201
.
Wagener
T.
Boyle
D. P.
Lees
M. J.
Wheater
H. S.
Gupta
H. V.
Sorooshian
S.
2001
A framework for development and application of hydrological models
.
Hydrology and Earth System Sciences Discussions
5
,
13
26
.
Wagener
T.
McIntyre
N.
Lees
M.
Wheater
H.
Gupta
H.
2003
Towards reduced uncertainty in conceptual rainfall-runoff modelling: dynamic identifiability analysis
.
Hydrological Processes
17
,
455
476
.
Wagener
T.
Sivapalan
M.
Troch
P. A.
McGlynn
B. L.
Harman
C. J.
Gupta
H. V.
Kumar
P.
Rao
P. S. C.
Basu
N. B.
Wilson
J. S.
2010
The future of hydrology: an evolving science for a changing world
.
Water Resources Research
46
(
5
),
W05301
.
Xu
C.-Y.
Vandewiele
G.
1994
Sensitivity of monthly rainfall-runoff models to input errors and data length
.
Hydrological Sciences Journal
39
,
157
176
.
Zeng
Q.
Chen
H.
Xu
C.-Y.
Jie
M.-X.
Hou
Y.-K.
2016
Feasibility and uncertainty of using conceptual rainfall-runoff models in design flood estimation
.
Hydrology Research
47
(
4
),
701
717
.
Zhang
D. J.
Chen
X. W.
Yao
H. X.
Lin
B. Q.
2015
Improved calibration scheme of SWAT by separating wet and dry seasons
.
Ecological Modelling
301
,
54
61
.