Abstract

The current research introduces a combined wavelet-emotional artificial neural network (WEANN) approach for one-time-ahead rainfall-runoff modeling of two watersheds with different geomorphological and land cover conditions at daily and monthly time scales, to utilize within a unique framework the ability of both wavelet transform (to mitigate the effects of non-stationary) and emotional artificial neural network (EANN, to identify and individualize wet and dry conditions by hormonal components of the artificial emotional system). To assess the efficiency of the proposed hybrid model, the model efficiency was also compared with so-called EANN models (as a new generation of ANN-based models) and wavelet-ANN (WANN) models (as a multi-resolution forecasting tool). The obtained results indicated that for daily scale modeling, WEANN outperforms the other models (EANN and WANN). Also, the obtained results for monthly modeling showed that WEANN could outperform the WANN and EANN models up to 17% and 35% in terms of validation and training efficiency criteria, respectively. Also, the obtained results highlighted the capability of the proposed WEANN approach to better learning of extraordinary and extreme conditions of the process in the training phase.

INTRODUCTION

Modeling of rainfall-runoff (r-r) processes performed by hydrologists can be helpful in obtaining information for environmental planning, flooding, and water resources management. A reliable r-r model can also alleviate drought influence on water resources, an important issue in arid and semi-arid areas. For this purpose, black-box modeling can be considered as an alternative with regard to fully distributed models because of the complex unknown and abstruse factors involved in the process (Salas et al. 1980; Solomatine & Ostfeld 2008; Adamowski et al. 2012a, 2012b; Danandeh Mehr et al. 2017; Farajzadeh & Alizadeh 2018).

The hydrological processes (e.g., r-r process) are considered to be complex systems due to the effects and interaction of the different spatio-temporal factors involved (Nayak et al. 2004). Recent studies have shown that classic statistical models such as auto-regressive integrated moving average (ARIMA) may not be capable of simulating the r-r process adequately due to the non-linear characteristics of the process. Nowadays, the artificial neural network (ANN), known as a self-learning and self-adaptive function approximator, has been widely used for modeling non-linear hydrological time series due to benefiting from the black box feature (no requirement of prior knowledge), applying a non-linear function to handle the non-linear properties of the process and the ability of analyzing multivariate inputs with different characteristics. This has led to a huge leap in the use of ANN models in hydrological simulations (e.g., Hsu et al. 1995; ASCE 2000; Maier & Dandy 2000; Dawson & Wilby 2001; Jain & Srinivasulu 2006; Zhang et al. 2011; Adamowski et al. 2012a, 2012b; Feng et al. 2017; Nourani et al. 2017).

In spite of the flexible nature of ANN in modeling hydrological processes, this algorithm may exhibit defects in dealing with complex hydrological signals. Therefore, in such a situation, spatial or temporal pre-processing of data can be a necessary step to overcome such problems. The ability of wavelet transform in decomposing complex hydrological time series to sub-series by extracting useful information at different scales can be effective for interpreting hydrological phenomena (Sang 2013; Kumar & Sahay 2018). The hybrid wavelet-ANN model (WANN) is a well designed method that utilizes the wavelet transform to obtain the different frequencies of r-r process and forecast the future runoff at desired scale by ANN (Kuo et al. 2010; Shiri & Kisi 2010; Kisi & Cimen 2011; Adamowski et al. 2012a, 2012b; Sang 2013; Nourani et al. 2015). The benefits of the hybrid Wavelet-Artificial Intelligence (AI) models (such as the ability of wavelet transform in multi-resolution analysis, de-noising and edge effect detection of a signal and the strong capability of AI methods in optimization and prediction of the processes) have been highlighted in a review paper by Nourani et al. (2014).

Recently a new generation of ANN models has been proposed by incorporating the artificial emotions into the classic ANN framework as emotional ANN (EANN) models (Khashman 2008; Lotfi & Akbarzadeh 2014, 2016). From a biological point of view, the neurophysiological response of animals can be affected by hormonal activities, so that the animals may provide different actions for the same task at different moods. Inspired by this biological concept and merging artificial emotion and ANN, the learning ability of the network could be enhanced due to the feedback loop between systems of hormones and neurons. As the first hydrological implementation of EANN, Nourani (2017) proposed revised Back Propagation (BP) training algorithm of a Multi-Layer Perceptron (MLP) network by incorporating emotional anxiety concept and investigated the efficiency of the EANN model to cope with the shortage of long observed training time series. In general, the advantages of EANN models in comparison with the other statistical and black box methods can be categorized as: the ability of EANN to cope with the lack of long observed data used for network training; the EANN model could lead to more accurate estimations of peak values; and emotional parameters of an EANN dynamically get/give information from/to inputs and outputs of the network at each time step to distinguish the extreme events (e.g. dry and wet days).

In spite of the flexible nature of EANN in coping with the lack of long observed data used for network training, clearly just like any other data-driven time series forecasting method, the performance of the EANN can be affected by the presence of anthropogenic and/or climatic influences and shifts of the observed time series. In the presence of such a strong non-stationary time series, reliable data preprocessing approaches may be employed prior to performing the forecasts. In this way, as a novel strategy, wavelet-based data-processing approach with the ability of multi-resolution analysis was linked to the EANN to enhance the modeling efficiency. In this study, the wavelet-EANN (WEANN) approach for r-r modelling (due to the ability of wavelet transform to mitigate the effects of non-stationary time series) was combined with the ability of EANN to identify and individualize wet (rainy days) and dry (rainless days) conditions by hormonal parameters of the artificial emotional system. The data for two watersheds with different geomorphological conditions were used to indicate the performance of the proposed WEANN approach for r-r modeling. For this purpose, at the first step the data were decomposed into sub-signals using wavelet transform. After that, the obtained sub-signals (as inputs) were applied into the EANN model to reconstruct the main and original time series. Finally, to evaluate the model capability, the results of the proposed WEANN model were also compared with the results of EANN and WANN models.

MATERIALS AND METHODS

Study area and dataset

In this paper, data for two watersheds with different geomorphological conditions were used for the modeling purpose, West Nishnabotna River (sub-basin of Missouri River) and Trinity River (sub-basin in California, United States). Two catchments with two different geomorphological conditions show almost distinct responses to the rainfall.

The Trinity River is a major branch of the Klamath River which flows through the Coast Ranges and Klamath Mountains (in northwestern California) (Figure 1). The river is at 123.42°W and 41.11°N and the longest stream length of the river is about 266 km with watershed area nearly 7,800 km2. The catchment elevation varies between 58 m (where the Trinity is surrounded by the Klamath River) and 2,709 m (Sawtooth Peak in the Trinity Alps). About 92% of the watershed is covered by oak, fir, and pine forests and the region on every side of the river has a population with 1.7 people per km2 and for this reason the basin is not much affected by humans. The overall climate regime of the basin is Mediterranean and it has hot and dry summers, but cool and wet winters. The coldest and hottest months are December and July with average temperatures of 4.6 °C and 28.2 °C, respectively, and the mean temperature is 15.5 °C; the driest and wettest months of year of the watershed are August and March, respectively. Thirty-five years of daily flow and monthly rainfall data (1981–2015) observed at the catchment outlet station and a gauging station located upstream of the outlet (see Figure 1) were retrieved via http://waterwatch.usgs.gov and used in this study. The statistics of the used daily and monthly data for the study period (1981–2015) are presented in Table 1. The dataset was split into two different parts: the first 75% of data (1981–2006) were allocated to calibration (training) and the rest (2006–2015) were used as validation (testing) set.

Table 1

The statistics of the observed daily and monthly time series for Trinity River and West Nishnabotna River watersheds

Scale Time series Statistical parameter Watershed
 
Trinity River
 
West Nishnabotna River
 
Calibration Validation Calibration Validation 
Daily Rainfall (mm) Mean 2.38 1.93 2.25 2.56 
Maximum 78.00 82.60 192.00 104.10 
Minimum 
Standard deviation 6.88 6.13 7.40 8.28 
Coefficient of variation (dimensionless) 2.89 3.17 3.29 3.23 
Sample sizes 9,375 3,126 11,160 3,721 
Runoff (m3/s) Mean 145.72 109.02 21.02 28.39 
Maximum 2,860.00 1,475.31 730.57 654.12 
Minimum 12.26 14.33 1.30 2.63 
Standard deviation 214.83 118.60 31.76 36.30 
Coefficient of variation (dimensionless) 1.47 1.09 1.51 1.28 
Sample sizes 9,375 3,126 11,160 3,721 
Monthly Rainfall (mm) Mean 68.69 58.95 68.58 78.08 
Maximum 462.40 289.40 291.60 320.50 
Minimum 
Standard deviation 83.16 67.27 54.91 70.37 
Coefficient of variation (dimensionless) 1.21 1.14 0.80 0.91 
Sample sizes 305 102 363 122 
Runoff (m3/s) Mean 4,454.33 3,326.40 641.53 859.65 
Maximum 28,299.86 12,895.78 5,580.12 5,105.81 
Minimum 377.94 232.76 56.86 123.07 
Standard deviation 5,079.00 2,932.42 657.64 829.08 
Coefficient of variation (dimensionless) 1.14 0.88 1.03 0.96 
Sample sizes 305 102 363 122 
Scale Time series Statistical parameter Watershed
 
Trinity River
 
West Nishnabotna River
 
Calibration Validation Calibration Validation 
Daily Rainfall (mm) Mean 2.38 1.93 2.25 2.56 
Maximum 78.00 82.60 192.00 104.10 
Minimum 
Standard deviation 6.88 6.13 7.40 8.28 
Coefficient of variation (dimensionless) 2.89 3.17 3.29 3.23 
Sample sizes 9,375 3,126 11,160 3,721 
Runoff (m3/s) Mean 145.72 109.02 21.02 28.39 
Maximum 2,860.00 1,475.31 730.57 654.12 
Minimum 12.26 14.33 1.30 2.63 
Standard deviation 214.83 118.60 31.76 36.30 
Coefficient of variation (dimensionless) 1.47 1.09 1.51 1.28 
Sample sizes 9,375 3,126 11,160 3,721 
Monthly Rainfall (mm) Mean 68.69 58.95 68.58 78.08 
Maximum 462.40 289.40 291.60 320.50 
Minimum 
Standard deviation 83.16 67.27 54.91 70.37 
Coefficient of variation (dimensionless) 1.21 1.14 0.80 0.91 
Sample sizes 305 102 363 122 
Runoff (m3/s) Mean 4,454.33 3,326.40 641.53 859.65 
Maximum 28,299.86 12,895.78 5,580.12 5,105.81 
Minimum 377.94 232.76 56.86 123.07 
Standard deviation 5,079.00 2,932.42 657.64 829.08 
Coefficient of variation (dimensionless) 1.14 0.88 1.03 0.96 
Sample sizes 305 102 363 122 
Figure 1

Map of Trinity River watershed.

Figure 1

Map of Trinity River watershed.

The second study area, the West Nishnabotna river watershed is the main sub-basin of the Missouri river catchment, located in southwestern Iowa its land cover is classified as agriculture (field). The topography of the Nishnabotna River basin is characterized by broad, rolling uplands and wide valleys. The soil cover generally consists of loess deposits that range in thickness from almost 33 m in places near the western edge of the basin to 1–3 m thick on the ridges in the northeast part of the basin (Wehmeyer et al. 2011). It is bounded on the east by the drainage areas of the Des Moines River, Nodaway River, and Tarkio Creek and on the west by the watersheds of the Boyer River and other smaller tributaries of the Missouri River (Figure 2). The river has coordinates of 95.67°W and 40.51°N and the longest stream length is about 190 km. The total basin area is 7,600 km2, of which 7,360 km2 is placed in portions of 12 Iowa counties. The topography of the Nishnabotna River basin is characterized by broad, rolling uplands and wide valleys. For the study period monthly and daily data were extracted from http://waterwatch.usgs.gov. The statistics of the used daily and monthly data for the study period (1981–2015) have been presented in Table 1. Similar to the first case study, the dataset was split into two different parts: the first 75% of data (1981–2006) were allocated to calibration (training) and the rest (2006–2015) for validation purpose.

Figure 2

Map of West Nishnabotna River watershed.

Figure 2

Map of West Nishnabotna River watershed.

The distance between Trinity River and West Nishnabotna River watersheds is approximately 1,460 km and they have different climatic regimes. As it can be seen in Table 1, the mean daily and monthly rainfall values at the West Nishnabotna River are greater than the average daily and monthly rainfall at the Trinity River watershed. Also the average daily and monthly runoff values at the Trinity River watershed are greater than the mean daily and monthly runoff at the West Nishnabotna River. As previously mentioned, these two watersheds have different geomorphological conditions. Figure 3 shows the land cover of Trinity River and West Nishabotna River watersheds.

Figure 3

Land cover of (a) Trinity River watershed and (b) West Nishnabotna River watershed.

Figure 3

Land cover of (a) Trinity River watershed and (b) West Nishnabotna River watershed.

As it can be seen in Figure 3, unlike the Trinity River watershed which is covered by forest, the land cover of West Nishnabotna River watershed is classified as field and forest (or grassland) accounts for about 10% (Wehmeyer et al. 2011).

Emotional ANN (EANN)

Researchers and scientists studied the role of emotions in artificial intelligence (AI) from a variety of viewpoints: to develop agents and robots that interact more gracefully with humans, to develop systems that use the analog of emotions to aid their own reasoning, or to create agents or robots that more closely model human emotional interactions and learning (Lewin 2001). Although computers do not have physiologies like humans, information signals and regulatory signals travel within them. From a biological point of view, the mood and emotion of an animal due to the activity of hormone glands can affect the neurophysiological responses of the animal, sometimes by providing different actions for a similar task at different moods. Similarly for an EANN, there will be a feedback loop between the hormonal and neural systems; each is influenced by the other and in turn the learning ability of the network is relatively enhanced (Nourani 2017).

Emotional anxiety and confidence factors are considered to modify BP learning algorithm of the multi-layer perceptron (MLP) networks. In the proposed emotional back propagation (EmBP) network for each training sample, anxiety factor is initialized according to the pattern of input sample and then it is modified through the training iteration. In a contrary manner, confidence factor is related to the anxiety factor and the network output at the first iteration. The anxiety and confidence levels are respectively high and low in the start of a training process and they are going to be decreased and increased, respectively, after a few successful training iterations. Within the training procedure of EmBP, assigning a high value to the anxiety factor forces the network to have less attention to the derivative of the errors (error gradient) in the network's output. However an increase in confidence factor (due to stress decrease) dictates the network to pay more heed to the alteration of the weights in the previous training step. This procedure is something like that used to magnify the inertia term to moderate the alteration degree from one pattern to the other as the learning iteration is progressed. From the mathematical point of view and apart from the biological concepts, with regard to a conventional ANN, an EANN includes a few extra parameters which are dynamically interacted with inputs, outputs and statistical weights of the network.

EANN models are new generation of the classic ANN models, containing an artificial emotion unit that can emit artificial hormones to adjust the performance of nodes (neurons) and in a feedback loop, the hormonal weights can be modified according to the input and output values of nodes (Figure 4).

Figure 4

A node of EANN and emotional unit (Nourani 2017).

Figure 4

A node of EANN and emotional unit (Nourani 2017).

As can be seen in Figure 4, in each hidden node of EANN, the information recurrently transforms between input and output units. These nodes also provide dynamic hormones of Ha, Hb and Hc, which in the training phase of model are initially initialized based on the output and input values and then are designed within the learning process. In the training process, the hormonal coefficients impact on other units of the node (Figure 4). In Figure 4, dotted and solid lines indicate the hormonal and neural paths of the information, respectively. In the EANN, the output of ith node with three hormones of Ha, Hb and Hc is calculated as (Nourani 2017; Sharghi et al. 2018): 
formula
(1)
where the artificial hormones are computed as (Nourani 2017; Sharghi et al. 2018): 
formula
(2)
In Equation (1), term 1 shows the imposed weight to the activation function (f). It includes the statistic (constant) neural weight of as well as the dynamic hormonal weight of . Term 2 stands for the imposed weight to the summation (net) function, term 3 indicates imposed weight to the Xi,j (an input from j th node of former layer) and term 4 shows the bias of the summation function, including both neural and hormonal weights of and , respectively.
The sharing of overall hormonal level of EANN (i.e., Hh) among the hormones should be controlled by , , and factors which in turn through the i th node output (Yi) will provide hormonal feedback of Hi,h to the network as (Nourani 2017; Sharghi et al. 2018): 
formula
(3)
where the glandity factor should be calibrated in the training phase of EANN to provide appropriate level of hormone to the glands. Some schemes may be applied to initialize the hormonal values of Hh according to the input samples, e.g., average of input vector of learning samples. Afterwards, considering the output of network (Yi) and Equations (1) and (2), the hormonal values are updated through the learning process to achieve an appropriate match between computed and observed time series of the target.

Wavelet-EANN (WEANN)

Here a brief description of wavelet transform is presented and then the framework of the hybrid WEANN is introduced. A literature review of wavelet applications in earth sciences can be reviewed in Foufoula-Georgiou & Kumar (1994), and the hydrological implementation can be found in Labat et al. (2000).

The time-scale wavelet transform of a continuous-time series, x(t), is defined as (Addison et al. 2001): 
formula
(4)
where n depicts the temporal translation of function that helps study the signal around n; m shows the dilation and the sign * depicts the complex has been used for conjugate and M(t) stands for the mother wavelet (wavelet function). In this transformation, the localization of a time-scale for the process is the main purpose. In practice, hydrologists do not deal with a continuous-time signal process but rather a discrete one. The following format shows a discrete mother wavelet (Addison et al. 2001): 
formula
(5)
In this formula, a and b represent integers so that a controls the wavelet dilation and b shows the wavelet translation. The usual used values for the parameters is m0 = 2 and n0= 1, where n0 indicates the location parameter and it should be greater than zero and m0 displays fined dilation step that is not less than one.
This logarithmic scaling for dilation and translation is called dyadic grid arrangement and therefore, the dyadic wavelet function is expressed as (Addison et al. 2001): 
formula
(6)
For a discrete time series, xi, the dyadic wavelet transform becomes (Sharghi et al. 2018): 
formula
(7)
where is wavelet coefficient for the discrete wavelet with scale m=2a and location n=2ab (i= 0, 1, 2, … , L − 1; and L is an integer power of 2: ).
In addition, the smoothed component of the signal, which represents overall trend of time series is considered as T. The inverse discrete transform can reconstruct the signal xi as (Sharghi et al. 2018): 
formula
(8)
in which is called approximation sub-signal at level A and Ta,b(t) are details sub-signals at levels a= 1, 2, … , A and time dimension of t (t= 1, 2, …, b).

The input layer of the WEANN model includes the wavelet neurons (nodes) fed with the sub-series of the runoff and rainfall time series extracted via the discrete wavelet transform. Figure 5 shows the schematic of the used proposed WEANN model.

Figure 5

Schematic of the WEANN model.

Figure 5

Schematic of the WEANN model.

In the employed WEANN approach at the first step, the available runoff and rainfall time series are decomposed to several sub-series at several time scales (i.e., a large scale sub-series and some small scale sub-series) to extract seasonal characteristics of the time series at different time scales (periods). For an observed time-series Ia(t) and Qa(t) denote to the approximation sub-series (large scale) of the original time series, and ith or jth detailed sub-series (small scale) are indicated by i or j (i.e., Ii(t) or Qj(t)) that i and j show the decomposition levels of the rainfall, I(t), and runoff, Q(t), time series, repsectively.

Considering the multi-frequency efficacies obtained through wavelet decomposition, the proposed hybrid WEANN approach was planned to harness the ability of the EANN and wavelet transform in r-r modeling, simultaneously. The dominant input selection is a significant issue in any AI-based modeling. For this purpose, the feature extraction method of Mutual Information (MI) proposed by Nourani et al. (2015) was useful in selecting effective parameters from potential inputs. MI measures the non-linear relation between two variables (random variables). The statistical definition of MI is the diminution in uncertainty with respect to Y due to observation of X. Equation (9) defines MI between two random variables X and Y (Nourani et al. 2015; Sharghi et al. 2018): 
formula
(9)
where H(X) is entropy of X (also referred to entropy function) and H(X,Y) is joint entropy of variables X and Y.
In the first step for modeling via EANN, the rainfall values at time steps t and t–1 (It and It−1) and antecedent runoff values up to lag time P (Qt, Qt−1, … , Qtp) were considered as potential inputs of the EANN model to forecast the runoff value one-time step-ahead (Qt+1) as the model output. Considering that the prior (antecedent) rainfall efficacies are considered implicitly by prior (antecedent) runoff values, only It and It−1 were considered in the potential input set. The explicit formula of such EANN can be written as: 
formula
(10)
where fn denotes to the network operation.

Efficiency criteria

A model for forecasting hydro-environmental time series must be evaluated in both training and validation phases. For this purpose, different statistics have been developed and used in hydro-environmental issues. Root mean square error (RMSE, Equation (9)) and determination coefficient (DC, Equation (10)) are widely used to assess the efficiency and accuracy of different hydrological models as (Nourani 2017): 
formula
(11)
 
formula
(12)

In Equations (9) and (10), N, Oi, Ci and indicate number of samples, measured values, calculated values and mean of observations, respectively. Legates & McCabe (1999) showed that a hydro-environmental model can be effectively assessed via RMSE and DC. The value of RMSE depends on the units of the predicted variable and for this reason RMSE of models with different units cannot be directly compared with each other, whereas DC is dimensionless and can be used to compare models with different units. However, the RMSE values can be used to distinguish model performance in a calibration period with that of a validation period as well as to compare the individual model performance to that of other predictive models.

It should be noted that DC specifies the relative value of the noise (residual variance) compared with the information (measured data variance). Further, because of critical role of the extreme values in modeling an r-r process, Equation (11) may be employed to evaluate the model performance to take and control the peak values of runoff as (Sharghi et al. 2018): 
formula
(13)
where DCpeak stands for DC of peak values, Np denotes to the number of peak values, Opi, Cpi and indicate observed, calculated and mean of observed peak values, respectively.

RESULTS AND DISCUSSION

The proposed WEANN was applied to simulate the r-r process of two watersheds, West Nishnabotna River, a sub-basin of the Missouri River and Trinity River, a sub-basin in California, United States. As can be seen in Figure 3, unlike Trinity River watershed which is covered by forest, the land cover of the West Nishnabotna River watershed is classified as field and forest (or grassland) only accounts for about 10%.

A simplified frame of the proposed EANN model trained by the BP algorithm (explained in the previous section) was used for r-r modeling of both watersheds and the results are showed in Table 2. The EANN model prepares the possibility of modeling nonlinear autoregressive (Markovian) processes. It should be noted that in an EANN model, the output of the model not only depends on suitable selection of input variables, but also requires accurate adjusting of the network parameters like transfer functions of layers, the number of hidden neurons and training iteration epoch. Determining the optimum architecture of the network, i.e., lag number p (number of input neurons will be p + 1), number of hidden neurons, hormones and best training epoch number have critical roles in EANN training process and their appropriate selection can prevent over-learning of the network. In this study tangent sigmoid and pure line were considered for activation functions of the hidden and output layers, respectively. The optimum structure and iteration epoch number networks were obtained via trial-error process. Different input sets were examined for r-r modeling by EANN. The results of EANN for Trinity River and West Nishnabotna River watersheds at both monthly and daily time scales are presented in Table 2. As can be seen in Table 2, the hormone levels for daily time scale are higher than those for monthly time scale due to the higher stochastic property of the process in the daily time scale. It is notable that the EANN is the ANN model with hormonal parameters and due to its proved superiority to ANN (Nourani 2017), the result of ANN model is not mentioned in this study.

Table 2

Results of WEANN, WANN, and EANN models for both West Nishnabotna River and Trinity River watersheds

Watersheds Time Scale Model Inputs Hormone No. hidden neuron Epoch DC
 
RMSE (Normalized)
 
RMSE (m3/s)
 
DC peak 
Train. Valid. Train. Valid. Train. Valid. Valid. 
West Nishnabotna River Daily WEANN Ia(t),Id1(t),Qa(t),Qd1(t),Qd5(t) 8 7 20 0.808 0.815 0.016 0.016 9.629 10.559 0.791 
EANN Q(t),Q(t 1),Q(t 2),Q(t 3),I(t) 10 30 0.694 0.662 0.022 0.023 13.390 15.852 0.667 
WANN Ia(t),Id1(t),Qa(t),Qd1(t),Qd5(t) 10 40 0.551 0.533 0.029 0.030 21.031 21.347 0.498 
Monthly WEANN Ia(t),Id3(t),Qa(t),Qd4(t) 5 6 10 0.914 0.885 0.011 0.013 82.821 102.842 0.891 
EANN Q(t),Q(t 12),I(t) 20 0.665 0.655 0.025 0.027 135.545 147.771 0.65 
WANN Ia(t),Id3(t),Qa(t),Qd4(t) 10 0.775 0.754 0.017 0.021 94.629 116.890 0.756 
Trinity River Daily WEANN Ia(t),Id4(t),Qa(t),Qd2(t),Qd5(t) 6 5 10 0.954 0.930 0.009 0.010 24.215 27.937 0.912 
EANN Q(t),Q(t 1),Q(t 2),I(t) 10 0.900 0.87 0.010 0.011 28.471 31.305 0.875 
WANN Ia(t),Id4(t),Qa(t),Qd2(t),Qd5(t) 30 0.781 0.772 0.016 0.017 45.681 48.786 0.76 
Monthly WEANN Ia(t),Id4(t),Qa(t),Qd3(t) 5 4 10 0.926 0.892 0.011 0.012 334.911 344.589 0.901 
EANN Q(t),Q(t 12),I(t) 20 0.792 0.774 0.015 0.017 424.395 464.985 0.77 
WANN Ia(t),Id4(t),Qa(t),Qd3(t) 10 0.882 0.876 0.012 0.014 345.251 398.113 0.85 
Watersheds Time Scale Model Inputs Hormone No. hidden neuron Epoch DC
 
RMSE (Normalized)
 
RMSE (m3/s)
 
DC peak 
Train. Valid. Train. Valid. Train. Valid. Valid. 
West Nishnabotna River Daily WEANN Ia(t),Id1(t),Qa(t),Qd1(t),Qd5(t) 8 7 20 0.808 0.815 0.016 0.016 9.629 10.559 0.791 
EANN Q(t),Q(t 1),Q(t 2),Q(t 3),I(t) 10 30 0.694 0.662 0.022 0.023 13.390 15.852 0.667 
WANN Ia(t),Id1(t),Qa(t),Qd1(t),Qd5(t) 10 40 0.551 0.533 0.029 0.030 21.031 21.347 0.498 
Monthly WEANN Ia(t),Id3(t),Qa(t),Qd4(t) 5 6 10 0.914 0.885 0.011 0.013 82.821 102.842 0.891 
EANN Q(t),Q(t 12),I(t) 20 0.665 0.655 0.025 0.027 135.545 147.771 0.65 
WANN Ia(t),Id3(t),Qa(t),Qd4(t) 10 0.775 0.754 0.017 0.021 94.629 116.890 0.756 
Trinity River Daily WEANN Ia(t),Id4(t),Qa(t),Qd2(t),Qd5(t) 6 5 10 0.954 0.930 0.009 0.010 24.215 27.937 0.912 
EANN Q(t),Q(t 1),Q(t 2),I(t) 10 0.900 0.87 0.010 0.011 28.471 31.305 0.875 
WANN Ia(t),Id4(t),Qa(t),Qd2(t),Qd5(t) 30 0.781 0.772 0.016 0.017 45.681 48.786 0.76 
Monthly WEANN Ia(t),Id4(t),Qa(t),Qd3(t) 5 4 10 0.926 0.892 0.011 0.012 334.911 344.589 0.901 
EANN Q(t),Q(t 12),I(t) 20 0.792 0.774 0.015 0.017 424.395 464.985 0.77 
WANN Ia(t),Id4(t),Qa(t),Qd3(t) 10 0.882 0.876 0.012 0.014 345.251 398.113 0.85 

At the second step, to improve the accuracy of the autoregressive AI-based approaches (i.e. EANN model), WEANN and WANN which can handle seasonality of the process via multi-resolution analysis of the wavelet were applied to both daily and monthly datasets. This was achieved by incorporating the wavelet-based decomposed sub-series in the EANN model, where approximation and detailed sub-series were employed to provide trustworthy predictions. The antecedent rainfall and runoff data were decomposed into sub-series component using dyadic discrete wavelet transform. In this way, the dyadic discrete wavelet transform was used to process data at several time scales by separating the small and large properties of the time series. The applied wavelet could decompose the input time series I(t) or Q(t) into one approximation sub-series, Ia(t) or Qa(t), and detailed sub-series, Idl(t), … , Idi(t) or Qdl(t), … , Qdi(t) (i denotes to the decomposition level), therefore each sub-series can represent a special time scale of the seasonality involved in the original time series. Eventually by applying trained weights and bias via the EANN to the input sub-series, output signal, Q(t + 1) can be calculated.

In early studies, the optimum decomposition level was usually determined through a trial-and-error process, but afterwards a formula which relates the minimum level of decomposition, L, to the number of data points within the time series Ns, was introduced in the literature (Wang & Ding 2003): 
formula
(14)

So in this study the initial point view to select of decomposition level was taken from L but since many seasonal characteristics may be embedded in hydrological signals, 2–7 resolution levels for the daily and 2–4 resolution levels for the monthly modeling were examined via the proposed WANN model which denote to 22 – day mode and 23 – day mode (which is nearly weekly mode), 24 – day mode (which is nearly semi-monthly mode), 25 – day mode (which is nearly monthly mode), 26 – day mode and 27 – day mode (which is nearly semi-yearly mode) in daily scale, and 22 –month, 23 – month and 24 – month mode in the monthly scale. Furthermore, one approximation sub-series for each runoff and rainfall time series was imposed to the input layer of the networks. The approximate sub-series shows overall trend of the main and original time series. It should be noted that the wavelet function and decomposition levels should be choose carefully. For this reason, Daubechies wavelet with an order 4 (db4) was selected as the most appropriate function for training the input time-series due to the main signal formation (i.e. rainfall-runoff signals) and previous works.

Among the obtained results, the best model structures (decomposition level five for daily scale that is nearly monthly mode and decomposition level four for monthly scale which is nearly yearly mode) were selected by MI. The overall results of WANN modeling are reported in Table 2. To evaluate the ability of the ANN-based models (WEANN, EANN and WANN) to deal with autoregressive and seasonal features of the time series which exist in an r-r process at both monthly and daily time scales, the modeling performed through two distinct steps via EANN model and wavelet based models of WEANN and WANN, respectively, and the obtained results were compared.

According to the presented results in Table 2, for both daily and monthly modeling the optimum epoch numbers of WEANN model was less than WANN and EANN models which displays their fast training process. Also the hormone numbers of WEANN were less than EANN. Existence of both the external hormonal parameters and wavelet-based data pre-processing unit in the WEANN can cope with the training iterations and there is practically no need for more hormones. Furthermore, it seems the input neuron numbers of all three models in the daily modeling are more than those in the monthly modeling which denotes the need for a long-memory model to deal with the higher stochastic property of the process at the daily time scale.

The computed versus observed runoff values and also the scatter plots of the validation step of the modeling step of WEANN, WANN and EANN models are depicted in Figures 68, respectively, for Trinity River and West Nishnabotna River watersheds in daily time scale. Overall comparison of experimental results (Table 2) shows the merit of WEANN in daily r-r modeling with regard to the EANN and WANN models. Combining autoregressive model of EANN (improved ANN model with extra hormonal parameters) with wavelet transform caused that the WEANN becomes more appropriate for daily runoff modeling. The autoregressive and non-stationary characteristics of the runoff time series is remarkable at the daily time scale and therefore, hybrid WEANN model could take simultaneous advantage of EANN (here autoregressive feature) and wavelet transform that can reliably eliminate AI model defects in dealing with non-stationary behavior of signals. Also, comparison of the obtained results indicated the superiority of EANN over WANN model because WANN is a seasonal model that has multi-frequency inputs with feed forward neural network (FFNN) core (that has a weaker performance versus EANN model) and for this reason it may not handle appropriately the autoregressive property which is more remarkable in the daily time scale.

Figure 6

Observed versus computed runoff at daily time scale for Trinity River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 6

Observed versus computed runoff at daily time scale for Trinity River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 7

Observed versus computed runoff at daily time scale for West Nishnabotna River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 7

Observed versus computed runoff at daily time scale for West Nishnabotna River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 8

Verification scatter plot of computed versus observed runoff at daily time scale for WEANN model: (a) Trinity River watershed and (b) West Nishnabotna River watershed.

Figure 8

Verification scatter plot of computed versus observed runoff at daily time scale for WEANN model: (a) Trinity River watershed and (b) West Nishnabotna River watershed.

According to Table 2, the proposed WEANN model with regard to the EANN and WANN models could lead to more accurate results of up to 23% and 52% for West Nishnabotna River and 6% and 20% for Trinity River watersheds, respectively, in daily time scale (validation phase). Two catchments with two different geomorphological conditions show almost distinct responses to the rainfall, but the Trinity River watershed has a wild geomorphological condition with respect to the West Nishnabotna watershed. Therefore for Trinity River watershed, the performance of models is lower as compared with the West Nishnabotna watershed (see Table 2). However this difference between modeling performances of two watersheds using WEANN is lower than in the other models.

Accurate prediction of peak value is an important task for hydrological processes. So calculated DCpeak values (see Table 2) and depicted scatter plots in Figure 8 show the capability of WEANN to compute the peak values of hydrographs, the second best performance is EANN while the least is WANN for both catchments. Since Markovian models, due to the autoregressive feature of the process, take account of the states of process at former time steps to forecast the process state one time-step ahead, they often underrate peak discharges that can happen as a result of the instantaneous imposing the external forces (severe storm) on the system. In such situation, the system is experiencing an emotional condition that is apart from the normal conditions of the system. Thus, in the training step of the proposed EANN, a hormonal parameter from the emotional unit acts as a dynamic coefficient to recurrently give feedback to the other components of the EANN and adjust the network for the emotional condition (i.e., severe rainfall). From the mathematical point of view, these dynamic coefficients are activated in abnormal conditions (e.g., severe rainfall) and impact and magnify the weights of EANN. Combining this feature with wavelet transform meant that the performance of WEANN in prediction of peak values was better than EANN (20%) and WANN (30%).

EANN serves only a few hormonal weights which can lead to good representation of abnormal and extreme conditions, but when it is linkage to the wavelet transform leads to the most effective model in this study.

In the monthly modeling, the computed versus observed runoff time series and also the scatter plots of validation results of modeling by WEANN, WANN and EANN models are depicted in Figures 911 for Trinity River and West Nishnabotna River watersheds, respectively. Computed values of DCpeak (see Table 2) and the presented scatter plots in Figure 11 indicate the ability of WEANN to catch the peak values of hydrographs better than other models in both watersheds. Since autoregressive models, according to the Markovian property of a process, consider states of the process at some previous time steps to predict the state of the system at the next time step, they usually underestimate the peak values which occur due to instantaneous imposition of an external force (intensive rainfall) to the system. In this condition, the system is experiencing an emotional situation which is different from normal conditions of the system. Therefore in the training phase, a hormone of the emotional unit of WEANN acts as a dynamic weight to recurrently give feedback to the other components of the network and regulate the model for the emotional situation.

Figure 9

Observed versus computed runoff at monthly time scale for Trinity River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 9

Observed versus computed runoff at monthly time scale for Trinity River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 10

Observed versus computed runoff at monthly time scale for West Nishnabotna River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 10

Observed versus computed runoff at monthly time scale for West Nishnabotna River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 11

Verification scatter plot of computed versus observed runoff at monthly time scale for WEANN model: (a) Trinity River watershed and (b) West Nishnabotna River watershed.

Figure 11

Verification scatter plot of computed versus observed runoff at monthly time scale for WEANN model: (a) Trinity River watershed and (b) West Nishnabotna River watershed.

In monthly timescale and according to the results presented in Table 2, the WEANN shows superior performance over WANN and EANN models of up to 17%, 35% for the West Nishnabotna River and 2%, 15% for the Trinity River watersheds (at validation phase), respectively. Also WEANN could catch the peak values of hydrographs better than others in both watersheds (see Figure 11). In monthly modeling, the seasonality of time series is more dominant and therefore, the wavelet-based models (WEANN and WANN) with the ability of handling seasonal features of the process were more efficient than the pure autoregressive model of EANN (Table 2). The reason behind this superiority may be related to the capability of wavelet for multi-resolution analysis of time series, so that the wavelet transform decomposes the original time series into multi-scale sub-series, each representing a separate seasonal scale and therefore, the multi-seasonality properties of the time series can be considered in the modeling. In other words, data pre-processing by the wavelet transform could enhance the performance of the modeling at different time scales, but this progress is more sensible in the large-scale time series (e.g., monthly) since the seasonal patterns included in the large-scale time series are more dominant with regard to the small-scale time series (e.g., daily). However, comparison of the results shows the superiority of the WEANN over the WANN model because the WEANN model, including an EANN, core uses a few more parameters (hormonal weights) which can lead to better outcomes.

With regard to the Trinity River watershed which is covered dominantly by forests, the response of the watershed to the storm is not as non-linear as West Nishnabotna River watershed. Therefore, in both time scales, all models (WANN, WEANN, EANN) could almost lead to acceptable results with only a few input hidden neurons and epoch number. But the efficiency improvement for West Nishnabotna River watershed models is more than the Trinity River watershed (at different time scales) due to its geomorphological conditions which lead to more non-linear behavior.

CONCLUSIONS

Each hydrological time series (such as rainfall and runoff time series) usually includes three principal components of autoregressive, seasonality and trend. Therefore, overall performance of the model is related to its ability to handle these components. An investigation was performed in this study to examine the ability of EANN (as a new generation of AI-based models inspired by neurophysiological form of brain), WEANN (as a novel hybrid AI-based model), WANN (as a hybrid model) for one-time-ahead daily and monthly modeling of an r-r process of two watersheds with distinct geomorphological and land cover conditions.

The comparison of the obtained results showed that for daily modeling, WEANN outperforms the other models (EANN and WANN) especially for the West Nishnabotna River watershed (the main sub-basins of the Missouri River catchment) because of its special geomorphological condition which was discussed above. Also, the obtained results for monthly modeling showed that WEANN could outperform the WANN and EANN models by up to 17% and 35%, respectively, in terms of efficiency criteria at validation step. In other words, the result indicated that for monthly modeling, WEANN outperforms the other models (WANN and EANN) due to significant seasonality patterns involved in the monthly time series of the process and EANN has a better performance over ANN because of implementation of a few hormonal weights.

Such a reliable performance of the WEANN in r-r modeling offers its application to model other hydrological processes (e.g., sediment load, groundwater, precipitation, etc.) especially in small-scale time series (e.g. daily). To complete this study, it is recommended to use the proposed method to predict the runoff two, three and more time-steps ahead. Moreover, it can be informative to exert the presented methodology on other basins for investigating of the overall impact of basin climatic conditions on the performance of the proposed model. The employed model in this study was a typical form of WEANN among broad classes of WEANNs trained by BP algorithm, future studies may focus on evaluating other types of WEANNs and other training algorithms (e.g., metaheuristic approaches) in hydrologic modeling.

REFERENCES

REFERENCES
Addison
P. S.
,
Murrary
K. B.
&
Watson
J. N.
2001
Wavelet transform analysis of open channel wake flows
.
Journal of Engineering Mechanics
127
,
58
70
.
ASCE Task Committee on Application of Artificial Neural Networks in Hydrology
2000
Artificial neural networks in hydrology. II: Hydrologic applications
.
Journal of Hydrologic Engineering
5
,
124
137
.
Dawson
C.
&
Wilby
R.
2001
Hydrological modelling using artificial neural networks
.
Progress in Physical Geography
25
,
80
108
.
Foufoula-Georgiou
E.
&
Kumar
P.
1994
Wavelet analysis in geophysics: an introduction
.
Wavelet Analysis and Its Applications
4
,
1
43
.
Hsu
K. L.
,
Gupta
H. V.
&
Sorooshian
S.
1995
Artificial neural network modeling of the rainfall-runoff process
.
Water Resources Research
31
,
2517
2530
.
Kumar
M.
&
Sahay
R. R.
2018
Wavelet-genetic programming conjunction model for flood forecasting in rivers
.
Hydrology Research
.
https://doi.org/10.2166/nh.2018.183
.
Lewin
D. I.
2001
Why is that computer laughing?
IEEE Intelligent Systems
16
,
79
81
.
Lotfi
E.
&
Akbarzadeh
T. M. R.
2014
Practical emotional neural networks
.
Neural Networks
59
,
61
72
.
Nayak
P. C.
,
Sudheer
K.
,
Rangan
D.
&
Ramasastri
K.
2004
A neuro-fuzzy computing technique for modeling hydrological time series
.
Journal of Hydrology
291
,
52
66
.
Nourani
V.
,
Baghanam
A. H.
,
Adamowski
J.
&
Kisi
O.
2014
Applications of hybrid wavelet – artificial intelligence models in hydrology: a review
.
Journal of Hydrology
514
,
358
377
.
Nourani
V.
,
Khangah
T. R.
&
Baghanam
A. H.
2015
Application of entropy concept for input selection of Wavelet-ANN based rainfall-runoff modeling
.
Journal of Environmental Informatics
26
,
52
70
.
Nourani
V.
,
Sattari
M. T.
&
Molajou
A.
2017
Threshold-based hybrid data mining method for long-term maximum precipitation forecasting
.
Water Resources Management
31
,
2645
2658
.
Salas
J. D.
,
Delleur
J. W.
,
Yevjevich
V.
&
Lane
W. L.
1980
Applied Modeling of Hydrological Time Series
,
1st edn
.
Water Resources Publications
,
Littleton
.
Sharghi
E.
,
Nourani
V.
,
Najafi
H.
&
Molajou
A.
2018
Emotional ANN (EANN) and wavelet-ANN (WANN) approaches for Markovian and seasonal based modeling of rainfall-runoff process
.
Water Resources Management
doi.org/10.1007/s11269-018-2000-y
.
Solomatine
D.
&
Ostfeld
A.
2008
Data-driven modelling: some past experiences and new approaches
.
Journal of Hydroinformatics
10
,
3
22
.
Wang
W.
&
Ding
J.
2003
Wavelet network model and its application to the prediction of hydrology
.
Nature and Science
1
,
67
71
.
Zhang
Q.
,
Wang
B. D.
,
He
B.
,
Peng
Y.
&
Ren
M. L.
2011
Singular spectrum analysis and ARIMA hybrid model for annual runoff forecasting
.
Water Resources Management
25
,
2683
2703
.