Conjunction of emotional ANN (EANN) and wavelet transform for rainfall-runoff modeling

Modeling of rainfall-runoff (r-r) processes performed by hydrologists can be helpful in obtaining information for environmental planning, flooding, and water resources management. A reliable r-r model can also alleviate drought influence on water resources, an important issue in arid and semi-arid areas. For this purpose, black-box modeling can be considered as an alternative with regard to fully distributed models because of the complex unknown and abstruse factors involved in the process (Salas et al. 1980; Solomatine & Ostfeld 2008; Adamowski et al. 2012a, 2012b; Danandeh Mehr et al. 2017; Farajzadeh & Alizadeh 2018).

The hydrological processes (e.g., r-r process) are considered to be complex systems due to the effects and interaction of the different spatio-temporal factors involved (Nayak et al. 2004). Recent studies have shown that classic statistical models such as auto-regressive integrated moving average (ARIMA) may not be capable of simulating the r-r process adequately due to the non-linear characteristics of the process. Nowadays, the artificial neural network (ANN), known as a self-learning and self-adaptive function approximator, has been widely used for modeling non-linear hydrological time series due to benefiting from the black box feature (no requirement of prior knowledge), applying a non-linear function to handle the non-linear properties of the process and the ability of analyzing multivariate inputs with different characteristics. This has led to a huge leap in the use of ANN models in hydrological simulations (e.g., Hsu et al. 1995; ASCE 2000; Maier & Dandy 2000; Dawson & Wilby 2001; Jain & Srinivasulu 2006; Zhang et al. 2011; Adamowski et al. 2012a, 2012b; Feng et al. 2017; Nourani et al. 2017).

In spite of the flexible nature of ANN in modeling hydrological processes, this algorithm may exhibit defects in dealing with complex hydrological signals. Therefore, in such a situation, spatial or temporal pre-processing of data can be a necessary step to overcome such problems. The ability of wavelet transform in decomposing complex hydrological time series to sub-series by extracting useful information at different scales can be effective for interpreting hydrological phenomena (Sang 2013; Kumar & Sahay 2018). The hybrid wavelet-ANN model (WANN) is a well designed method that utilizes the wavelet transform to obtain the different frequencies of r-r process and forecast the future runoff at desired scale by ANN (Kuo et al. 2010; Shiri & Kisi 2010; Kisi & Cimen 2011; Adamowski et al. 2012a, 2012b; Sang 2013; Nourani et al. 2015). The benefits of the hybrid Wavelet-Artificial Intelligence (AI) models (such as the ability of wavelet transform in multi-resolution analysis, de-noising and edge effect detection of a signal and the strong capability of AI methods in optimization and prediction of the processes) have been highlighted in a review paper by Nourani et al. (2014).

Recently a new generation of ANN models has been proposed by incorporating the artificial emotions into the classic ANN framework as emotional ANN (EANN) models (Khashman 2008; Lotfi & Akbarzadeh 2014, 2016). From a biological point of view, the neurophysiological response of animals can be affected by hormonal activities, so that the animals may provide different actions for the same task at different moods. Inspired by this biological concept and merging artificial emotion and ANN, the learning ability of the network could be enhanced due to the feedback loop between systems of hormones and neurons. As the first hydrological implementation of EANN, Nourani (2017) proposed revised Back Propagation (BP) training algorithm of a Multi-Layer Perceptron (MLP) network by incorporating emotional anxiety concept and investigated the efficiency of the EANN model to cope with the shortage of long observed training time series. In general, the advantages of EANN models in comparison with the other statistical and black box methods can be categorized as: the ability of EANN to cope with the lack of long observed data used for network training; the EANN model could lead to more accurate estimations of peak values; and emotional parameters of an EANN dynamically get/give information from/to inputs and outputs of the network at each time step to distinguish the extreme events (e.g. dry and wet days).

In spite of the flexible nature of EANN in coping with the lack of long observed data used for network training, clearly just like any other data-driven time series forecasting method, the performance of the EANN can be affected by the presence of anthropogenic and/or climatic influences and shifts of the observed time series. In the presence of such a strong non-stationary time series, reliable data preprocessing approaches may be employed prior to performing the forecasts. In this way, as a novel strategy, wavelet-based data-processing approach with the ability of multi-resolution analysis was linked to the EANN to enhance the modeling efficiency. In this study, the wavelet-EANN (WEANN) approach for r-r modelling (due to the ability of wavelet transform to mitigate the effects of non-stationary time series) was combined with the ability of EANN to identify and individualize wet (rainy days) and dry (rainless days) conditions by hormonal parameters of the artificial emotional system. The data for two watersheds with different geomorphological conditions were used to indicate the performance of the proposed WEANN approach for r-r modeling. For this purpose, at the first step the data were decomposed into sub-signals using wavelet transform. After that, the obtained sub-signals (as inputs) were applied into the EANN model to reconstruct the main and original time series. Finally, to evaluate the model capability, the results of the proposed WEANN model were also compared with the results of EANN and WANN models.

In this paper, data for two watersheds with different geomorphological conditions were used for the modeling purpose, West Nishnabotna River (sub-basin of Missouri River) and Trinity River (sub-basin in California, United States). Two catchments with two different geomorphological conditions show almost distinct responses to the rainfall.

The Trinity River is a major branch of the Klamath River which flows through the Coast Ranges and Klamath Mountains (in northwestern California) (Figure 1). The river is at 123.42°W and 41.11°N and the longest stream length of the river is about 266 km with watershed area nearly 7,800 km². The catchment elevation varies between 58 m (where the Trinity is surrounded by the Klamath River) and 2,709 m (Sawtooth Peak in the Trinity Alps). About 92% of the watershed is covered by oak, fir, and pine forests and the region on every side of the river has a population with 1.7 people per km² and for this reason the basin is not much affected by humans. The overall climate regime of the basin is Mediterranean and it has hot and dry summers, but cool and wet winters. The coldest and hottest months are December and July with average temperatures of 4.6 °C and 28.2 °C, respectively, and the mean temperature is 15.5 °C; the driest and wettest months of year of the watershed are August and March, respectively. Thirty-five years of daily flow and monthly rainfall data (1981–2015) observed at the catchment outlet station and a gauging station located upstream of the outlet (see Figure 1) were retrieved via http://waterwatch.usgs.gov and used in this study. The statistics of the used daily and monthly data for the study period (1981–2015) are presented in Table 1. The dataset was split into two different parts: the first 75% of data (1981–2006) were allocated to calibration (training) and the rest (2006–2015) were used as validation (testing) set.

The second study area, the West Nishnabotna river watershed is the main sub-basin of the Missouri river catchment, located in southwestern Iowa its land cover is classified as agriculture (field). The topography of the Nishnabotna River basin is characterized by broad, rolling uplands and wide valleys. The soil cover generally consists of loess deposits that range in thickness from almost 33 m in places near the western edge of the basin to 1–3 m thick on the ridges in the northeast part of the basin (Wehmeyer et al. 2011). It is bounded on the east by the drainage areas of the Des Moines River, Nodaway River, and Tarkio Creek and on the west by the watersheds of the Boyer River and other smaller tributaries of the Missouri River (Figure 2). The river has coordinates of 95.67°W and 40.51°N and the longest stream length is about 190 km. The total basin area is 7,600 km², of which 7,360 km² is placed in portions of 12 Iowa counties. The topography of the Nishnabotna River basin is characterized by broad, rolling uplands and wide valleys. For the study period monthly and daily data were extracted from http://waterwatch.usgs.gov. The statistics of the used daily and monthly data for the study period (1981–2015) have been presented in Table 1. Similar to the first case study, the dataset was split into two different parts: the first 75% of data (1981–2006) were allocated to calibration (training) and the rest (2006–2015) for validation purpose.

The distance between Trinity River and West Nishnabotna River watersheds is approximately 1,460 km and they have different climatic regimes. As it can be seen in Table 1, the mean daily and monthly rainfall values at the West Nishnabotna River are greater than the average daily and monthly rainfall at the Trinity River watershed. Also the average daily and monthly runoff values at the Trinity River watershed are greater than the mean daily and monthly runoff at the West Nishnabotna River. As previously mentioned, these two watersheds have different geomorphological conditions. Figure 3 shows the land cover of Trinity River and West Nishabotna River watersheds.

As it can be seen in Figure 3, unlike the Trinity River watershed which is covered by forest, the land cover of West Nishnabotna River watershed is classified as field and forest (or grassland) accounts for about 10% (Wehmeyer et al. 2011).

Researchers and scientists studied the role of emotions in artificial intelligence (AI) from a variety of viewpoints: to develop agents and robots that interact more gracefully with humans, to develop systems that use the analog of emotions to aid their own reasoning, or to create agents or robots that more closely model human emotional interactions and learning (Lewin 2001). Although computers do not have physiologies like humans, information signals and regulatory signals travel within them. From a biological point of view, the mood and emotion of an animal due to the activity of hormone glands can affect the neurophysiological responses of the animal, sometimes by providing different actions for a similar task at different moods. Similarly for an EANN, there will be a feedback loop between the hormonal and neural systems; each is influenced by the other and in turn the learning ability of the network is relatively enhanced (Nourani 2017).

Emotional anxiety and confidence factors are considered to modify BP learning algorithm of the multi-layer perceptron (MLP) networks. In the proposed emotional back propagation (EmBP) network for each training sample, anxiety factor is initialized according to the pattern of input sample and then it is modified through the training iteration. In a contrary manner, confidence factor is related to the anxiety factor and the network output at the first iteration. The anxiety and confidence levels are respectively high and low in the start of a training process and they are going to be decreased and increased, respectively, after a few successful training iterations. Within the training procedure of EmBP, assigning a high value to the anxiety factor forces the network to have less attention to the derivative of the errors (error gradient) in the network's output. However an increase in confidence factor (due to stress decrease) dictates the network to pay more heed to the alteration of the weights in the previous training step. This procedure is something like that used to magnify the inertia term to moderate the alteration degree from one pattern to the other as the learning iteration is progressed. From the mathematical point of view and apart from the biological concepts, with regard to a conventional ANN, an EANN includes a few extra parameters which are dynamically interacted with inputs, outputs and statistical weights of the network.

EANN models are new generation of the classic ANN models, containing an artificial emotion unit that can emit artificial hormones to adjust the performance of nodes (neurons) and in a feedback loop, the hormonal weights can be modified according to the input and output values of nodes (Figure 4).

Here a brief description of wavelet transform is presented and then the framework of the hybrid WEANN is introduced. A literature review of wavelet applications in earth sciences can be reviewed in Foufoula-Georgiou & Kumar (1994), and the hydrological implementation can be found in Labat et al. (2000).

The input layer of the WEANN model includes the wavelet neurons (nodes) fed with the sub-series of the runoff and rainfall time series extracted via the discrete wavelet transform. Figure 5 shows the schematic of the used proposed WEANN model.

In the employed WEANN approach at the first step, the available runoff and rainfall time series are decomposed to several sub-series at several time scales (i.e., a large scale sub-series and some small scale sub-series) to extract seasonal characteristics of the time series at different time scales (periods). For an observed time-series I_a(t) and Q_a(t) denote to the approximation sub-series (large scale) of the original time series, and ith or jth detailed sub-series (small scale) are indicated by i or j (i.e., I_i(t) or Q_j(t)) that i and j show the decomposition levels of the rainfall, I(t), and runoff, Q(t), time series, repsectively.

In Equations (9) and (10), N, O_i, C_i and indicate number of samples, measured values, calculated values and mean of observations, respectively. Legates & McCabe (1999) showed that a hydro-environmental model can be effectively assessed via RMSE and DC. The value of RMSE depends on the units of the predicted variable and for this reason RMSE of models with different units cannot be directly compared with each other, whereas DC is dimensionless and can be used to compare models with different units. However, the RMSE values can be used to distinguish model performance in a calibration period with that of a validation period as well as to compare the individual model performance to that of other predictive models.

The proposed WEANN was applied to simulate the r-r process of two watersheds, West Nishnabotna River, a sub-basin of the Missouri River and Trinity River, a sub-basin in California, United States. As can be seen in Figure 3, unlike Trinity River watershed which is covered by forest, the land cover of the West Nishnabotna River watershed is classified as field and forest (or grassland) only accounts for about 10%.

A simplified frame of the proposed EANN model trained by the BP algorithm (explained in the previous section) was used for r-r modeling of both watersheds and the results are showed in Table 2. The EANN model prepares the possibility of modeling nonlinear autoregressive (Markovian) processes. It should be noted that in an EANN model, the output of the model not only depends on suitable selection of input variables, but also requires accurate adjusting of the network parameters like transfer functions of layers, the number of hidden neurons and training iteration epoch. Determining the optimum architecture of the network, i.e., lag number p (number of input neurons will be p + 1), number of hidden neurons, hormones and best training epoch number have critical roles in EANN training process and their appropriate selection can prevent over-learning of the network. In this study tangent sigmoid and pure line were considered for activation functions of the hidden and output layers, respectively. The optimum structure and iteration epoch number networks were obtained via trial-error process. Different input sets were examined for r-r modeling by EANN. The results of EANN for Trinity River and West Nishnabotna River watersheds at both monthly and daily time scales are presented in Table 2. As can be seen in Table 2, the hormone levels for daily time scale are higher than those for monthly time scale due to the higher stochastic property of the process in the daily time scale. It is notable that the EANN is the ANN model with hormonal parameters and due to its proved superiority to ANN (Nourani 2017), the result of ANN model is not mentioned in this study.

At the second step, to improve the accuracy of the autoregressive AI-based approaches (i.e. EANN model), WEANN and WANN which can handle seasonality of the process via multi-resolution analysis of the wavelet were applied to both daily and monthly datasets. This was achieved by incorporating the wavelet-based decomposed sub-series in the EANN model, where approximation and detailed sub-series were employed to provide trustworthy predictions. The antecedent rainfall and runoff data were decomposed into sub-series component using dyadic discrete wavelet transform. In this way, the dyadic discrete wavelet transform was used to process data at several time scales by separating the small and large properties of the time series. The applied wavelet could decompose the input time series I(t) or Q(t) into one approximation sub-series, I_a(t) or Q_a(t), and detailed sub-series, I_dl(t), … , I_di(t) or Q_dl(t), … , Q_di(t) (i denotes to the decomposition level), therefore each sub-series can represent a special time scale of the seasonality involved in the original time series. Eventually by applying trained weights and bias via the EANN to the input sub-series, output signal, Q(t + 1) can be calculated.

So in this study the initial point view to select of decomposition level was taken from L but since many seasonal characteristics may be embedded in hydrological signals, 2–7 resolution levels for the daily and 2–4 resolution levels for the monthly modeling were examined via the proposed WANN model which denote to 2² – day mode and 2³ – day mode (which is nearly weekly mode), 2⁴ – day mode (which is nearly semi-monthly mode), 2⁵ – day mode (which is nearly monthly mode), 2⁶ – day mode and 2⁷ – day mode (which is nearly semi-yearly mode) in daily scale, and 2² –month, 2³ – month and 2⁴ – month mode in the monthly scale. Furthermore, one approximation sub-series for each runoff and rainfall time series was imposed to the input layer of the networks. The approximate sub-series shows overall trend of the main and original time series. It should be noted that the wavelet function and decomposition levels should be choose carefully. For this reason, Daubechies wavelet with an order 4 (db4) was selected as the most appropriate function for training the input time-series due to the main signal formation (i.e. rainfall-runoff signals) and previous works.

Among the obtained results, the best model structures (decomposition level five for daily scale that is nearly monthly mode and decomposition level four for monthly scale which is nearly yearly mode) were selected by MI. The overall results of WANN modeling are reported in Table 2. To evaluate the ability of the ANN-based models (WEANN, EANN and WANN) to deal with autoregressive and seasonal features of the time series which exist in an r-r process at both monthly and daily time scales, the modeling performed through two distinct steps via EANN model and wavelet based models of WEANN and WANN, respectively, and the obtained results were compared.

According to the presented results in Table 2, for both daily and monthly modeling the optimum epoch numbers of WEANN model was less than WANN and EANN models which displays their fast training process. Also the hormone numbers of WEANN were less than EANN. Existence of both the external hormonal parameters and wavelet-based data pre-processing unit in the WEANN can cope with the training iterations and there is practically no need for more hormones. Furthermore, it seems the input neuron numbers of all three models in the daily modeling are more than those in the monthly modeling which denotes the need for a long-memory model to deal with the higher stochastic property of the process at the daily time scale.

The computed versus observed runoff values and also the scatter plots of the validation step of the modeling step of WEANN, WANN and EANN models are depicted in Figures 6–8, respectively, for Trinity River and West Nishnabotna River watersheds in daily time scale. Overall comparison of experimental results (Table 2) shows the merit of WEANN in daily r-r modeling with regard to the EANN and WANN models. Combining autoregressive model of EANN (improved ANN model with extra hormonal parameters) with wavelet transform caused that the WEANN becomes more appropriate for daily runoff modeling. The autoregressive and non-stationary characteristics of the runoff time series is remarkable at the daily time scale and therefore, hybrid WEANN model could take simultaneous advantage of EANN (here autoregressive feature) and wavelet transform that can reliably eliminate AI model defects in dealing with non-stationary behavior of signals. Also, comparison of the obtained results indicated the superiority of EANN over WANN model because WANN is a seasonal model that has multi-frequency inputs with feed forward neural network (FFNN) core (that has a weaker performance versus EANN model) and for this reason it may not handle appropriately the autoregressive property which is more remarkable in the daily time scale.

According to Table 2, the proposed WEANN model with regard to the EANN and WANN models could lead to more accurate results of up to 23% and 52% for West Nishnabotna River and 6% and 20% for Trinity River watersheds, respectively, in daily time scale (validation phase). Two catchments with two different geomorphological conditions show almost distinct responses to the rainfall, but the Trinity River watershed has a wild geomorphological condition with respect to the West Nishnabotna watershed. Therefore for Trinity River watershed, the performance of models is lower as compared with the West Nishnabotna watershed (see Table 2). However this difference between modeling performances of two watersheds using WEANN is lower than in the other models.

Accurate prediction of peak value is an important task for hydrological processes. So calculated DC_peak values (see Table 2) and depicted scatter plots in Figure 8 show the capability of WEANN to compute the peak values of hydrographs, the second best performance is EANN while the least is WANN for both catchments. Since Markovian models, due to the autoregressive feature of the process, take account of the states of process at former time steps to forecast the process state one time-step ahead, they often underrate peak discharges that can happen as a result of the instantaneous imposing the external forces (severe storm) on the system. In such situation, the system is experiencing an emotional condition that is apart from the normal conditions of the system. Thus, in the training step of the proposed EANN, a hormonal parameter from the emotional unit acts as a dynamic coefficient to recurrently give feedback to the other components of the EANN and adjust the network for the emotional condition (i.e., severe rainfall). From the mathematical point of view, these dynamic coefficients are activated in abnormal conditions (e.g., severe rainfall) and impact and magnify the weights of EANN. Combining this feature with wavelet transform meant that the performance of WEANN in prediction of peak values was better than EANN (20%) and WANN (30%).

EANN serves only a few hormonal weights which can lead to good representation of abnormal and extreme conditions, but when it is linkage to the wavelet transform leads to the most effective model in this study.

In the monthly modeling, the computed versus observed runoff time series and also the scatter plots of validation results of modeling by WEANN, WANN and EANN models are depicted in Figures 9–11 for Trinity River and West Nishnabotna River watersheds, respectively. Computed values of DC_peak (see Table 2) and the presented scatter plots in Figure 11 indicate the ability of WEANN to catch the peak values of hydrographs better than other models in both watersheds. Since autoregressive models, according to the Markovian property of a process, consider states of the process at some previous time steps to predict the state of the system at the next time step, they usually underestimate the peak values which occur due to instantaneous imposition of an external force (intensive rainfall) to the system. In this condition, the system is experiencing an emotional situation which is different from normal conditions of the system. Therefore in the training phase, a hormone of the emotional unit of WEANN acts as a dynamic weight to recurrently give feedback to the other components of the network and regulate the model for the emotional situation.

In monthly timescale and according to the results presented in Table 2, the WEANN shows superior performance over WANN and EANN models of up to 17%, 35% for the West Nishnabotna River and 2%, 15% for the Trinity River watersheds (at validation phase), respectively. Also WEANN could catch the peak values of hydrographs better than others in both watersheds (see Figure 11). In monthly modeling, the seasonality of time series is more dominant and therefore, the wavelet-based models (WEANN and WANN) with the ability of handling seasonal features of the process were more efficient than the pure autoregressive model of EANN (Table 2). The reason behind this superiority may be related to the capability of wavelet for multi-resolution analysis of time series, so that the wavelet transform decomposes the original time series into multi-scale sub-series, each representing a separate seasonal scale and therefore, the multi-seasonality properties of the time series can be considered in the modeling. In other words, data pre-processing by the wavelet transform could enhance the performance of the modeling at different time scales, but this progress is more sensible in the large-scale time series (e.g., monthly) since the seasonal patterns included in the large-scale time series are more dominant with regard to the small-scale time series (e.g., daily). However, comparison of the results shows the superiority of the WEANN over the WANN model because the WEANN model, including an EANN, core uses a few more parameters (hormonal weights) which can lead to better outcomes.

With regard to the Trinity River watershed which is covered dominantly by forests, the response of the watershed to the storm is not as non-linear as West Nishnabotna River watershed. Therefore, in both time scales, all models (WANN, WEANN, EANN) could almost lead to acceptable results with only a few input hidden neurons and epoch number. But the efficiency improvement for West Nishnabotna River watershed models is more than the Trinity River watershed (at different time scales) due to its geomorphological conditions which lead to more non-linear behavior.

Each hydrological time series (such as rainfall and runoff time series) usually includes three principal components of autoregressive, seasonality and trend. Therefore, overall performance of the model is related to its ability to handle these components. An investigation was performed in this study to examine the ability of EANN (as a new generation of AI-based models inspired by neurophysiological form of brain), WEANN (as a novel hybrid AI-based model), WANN (as a hybrid model) for one-time-ahead daily and monthly modeling of an r-r process of two watersheds with distinct geomorphological and land cover conditions.

The comparison of the obtained results showed that for daily modeling, WEANN outperforms the other models (EANN and WANN) especially for the West Nishnabotna River watershed (the main sub-basins of the Missouri River catchment) because of its special geomorphological condition which was discussed above. Also, the obtained results for monthly modeling showed that WEANN could outperform the WANN and EANN models by up to 17% and 35%, respectively, in terms of efficiency criteria at validation step. In other words, the result indicated that for monthly modeling, WEANN outperforms the other models (WANN and EANN) due to significant seasonality patterns involved in the monthly time series of the process and EANN has a better performance over ANN because of implementation of a few hormonal weights.

Such a reliable performance of the WEANN in r-r modeling offers its application to model other hydrological processes (e.g., sediment load, groundwater, precipitation, etc.) especially in small-scale time series (e.g. daily). To complete this study, it is recommended to use the proposed method to predict the runoff two, three and more time-steps ahead. Moreover, it can be informative to exert the presented methodology on other basins for investigating of the overall impact of basin climatic conditions on the performance of the proposed model. The employed model in this study was a typical form of WEANN among broad classes of WEANNs trained by BP algorithm, future studies may focus on evaluating other types of WEANNs and other training algorithms (e.g., metaheuristic approaches) in hydrologic modeling.