Conjunction of emotional ANN (EANN) and wavelet transform for rainfall-runoff modeling

The current research introduces a combined wavelet-emotional artificial neural network (WEANN) approach for one-time-ahead rainfall-runoff modeling of two watersheds with different geomorphological and land cover conditions at daily and monthly time scales, to utilize within a unique framework the ability of both wavelet transform (to mitigate the effects of non-stationary) and emotional artificial neural network (EANN, to identify and individualize wet and dry conditions by hormonal components of the artificial emotional system). To assess the efficiency of the proposed hybrid model, the model efficiency was also compared with so-called EANN models (as a new generation of ANN-based models) and wavelet-ANN (WANN) models (as a multi-resolution forecasting tool). The obtained results indicated that for daily scale modeling, WEANN outperforms the other models (EANN and WANN). Also, the obtained results for monthly modeling showed that WEANN could outperform the WANN and EANN models up to 17% and 35% in terms of validation and training efficiency criteria, respectively. Also, the obtained results highlighted the capability of the proposed WEANN approach to better learning of extraordinary and extreme conditions of the process in the training phase. doi: 10.2166/hydro.2018.054 om https://iwaponline.com/jh/article-pdf/21/1/136/517578/jh0210136.pdf er 2019 Elnaz Sharghi (corresponding author) Vahid Nourani Hessam Najafi Deptartment of Water Resources Engineering, Faculty of Civil Engineering, University of Tabriz, P.O. Box: 51666, Tabriz, Iran E-mail: sharghi@tabrizu.ac.ir Vahid Nourani Faculty of Civil Engineering, Near East University, P.O. Box 99138, Nicosia, North Cyprus, Mersin 10, Turkey Amir Molajou Deptartment of Water Resources Engineering, Faculty of Civil Engineering, Iran University of Science & Technology, Tehran, Iran


INTRODUCTION
Modeling of rainfall-runoff (r-r) processes performed by hydrologists can be helpful in obtaining information for environmental planning, flooding, and water resources management.A reliable r-r model can also alleviate drought influence on water resources, an important issue in arid and semi-arid areas.For this purpose, black-box modeling can be considered as an alternative with regard to fully distributed models because of the complex unknown and abstruse factors involved in the process (Salas et  The hydrological processes (e.g., r-r process) are considered to be complex systems due to the effects and interaction of the different spatio-temporal factors involved (Nayak et al. ).Recent studies have shown that classic statistical models such as auto-regressive integrated moving average (ARIMA) may not be capable of simulating the r-r process adequately due to the non-linear characteristics of the process.Nowadays, the artificial neural network (ANN), known as a self-learning and self-adaptive function approximator, has been widely used for modeling non-linear hydrological time series due to benefiting from the black box feature (no requirement of prior knowledge), applying a non-linear function to handle the non-linear properties of the process and the ability of analyzing multivariate inputs with different characteristics.This has led to a huge leap in the use of ANN models in hydrological simulations (e.g., Hsu et

Study area and dataset
In this paper, data for two watersheds with different geomorphological conditions were used for the modeling purpose,   observed at the catchment outlet station and a gauging station located upstream of the outlet (see Figure 1) were retrieved via http://waterwatch.usgs.govand used in this study.The statistics of the used daily and monthly data for the study period  are presented in Table 1.The dataset was split into two different parts: the first 75% of data     As can be seen in Figure 4, in each hidden node of EANN, the information recurrently transforms between input and output units.These nodes also provide dynamic where the artificial hormones are computed as (Nourani In Equation ( 1 where the glandity factor should be calibrated in the training phase of EANN to provide appropriate level of hormone to the glands.Some schemes may be applied to initialize the hormonal values of H h according to the input samples, e.g., average of input vector of learning samples.Afterwards, considering the output of network (Y i ) and Equations ( 1) and ( 2), the hormonal values are updated through the learning process to achieve an appropriate match between computed and observed time series of the target.

Wavelet-EANN (WEANN)
Here a brief description of wavelet transform is presented and then the framework of the hybrid WEANN is intro- where n depicts the temporal translation of function that helps study the signal around n; m shows the dilation and the sign * depicts the complex has been used for conjugate and M(t) stands for the mother wavelet (wavelet function).
In this transformation, the localization of a time-scale for the process is the main purpose.In practice, hydrologists For a discrete time series, x i , the dyadic wavelet transform becomes (Sharghi et al. ): where T a,b is wavelet coefficient for the discrete wavelet and L is an integer power of 2: L ¼ 2 A ).
In addition, the smoothed component of the signal, which represents overall trend of time series is considered as T. The inverse discrete transform can reconstruct the signal x i as (Sharghi et al. ): where H(X) is entropy of X (also referred to entropy function) and H(X,Y) is joint entropy of variables X and Y.
In the first step for modeling via EANN, the rainfall values at time steps t and t-1 (I t and I tÀ1 ) and antecedent runoff values up to lag time where f n denotes to the network operation.

Efficiency criteria
A model for forecasting hydro-environmental time series must be evaluated in both training and validation phases.
For this purpose, different statistics have been developed and used in hydro-environmental issues.Root mean square error (RMSE, Equation ( 9)) and determination coefficient (DC, Equation ( 10)) are widely used to assess the efficiency and accuracy of different hydrological models as (Nourani ): In Equations ( 9) and ( 10  the extreme values in modeling an r-r process, Equation ( 11) may be employed to evaluate the model performance to take and control the peak values of runoff as (Sharghi et al. ):

RESULTS AND DISCUSSION
The proposed WEANN was applied to simulate the r-r pro- can be calculated.
In early studies, the optimum decomposition level was usually determined through a trial-and-error process, but afterwards a formula which relates the minimum level of decomposition, L, to the number of data points within the time series Ns, was introduced in the literature (Wang & Ding ): So in this study the initial point view to select of decomposition level was taken from L but since many seasonal characteristics may be embedded in hydrological signals, 2-7 resolution levels (L ± x) for the daily and 2-4 resolution levels (L ± x) for the monthly modeling were examined via the proposed WANN model which denote to 2 2day mode and 2 3day mode (which is nearly weekly mode), 2 4day mode (which is nearly semi-monthly mode), 2 5day mode (which is nearly monthly mode), 2 6day mode and 2 7day mode (which is nearly semi-yearly mode) in daily scale, and 2 2 -month, 2 3month and 2 4month mode in the monthly scale.Furthermore, one approximation sub-series for each runoff and rainfall time series was imposed to the input layer of the networks.watershed (see Table 2).However this difference between modeling performances of two watersheds using WEANN is lower than in the other models.
Accurate prediction of peak value is an important task for hydrological processes.So calculated DC peak values (see Table 2) and depicted scatter plots in Figure 8 show  In monthly timescale and according to the results presented in Table 2, the WEANN shows superior performance over WANN and EANN models of up to 17%, 35% for the West Nishnabotna River and 2%, 15% for the Trinity River watersheds (at validation phase), respectively.Also WEANN could catch the peak values of hydrographs better than others in both watersheds (see Figure 11).In monthly modeling, the seasonality of time series is more dominant and therefore, the wavelet-based models (WEANN and WANN) with the ability of handling seasonal features of the process were more efficient than the pure autoregressive model of EANN (Table 2).The reason behind this superiority may be related to the capa- al. ; Solomatine & Ostfeld ; Adamowski et al. a, b; Danandeh Mehr et al. ; Farajzadeh & Alizadeh ).
al. ; ASCE ; Maier & Dandy ; Dawson & Wilby ; Jain & Srinivasulu ; Zhang et al. ; Adamowski et al. a, b; Feng et al. ; Nourani et al. ).In spite of the flexible nature of ANN in modeling hydrological processes, this algorithm may exhibit defects in dealing with complex hydrological signals.Therefore, in such a situation, spatial or temporal pre-processing of data can be a necessary step to overcome such problems.The ability of wavelet transform in decomposing complex hydrological time series to sub-series by extracting useful information at different scales can be effective for interpreting hydrological phenomena (Sang ; Kumar & Sahay ).The hybrid wavelet-ANN model (WANN) is a well designed method that utilizes the wavelet transform to obtain the different frequencies of r-r process and forecast the future runoff at desired scale by ANN (Kuo et al. ; Shiri & Kisi ; Kisi & Cimen ; Adamowski et al. a, b; Sang ; Nourani et al. ).The benefits of the hybrid Wavelet-Artificial Intelligence (AI) models (such as the ability of wavelet transform in multi-resolution analysis, de-noising and edge effect detection of a signal and the strong capability of AI methods in optimization and prediction of the processes) have been highlighted in a review paper by Nourani et al. ().Recently a new generation of ANN models has been proposed by incorporating the artificial emotions into the classic ANN framework as emotional ANN (EANN) models (Khashman ; Lotfi & Akbarzadeh , ).

From a biological
point of view, the neurophysiological response of animals can be affected by hormonal activities, so that the animals may provide different actions for the same task at different moods.Inspired by this biological concept and merging artificial emotion and ANN, the learning ability of the network could be enhanced due to the feedback loop between systems of hormones and neurons.As the first hydrological implementation of EANN, Nourani () proposed revised Back Propagation (BP) training algorithm of a Multi-Layer Perceptron (MLP) network by incorporating emotional anxiety concept and investigated the efficiency of the EANN model to cope with the shortage of long observed training time series.In general, the advantages of EANN models in comparison with the other statistical and black box methods can be categorized as: the ability of EANN to cope with the lack of long observed data used for network training; the EANN model could lead to more accurate estimations of peak values; and emotional parameters of an EANN dynamically get/give information from/to inputs and outputs of the network at each time step to distinguish the extreme events (e.g.dry and wet days).In spite of the flexible nature of EANN in coping with the lack of long observed data used for network training, clearly just like any other data-driven time series forecasting method, the performance of the EANN can be affected by the presence of anthropogenic and/or climatic influences and shifts of the observed time series.In the presence of such a strong non-stationary time series, reliable data preprocessing approaches may be employed prior to performing the forecasts.In this way, as a novel strategy, wavelet-based data-processing approach with the ability of multi-resolution analysis was linked to the EANN to enhance the modeling efficiency.In this study, the wavelet-EANN (WEANN) approach for r-r modelling (due to the ability of wavelet transform to mitigate the effects of non-stationary time series) was combined with the ability of EANN to identify and individualize wet (rainy days) and dry (rainless days) conditions by hormonal parameters of the artificial emotional system.The data for two watersheds with different geomorphological conditions were used to indicate the performance of the proposed WEANN approach for r-r modeling.For this purpose, at the first step the data were decomposed into sub-signals using wavelet transform.After that, the obtained sub-signals (as inputs) were applied into the EANN model to reconstruct the main and original time series.Finally, to evaluate the model capability, the results of the proposed WEANN model were also compared with the results of EANN and WANN models.

West
Nishnabotna River (sub-basin of Missouri River) and Trinity River (sub-basin in California, United States).Two catchments with two different geomorphological conditions show almost distinct responses to the rainfall.The Trinity River is a major branch of the Klamath River which flows through the Coast Ranges and Klamath Mountains (in northwestern California) (Figure 1).The river is at 123.42 W and 41.11 N and the longest stream length of the river is about 266 km with watershed area nearly 7,800 km 2 .The catchment elevation varies between 58 m (where the Trinity is surrounded by the Klamath River) and 2,709 m (Sawtooth Peak in the Trinity Alps).About 92% of the watershed is covered by oak, fir, and pine forests and the region on every side of the river has a population with 1.7 people per km 2 and for this reason the basin is not much affected by humans.The overall climate regime of the basin is Mediterranean and it has hot and dry summers, but cool and wet winters.The coldest and hottest months are December and July with average temperatures of 4.6 C and 28.2 C, respectively, and the mean temperature is 15.5 C; the driest and wettest months of year of the watershed are August and March, respectively.Thirty-five years of daily flow and monthly rainfall data Figure 1 | Map of Trinity River watershed.
Figure 3 | Land cover of (a) Trinity River watershed and (b) West Nishnabotna River watershed.
hormones of H a , H b and H c , which in the training phase of model are initially initialized based on the output and input values and then are designed within the learning process.In the training process, the hormonal coefficients impact on other units of the node (Figure 4).In Figure 4, dotted and solid lines indicate the hormonal and neural paths of the information, respectively.In the EANN, the output of ith node with three hormones of H a , H b and H c is calculated as (Nourani ; Sharghi et al. ): ), term 1 shows the imposed weight to the activation function ( f ).It includes the statistic (constant) neural weight of γ i as well as the dynamic hormonal weight of P h @ i,h H h .Term 2 stands for the imposed weight to the summation (net) function, term 3 indicates imposed weight to the X i,j (an input from j th node of former layer) and term 4 shows the bias of the summation function, including both neural and hormonal weights of μ i and P h ψ i,h H h , respectively.
duced.A literature review of wavelet applications in earth sciences can be reviewed in Foufoula-Georgiou & Kumar (), and the hydrological implementation can be found in Labat et al. ().The time-scale wavelet transform of a continuous-time series, x(t), is defined as (Addison et al. ): do not deal with a continuous-time signal process but rather a discrete one.The following format shows a discrete mother wavelet (Addison et al. ):M a,b (t) , a and b represent integers so that a controls the wavelet dilation and b shows the wavelet translation.The usual used values for the parameters is m 0 ¼ 2 and n 0 ¼ 1, where n 0 indicates the location parameter and it should be greater than zero and m 0 displays fined dilation step that is not less than one.This logarithmic scaling for dilation and translation is called dyadic grid arrangement and therefore, the dyadic wavelet function is expressed as (Addison et al. ): ) in which T(t) is called approximation sub-signal at level A and T a,b (t) are details sub-signals at levels a ¼ 1, 2, … , A and time dimension of t (t ¼ 1, 2, …, b).The input layer of the WEANN model includes the wavelet neurons (nodes) fed with the sub-series of the runoff and rainfall time series extracted via the discrete wavelet transform.Figure 5 shows the schematic of the used proposed WEANN model.In the employed WEANN approach at the first step, the available runoff and rainfall time series are decomposed to several sub-series at several time scales (i.e., a large scale sub-series and some small scale sub-series) to extract seasonal characteristics of the time series at different time scales (periods).For an observed timeseries I a (t) and Q a (t) denote to the approximation sub-series (large scale) of the original time series, and ith or jth detailed sub-series (small scale) are indicated by i or j (i.e., I i (t) or Q j (t)) that i and j show the decomposition levels of the rainfall, I(t), and runoff, Q(t), time series, repsectively.Considering the multi-frequency efficacies obtained through wavelet decomposition, the proposed hybrid WEANN approach was planned to harness the ability of the EANN and wavelet transform in r-r modeling, simultaneously.The dominant input selection is a significant issue in any AI-based modeling.For this purpose, the feature extraction method of Mutual Information (MI) proposed by Nourani et al. () was useful in selecting effective parameters from potential inputs.MI measures the non-linear relation between two variables (random variables).The statistical definition of MI is the diminution in uncertainty with respect to Y due to observation of X. Equation (9) defines MI between two random variables X and Y (Nourani et al. ; Sharghi et al. ): were considered as potential inputs of the EANN model to forecast the runoff value one-time step-ahead (Q tþ1 ) as the model output.Considering that the prior (antecedent) rainfall efficacies are considered implicitly by prior (antecedent) runoff values, only I t and I tÀ1 were considered in the potential input set.The explicit formula of such EANN can be written as:
13) where DC peak stands for DC of peak values, N p denotes to the number of peak values, O pi , C pi and O indicate observed, calculated and mean of observed peak values, respectively.
cess of two watersheds, West Nishnabotna River, a subbasin of the Missouri River and Trinity River, a sub-basin in California, United States.As can be seen in Figure 3, unlike Trinity River watershed which is covered by forest, the land cover of the West Nishnabotna River watershed is classified as field and forest (or grassland) only accounts for about 10%.A simplified frame of the proposed EANN model trained by the BP algorithm (explained in the previous section) was used for r-r modeling of both watersheds and the results are showed in Table 2.The EANN model prepares the possibility of modeling nonlinear autoregressive (Markovian) processes.It should be noted that in an EANN model, the output of the model not only depends on suitable selection of input variables, but also requires accurate adjusting of the network parameters like transfer functions of layers, the number of hidden neurons and training iteration epoch.Determining the optimum architecture of the network, i.e., lag number p (number of input neurons will be p þ 1), number of hidden neurons, hormones and best training epoch number have critical roles in EANN training process and their appropriate selection can prevent overlearning of the network.In this study tangent sigmoid and pure line were considered for activation functions of the hidden and output layers, respectively.The optimum structure and iteration epoch number networks were obtained via trial-error process.Different input sets were examined for r-r modeling by EANN.The results of EANN for Trinity

Figure 6 |Figure 7 |
Figure 6 | Observed versus computed runoff at daily time scale for Trinity River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 8 |
Figure 8 | Verification scatter plot of computed versus observed runoff at daily time scale for WEANN model: (a) Trinity River watershed and (b) West Nishnabotna River watershed.
Figure 9 | Observed versus computed runoff at monthly time scale for Trinity River watershed: (a) whole time series, (b) and (c) show selected details.

Figure 11 |
Figure 11 | Verification scatter plot of computed versus observed runoff at monthly time scale for WEANN model: (a) Trinity River watershed and (b) West Nishnabotna River watershed.
bility of wavelet for multi-resolution analysis of time series, so that the wavelet transform decomposes the original time series into multi-scale sub-series, each representing a separate seasonal scale and therefore, the multi-seasonality properties of the time series can be considered in the modeling.In other words, data pre-processing by the wavelet transform could enhance the performance of the modeling at different time scales, but this progress is more sensible in the large-scale time series (e.g., monthly) since the seasonal patterns included in the large-scale time series are more dominant with regard to the small-scale time series (e.g., daily).However, comparison of the results shows the superiority of the WEANN over the WANN model because the WEANN model, including an EANN, core uses a few more parameters (hormonal weights) which can lead to better outcomes.With regard to the Trinity River watershed which is covered dominantly by forests, the response of the watershed to the storm is not as non-linear as West Nishnabotna River watershed.Therefore, in both time scales, all models (WANN, WEANN, EANN) could almost lead to acceptable results with only a few input hidden neurons and epoch number.But the efficiency improvement for West Nishnabotna River watershed models is more than the Trinity River watershed (at different time scales) due to its geomorphological conditions which lead to more non-linear behavior.CONCLUSIONS Each hydrological time series (such as rainfall and runoff time series) usually includes three principal components of autoregressive, seasonality and trend.Therefore, overall performance of the model is related to its ability to handle these components.An investigation was performed in this study to examine the ability of EANN (as a new generation of AI-based models inspired by neurophysiological form of brain), WEANN (as a novel hybrid AI-based model), WANN (as a hybrid model) for onetime-ahead daily and monthly modeling of an r-r process of two watersheds with distinct geomorphological and land cover conditions.The comparison of the obtained results showed that for daily modeling, WEANN outperforms the other models (EANN and WANN) especially for the West Nishnabotna River watershed (the main sub-basins of the Missouri River catchment) because of its special geomorphological condition which was discussed above.Also, the obtained results for monthly modeling showed that WEANN could outperform the WANN and EANN models by up to 17% and 35%, respectively, in terms of efficiency criteria at validation step.In other words, the result indicated that for monthly modeling, WEANN outperforms the other models (WANN and EANN) due to significant seasonality patterns involved in the monthly time series of the process and EANN has a better performance over ANN because of implementation of a few hormonal weights.

Table 1 |
The statistics of the observed daily and monthly time series for Trinity River and West Nishnabotna River watersheds

Table 1
, the mean daily and monthly rainfall values at the West Nishnabotna River are greater than the average daily and monthly rainfall at the Trinity River watershed.Also The sharing of overall hormonal level of EANN (i.e., H h ) among the hormones should be controlled by @ i,h , χ i,h , Φ i,j,k and ψ i,h factors which in turn through the i th node output (Y i ) will provide hormonal feedback of H i,h to the network as (Nourani ; Sharghi et al. ):

Table 2 |
Results of WEANN, WANN, and EANN models for both West Nishnabotna River and Trinity River watersheds sub-series in the EANN model, where approximation and detailed sub-series were employed to provide trustworthy predictions.The antecedent rainfall and runoff data were decomposed into sub-series component using dyadic discrete wavelet transform.In this way, the dyadic discrete wavelet transform was used to process data at several time scales by separating the small and large properties of the time series.The applied wavelet could decompose the input time series I(t) or Q(t) into one approximation subseries, I a (t) or Q a (t), and detailed sub-series, I dl (t), … , I di(t)or Q dl (t), … , Q di (t) (i denotes to the decomposition level), therefore each sub-series can represent a special time scale of the seasonality involved in the original time series.Eventually by applying trained weights and bias via the EANN to the input sub-series, output signal,