Prediction of daily streamflow using artificial neural networks (ANNs), wavelet neural networks (WNNs), and adaptive neuro-fuzzy inference system (ANFIS) models

In recent years, streamflow prediction has been considered to be one of the most important issues in the fields of hydrology, water resources, and water resources management. An accurate streamflow estimation can play a positive role in enhancing the capacity of reservoirs, flood prevention, water supply, design of hydroelectric projects, and water resources management. It can therefore have a significant impact on reducing the effects of climatic events on the environment and improve the efficiency of the outcomes.

In the past few years, data-driven models, including artificial neural networks (ANNs), wavelet neural networks (WNNs), and adaptive neuro-fuzzy inference systems (ANFIS) models, have been applied as effective tools for modeling nonlinear and complex hydrological systems (Wu et al. 2009; Seo et al. 2013a,, 2013b; Seo & Kim 2016). Despite the wide usage of the ANN method to predict hydrological variables, it may not give accurate and reliable results in the estimation of unstable data. Therefore, to estimate a hydrologic time series comprising nonlinear relationships, the use of data pre-processing techniques is needed to improve the performance of ANNs (Okkan 2013). One of these methods is the wavelet transform, which is a signal processing technique. In contrast to Fourier transform, it is possible to calculate both the low- and high-frequency components of the signal at each time interval with wavelet transform. Many studies have achieved successful predictions with the WNN model, which is formed by combining wavelet analysis and ANN usage. Coulibaly & Burn (2004), Partal (2007) and Kisi (2009) used the streamflow values of the preceding days as an input parameter to estimate the streamflow of the Aegean River and to develop the WNN model. They stated that WNN models have lower error values than ANN models.

ANFIS was first proposed by Jang (1993). It is also a powerful data-driven technique that can be used to model hydrologic processes such as rainfall-runoff forecasting and flood risk management. The basic construction of ANFIS is IF-THEN rules with appropriate membership functions to be able to generate input-output pairs. To model the flow of the Baitarani River in India, Nayak et al. (2004) made a prediction with ANFIS using the hydrological time series. In the estimation made with ANFIS, it was observed that the statistical characteristics of the original flow series were preserved and a successful estimation emerged.

These significant models used together in this study are better than a single model, as the models' forecasted results can be used to compare their forecasting ability and therefore recognize the powers and precision of each model for a given dataset. The multi artificial intelligence models for hydrological modeling that show the differences and compare the models' performances were applied in this study. To have a better conception of the estimated data through the models mentioned, some highly recommended and effective model evaluation indexes were applied in this study. Using these evaluation indexes, the root mean square error (RMSE), determination coefficient (R²), Nash–Sutcliffe (NS), and RMSE-observation standard deviation ratio (RSR), the results can be readily interpreted and used as a reliable outcome for future studies or in hydrological processes.

This study aimed to determine the daily streamflow values of four different stations located on the Büyük Menderes River. This study developed and applied three different models to asses the efficiency and accuracy of each model in relation to the original (measured) data. Furthermore, the results of each model were comparable and did not have a significant difference. However, in terms of performance, WNN demonstrated the best performance compared to the ANFIS and ANN models.

The Büyük Menderes River (historically the Maeander or Meander, from Ancient Greek) is the largest river in Western Anatolia and the main irrigation source in the Menderes Basin: 79% of the water in the basin is used in the agriculture sector and in agricultural irrigation (Büyük Menderes Havza Atlası 2012). According to DSİ (1975) and EİE (1993) records, the length of the Büyük Menderes River is 584 km and has an area of 24,300 km². The river rises in Dinar, Afyonkarahisar province, (38° 4′ 15″ N, 30° 10′ 37″ E), at an elevation of 880 m.

Various inputs (mean temperature, precipitation, relative humidity, and the previous 3, 2, and 1 days' flow data) from the Muğla meteorological station were used to run the models. The streamflow monitoring stations E07A004, E07A0032, E07A0033, and E07A0037, which belong to the Electrical Power Resources Survey and Development Administration, are located near the Büyük Menderes River (Figure 1) and were chosen as they had not been much used in previous studies. This significant area has not been researched sufficiently to date, so the aim of this paper is to improve the productivity of the Büyük Menderes River as much as possible. The dataset for this study covers the period 1996 to 2011, as it is the most recent dataset available.

ANNs are a new information processing approach and they are one of the most popular subjects of contemporary science with their learning ability, adaptation, rapid operation, and ease of identification. ANNs are completely parallel distributed data processing systems that consist of a wide number of highly interconnected elements, called artificial neurons or nodes, which resemble the biological neural networks of the human brain (Tsoukala & Uhrig 1996). Neurons work on the principle of generating results by learning knowledge and increasing experience and information. Every neuron receives an input signal from another neuron that sends it through an activation or transfer function and utilizes an altered output to the other outputs (Figure 2). Even though each neuron implements its function rather slowly and imperfectly, a network can perform a wide number of tasks quite efficiently if they combine to work collectively. ANNs, which are nonlinear black-box models, are accepted as being amongst the most popular methods, especially in modeling, because of their precise and reliable results for the prediction of hydrological variables. Recently, ANNs have not been used regularly, as they do not present hydrological modeling results that are as successful and accurate as WNNs and ANFIS models. There are a few ANN models that have been proposed since the 1980s, and probably the most influential of them are the multi-layer perceptron (MLP), the Hopfield network, and Kohonen's self-organizing networks.

The FFNN is the most commonly used neural network in water resources. It is generally used for forecasting hydrological variables and generates a general substructure for showing nonlinear operational surveying between a collection of input and output variables. Typically, it has two stages, forward and backward calculating and three layers of neural networks: input, output, and hidden layer. Each layer consists of many neurons that are connected by weight clusters, and their connection forms and number of neurons can change in each layer. The relationship between forward computing and backward computing is acquired by the training procedures.

Here, J, I, e, are, respectively, the Jacobian matrix, identity matrix, network error, and Marquardt parameter that are to be updated using the decay rate relay on the result. Further, is multiplied by the decay rate (⁠⁠) whenever E(W) decreases, and E(W) increases in a new step (Coulibaly et al. 2000).

In this study, the Levenberg-Marquardt backpropagation algorithm was used in the training of the FFNN. The Levenberg-Marquardt backpropagation algorithm is usually quicker and more reliable than any other backpropagation technique, and it is a second-order nonlinear optimization technique (Coulibaly et al. 2000; Kisi 2004; Fistikoglu & Okkan 2011). The Levenberg-Marquardt optimization algorithm shows a shortened version of the Newton method applied to the training phase of the FFNN.

As shown in Figure 2, the procedure of error-propagation contains two moves in different layers of the network. In the frontward movement, the input vector of the algorithm is applied to the neurons of the network, and in the backward movement, all of the weights are modified in line with the error-correction rule. Then the error signal spreads backward via the network and the weights are modified to make the actual response of the network closer to the desired response. K: epoch refers to the number of rounds of optimization applied during training. With more rounds of optimization, the error in the training data will reduce further and further until there is a constant number of errors. W: weight is the parameter within a neural network that transforms input data within the network's hidden layers. Bias is simply a constant value added to the product of inputs and weights. Bias is utilized to offset the result. The activation function is a mathematical function that can normalize the inputs of a network and the Error (E) is equal to the sum of the squared difference between observed data and the model's result as is shown in Figure 2.

This study trained a neural network using 15 years (1996–2011) of daily flow data from four stations in the Büyük Menderes Basin to estimate the near future's daily streamflow. To increase the precision of the estimation process, 70% of the data was used for training and 30% of the data was used to test the model. The model's input parameters were precipitation, previous flow, average temperature, and relative humidity, which always show continuous variation. Parameters that change constantly are difficult to define in a mathematical model. Therefore, here, the ANN method was used for the solution of nonlinear problems. The ANN model was designed as an FFNN and it consisted of different inputs (Qt-3, Qt-2, Qt-1, Pt, Tavg(t), Rh(t), and one output (Qt). Here, t is the time (daily); Qt-3, Qt-2, Qt-1 are defined as the flow rate in the previous 3, 2, and 1 days; Pt is the precipitation; and Tavg(t) and Rh(t) are the average temperature and relative humidity, respectively. The designed structure of the ANN model consisted of three layers and 10 neurons in its hidden layers. In the trial and error studies of the ANN model, the most effective and minimum error rate of results were obtained with 10 neurons and three layers using the MATLAB (v.R2018a) program. The network is based on the backpropagation model, correcting the error in the network by spreading it backward. In this model, the Levenberg-Marquardt optimization was used as the training algorithm. Using these methods together, the weight values of ANN were calculated in epochs.

The training process is the re-determination of the weights to minimize the error between the results obtained from the output and the expected values in each iteration of the epochs. Here, the mean square error (MSE) function was used as the error reduction function. The model's inputs that involved Qt-2, Qt-1, and Qt-1inputs, developed to estimate the daily streamflow, demonstrated the best performance as shown below. Here, the ANN (1,10,1) model structure shows that there is one cell in the input layer, 10 in the hidden layer, and one in the output layer. When the models developed are examined, the first two statistical parameters in Table 1 give a close correlation value. The third statistical parameter has the biggest error value among the other parameters used in Station E07A0037.

The WNN model is formed by the application of discrete wavelet transform (DWT), which is an implementation of the wavelet transform using a discrete set of the wavelet scales and translations acting by some predetermined rules. In other words, the significant features of many natural signals are captured by a subset of DWT coefficients that is normally much smaller than the original signal, and the ANN method, and it is a neural network prediction algorithm that stands based on wavelet theory. Among the functions of the WNN, one of the most significant is the estimation function. Neurons are replaced in the hidden layer of the neural network with the scaling function to have more space and therefore have a stronger function approximation ability (Khodadadi & Razavi 2012). A wavelet network is trained to learn the composition of an observed data function and to calculate an expected value for a given value. It has also the ability of self-learning, adaptivity, and calculating the nonlinear parameters of a system (Hou et al. 2013). Compared to the traditional backpropagation neural network, the wavelet network can compensate for the deficiencies of the incomplete information extraction of the conventional neural network, due to the time-frequency localization capability of the wavelet function (Heinermann & Kramer 2016). To set up the WNN model, the original time series is subdivided using a DWT, then each sub-series obtained behaves differently from the original series. Then, the correlation coefficients are calculated between the obtained sub-series and the model output. The WNN inputs are determined by appropriate sub-series according to the correlation coefficient values. Common wavelets used in the wavelet analysis are Haar Shannon, Mexican Hat, Coiflet, Daubechies 4, Daubechies 20, Gaussian and Biorthogonal wavelets and are shown in Figure 3.

Firstly, daily meteorological data (average temperature, relative humidity, and total precipitation) were divided into five details (2-4-8-16-32 daily periodic components), and one approximation component using DWT. With the help of the algorithm given by Mallat (1989), the original data (signal) are divided into several series, called approximation and detail components (AD), and the original data are recovered again by a collection of these series in a row. The first level of detail component is named AD1 and refers to the 2-day scale component. This gives us the highest frequency variations. AD5 is the lowest frequency detail component and has a 32-day scale. It is necessary to determine which AD components have to be selected to obtain the desired successful results in streamflow prediction with the ANN. For this purpose, the correlations between the AD components and the original streamflow were calculated. Table 2 shows the results obtained for Station E07A0037.

To develop the streamflow estimation models, three different input combinations that are dependent on the previous periods of the streamflow were evaluated. These input combinations were as follows: (1) Qt-3, Qt-2, Qt-1; (2) Qt-2, Qt-1; (3) Qt-1; (4) Qt-1, Pt; and (5) Qt-1, Pt, Tavg(t), Rh(t). To assess the performance of the ANN and WNN models, the MSE, the mean absolute error (MAE), the correlation coefficient (R), and the R² values were used between the model results and measured data. Here, the Levenberg-Marquardt method was chosen as the learning algorithm. The WNN model was developed by sub-series as an input parameter, which was obtained with the Haar wavelet from the ANN model, as in the literature reviewed. Adamowski & Chan (2011), Kisi & Partal (2011), and Tiwari & Chatterjee (2011) collected the sub-series obtained from the DWT in their studies and used them as inputs for the ANN model. The statistical parameters of the training and test sets of the WNN models that were developed are shown here just for one station, E07A0037, in Table 3.

The network is trained based on learning with supervision. So, the target is to teach ANFIS to estimate uncertain functions derived from instructional information and accurately estimate the unknown parameters (Figure 3). To design an ANFIS model, it was not easy to select a better fuzzy inference system for a specific goal. Different types of FIS have been stated in the literature (Takagi & Sugeno 1985). The adaptive neuro-fuzzy system is usually used with the Sugeno fuzzy system as a progressive network structure, as presented in Figure 4.

Here, for generating an FIS, various membership function types were evaluated, and the most effective membership function type (trimf) for input data and output data (linear) shows the best results for all four different stations. To train the FIS, the backpropagation method was used to cope with the parameter recognition problem in an FIS, which is the basic rule of an adaptive network, and based on a descent rule that can successfully estimate the parameters. The outcomes for the ANFIS model gained are shown in Table 4. On the other hand, the characteristics of the model's performance criteria, that the outputs of the models examined by the evaluators, fortunately remained between those ranges that are shown in Table 5.

In this study, to estimate daily streamflow values, three models, ANN, WNN, and ANFIS, were used in four different stations located near the Büyük Menderes River. RMSE, R², NS, and RSR were used as the evaluation indicators of the models' outcomes to represent the satisfaction or dissatisfaction ratio of the models. Therefore, the best consequence from different parameters of the models' tests for the ANN are listed in Table 6, for the WNN in Table 7, and for the ANFIS in Table 8. The input data were classified into five parameters and each parameter had one or more than one combination of the input data, which were considered to be the same for the four stations used in this study. Despite presenting the most important details about each model's structures and development processes for estimating daily streamflow, it is better to sum over each model's performance and quality individually. The most important step in modeling is choosing the right combination of input variables. Thus, the correlation between input variables and output was calculated, and the input variables were selected to model the system to estimate the daily streamflow of the Büyük Menderes River, as shown in Table 6.

In this table, the parameters of the flow of the previous 1, 2, and 3 days, precipitation, average temperature, and relative humidity were the inputs in a period of (t) and flow as an output in a period of (t) were described. According to the meaningful correlation between the inputs and output, which are shown above, a few different combinations of input parameters were used to estimate the daily streamflow values efficiently that are shown in Table 7.

As is shown in Table 8, the best result was for the ANN, as demonstrated by the minimum error rate, RMSE = 4.466, and by the best correlation rate, R² = 0.932, which includes three input layers, 10 hidden layers, and one output layer. Additionally, as can be seen from Table 9, the WNN model also performed well, with the minimum error rate, RMSE = 0.700, and the best correlation rate, R² = 0.971, which refers to the most precise correlation value. Furthermore, as shown in Table 10, the ANFIS model result is also considered to be good, with the minimum error rate of RMSE = 5.068, and the best correlation rate of R² = 0.947. Figure 5 shows the time series and plot graphs of the models. The correlation and proximity of all the models' outcomes (output) with the observed (original) output of the data once again can be observed in Figure 6. The study aimed to find the minimum error rate, and conversely the highest correlation rate, for the observed output through modeling the data.

The main purpose of the study was to achieve the minimum error rate, and conversely the highest correlation coefficient between the calculated data and the observed data using ANN, WNN and ANFIS models. As shown, the outcomes of the models were meaningful and successful by the end of the calculations. Simultaneously, assembling the most recommended models that are commonly cited in the literature was the other reason to understand and show the performances of the models through sequencing their outcomes by their precision rates.

The data were divided into 70% for training, 30% for testing the network, and the number of hidden neurons were selected by the trail-test method, which is considered to be 10 neurons in this study. The training algorithm selected for this model was the Levenberg-Marquardt because it is quicker than other algorithms, and it is the preferred algorithm for estimating a large number of daily intervals of data. To increase the performance and accuracy of the ANN model, the original data were decomposed into sub-series, their noises removed and the most efficient data were selected by calculating the correlation coefficients for each of the given parameters. ANFIS is a combination model of FIS and ANN. Each one has its positive aspects, and when they are joined together, the ANFIS model performs better. As a result, in this study, the WNN showed the most accurate result, with R² = 0.971, compared to the two other models. The ANFIS model, with R² = 0.947, and the ANN model, with R² = 0.932, were second and third in this study. Briefly, the main reason that WNN gave the most accurate result was that decomposing the original data into sub-series and removing their noises, and again using them as clean data, significantly affected the outcomes.