Deep and machine learning for daily streamflow estimation: a focus on LSTM, RFR and XGBoost

Flow estimation in the ungauged parts of the rivers is one of the frequently encountered problems for hydrologists. Making reliable flow estimations plays an important role in the planning of water resources. The reliable predictions can minimize the possible errors occurring in the studies like flood control and planning of hydraulic structures. However, it is not always possible to reach the required flow data at the desired places due to some reasons. Therefore, it has been necessary to develop various flow estimation methods. These methods can generally be classified as deterministic basin models (Box & Jenkins 1970; Salas 1980; Bender & Simonovic 1994; Modarres 2007; Valipour et al. 2013). However, these techniques are time-consuming and require too many parameters. Apart from these, there are artificial intelligence methods such as artificial neural networks (ANNs), which are often used in recent years and easier to use in flow prediction studies because they do not require too many parameters. In water resources studies, hydrologists frequently use ANNs. Indeed, ANNs have been used in forecasting rainfall (Zhang et al. 1997; Chiang et al. 2007), sediment (Partal & Cigizoglu 2008; Jothiprakash & Garg 2009), floods (Campolo et al. 2003; Chang et al. 2007; Aziz et al. 2015), evaporation (Moghaddamnia et al. 2009) and flow (Panagoulia 2006; Pramanik & Panda 2009; Kostić et al. 2016; Veintimilla-Reyesa et al. 2016; Zemzami & Benaabidate 2016). Uysal et al. (2016) developed snowmelt models with ANNs (multi-layer perceptron (MLP) and radial basis function) for mountainous region in Turkey. They showed that both models give similar results. Yin et al. (2016) compared the rotated general regression neural network, the general regression neural network, the feed-forward error back-propagation model and the soil moisture accounting and routing model to forecast monthly river flow in Heihe River, China. They stated that the performance of the rotated general regression neural network model was better than other models.

ANNs have been successful in many areas and problems. However, the hardware improvements are insufficient when the number of hidden layers and nodes is increased. Therefore, its use has come to a standstill due to hardware constraints. However, ANNs have switched from shallow networks to deep networks through GPU and other hardware improvements. In recent years, deep neural networks (DNNs) have attracted great attention by their capability of modeling the huge amount of data with multiple hidden layers, which makes them more advantageous than the traditional neural networks (Coulibaly et al. 2000; Zhang et al. 2001; Kentel 2009; Yaseen et al. 2016; Zhu et al. 2020; Mohammadi et al. 2021). Li et al. (2016) developed DNN models to forecast daily inflow for two reservoirs in China. They compared the models to the basic feed forward neural network and the autoregressive (AR) integrated moving average models. The results show that the DNN models are suitable for performance criteria. Bai et al. (2016) proposed a multiscale deep feature learning method for forecasting the daily inflow values of the Three Gorges Reservoir in China. They stated that the used method has good performance of forecasting peak values. Assem et al. (2017) aimed to provide DNNs model for predicting flow and water level of Shannon River, Ireland. They show that the model performs better than time-series prediction models. They stated that the proposed method could be very convenient for better planning of water resources. Tao et al. (2016) applied a DNNs framework for correcting the estimation bias to satellite-based precipitation estimation products. They indicated that DNNs could extract utility information for estimation of precipitation. Also, the methodology can help to find out further features from satellite datasets for reducing bias. Chen et al. (2012) developed the drought model using deep belief networks (DBNs). The drought index obtained from the standardized precipitation index is estimated with DBNs model for Huaihe River Basin in China. They noted that the model has better prediction performance than the back-propagation neural network.

Ensemble learning, one of the popular methods for various machine learning tasks, is frequently used in hydrology studies (Fan et al. 2019; Ma et al. 2021). Erdal & Karakurt (2013) investigated the use of two ensemble learning paradigms, namely bagging and stochastic gradient boosting in classification and regression trees (CART) in the field of flow prediction. They compared ensemble models with support vector machine (SVM) models. They found that the ensemble learning paradigms were successful in the training phase but did not achieve the same success in the testing phase. They combined bagging and stochastic gradient boosting paradigms to increase the success of the testing phase. As a result, they showed that the ensemble learning paradigms significantly increased success in CART algorithms. Galelli & Castelletti (2013) investigated the prediction ability of randomized trees in terms of accuracy, clarification ability and calculation efficiency in the flow modeling study. For this purpose, they identified Marina catchment (Singapore) and Canning River (Australia) as the study area. They showed that randomized trees had better performance compared to CART, M5, ANNs and multiple linear regressions. Zhao & Chen (2015) proposed a hybrid model based on ensemble empirical mode decomposition (EEMD) and AR for annual flow estimation. They tested the proposed model by using annual flow data obtained from four hydrological stations upstream of the Fenhe River Basin, China. Consequently, they found that the proposed hybrid model was more successful when they compared the EMD-AR and AR models with the proposed EEMD-AR. Zhang et al. (2018) suggested the hybrid EEMD-ENN model consisting of a combination of EEMD and Elman neural network (ENN) to overcome the difficulties of modeling and to increase the accuracy of prediction. In order to test this model, they used annual flow time series values obtained from four main streams in the Dongting Lake basin and four hydrological stations at the outlet of the lake. For this purpose, they have established four different models including back-propagation (BP) neural network, EEMD-BP, ENN and the proposed hybrid EEMD-ENN model. As a result, they have shown that the hybrid model provided better results. Nguyen (2015) used the random forest (RF) algorithm to estimate the incoming flow of Hoa Binh reservoir for 10 flow days. In conclusion, the developed model using the RF algorithm was suitable for predicting incoming flow values.

Extreme gradient boosting (XGBoost) is one of the ensemble learning algorithms. Compared to other algorithms, they offer more robust models with regular terms and column sampling (Chen & Guestrin 2016; Budholiya et al. 2022). This algorithm, which is used in different fields, has also become preferred in hydrology. Ni et al. (2020) created a hybrid model that uses a combination of XGBoost and Gaussian Mixture Model (GMM) for monthly streamflow estimation. They used monthly flow data from Cuntan and Hankou stations in the Yangtze River Basin to model. They proposed the model as a superior alternative for optimum management of water resources. Yu et al. (2020) made a 10-day flow forecast for the Three Gorges Dam in China with the FT-SVR (Fourier transform support vector regression) which was previously developed. The 10-day inflow time series was decomposed into seven components. They estimated each separated component with XGBoost and achieved better prediction results with FT-XGBoost. Using XGBoost, Venkatesan & Mahindrakar (2019) predicted 1–5 h ahead flooding for the Kolar Basin in India with hourly precipitation and runoff data from 1987 to 1989. The model results were compared with RF and SVM and it was stated that the XGBoost method performed better. Li et al. (2020) used elastic net regression (ENR), SVR, RF and XGBoost models for monthly flow forecast and a modified multi-model named modified stacking ensemble strategy (MSES) suggested as an integration method. They said that the RF and XGBoost models have better prediction performance than ENR and SVR. Vogeti et al. (2022) used bi-directional long short-term memory (Bi-LSTM), wavelet neural network (WNN) and XGBoost methods for flow estimation of the Lower Godavari Basin. 80% of 39 years of daily precipitation, evapotranspiration and flow data were reserved for model training and 20% for validation. XGBoost outperformed WNN and Bi-LSTM.

Long short-term memory (LSTM), which is frequently used by researchers in recent years (Katipoğlu & Sarıgöl 2023), was proposed by Hochreiter & Schmidhuber (1997). It can predict the periodic and complex structure of time series with high accuracy. Cheng et al. (2020) used ANN and LSTM models for flow prediction by using precipitation and flow data covering the 1974–2014 periods of Nan River Basin and Ping River Basin in Thailand. They stated that LSTM was better in daily flow prediction than ANN. Rahimzad et al. (2021) estimated daily flows with linear regression (LR), MLP, SVM and LSTM methods using 26 years of precipitation and flow data from the Kentucky River Basin in the USA. They said that LSTM performed better than others. Hu et al. (2020) used the flow data of a station in Tunxi, China and the precipitation data of 11 stations in the region and estimated the flow data at 6 h in the future with LSTM. They compared LSTM results with SVR and MLP. They stated that LSTM had a better performance with R² (0.97). Fu et al. (2020) used LSTM and classical back-propagation neural network model for flow prediction of the Kelantan River on the Malaysia Peninsula. They stated that the LSTM model better predicted rapidly changing flows during dry and rainy periods. Girihagama et al. (2022) used standard and attention-based encoder–decoder LSTM models to estimate flows from 10 different basins in the Ottawa River Basin in Canada. They observed that the encoder–decoder LSTM model had better performance in all basins.

This study addresses the follow aims: (i) it evaluates the performance of deep and machine learning methods in daily streamflow estimation, (ii) it identifies the optimal hyperparameters for daily streamflow estimation and (iii) it assesses the impact of various input combinations on daily streamflow estimation. For these purposes, daily streamflow values in the Göksu River, Turkey were used. The Karahacılı, Kırkkavak and Hamam stations located on the Göksu River were selected in developing models. In this context, deep learning (LSTM) and machine learning (RFR and XGBoost) algorithms were used as modeling approaches. Hyperparameter optimization was performed for LSTM, RFR and XGBoost algorithms using different input combinations and the most suitable model was selected. The rest of the article is structured as follows. In the second section, LSTM architecture and in the third section, ensemble learning algorithms (RFR and XGBoost) are given. In the fourth section, the study area and the data used in the models are introduced. In the fifth section, the selection of hyperparameters of the models and the model results are presented. In the sixth section, the results of the study are examined.

Flow data of three observation stations belong to Electrical Power Resources Survey and Development Administration is used in the Göksu River to develop LSTM, RFR and XGBoost models. The 7,665 daily flow values have been obtained for the Karahacılı (1714) on Gökçay River, Kırkkavak (1719) on Gökdere River and Hamam (1720) on Göksu River stations for the years 1990–2010.

The principal advantages of ensemble models over single models are described with their advanced generalization capabilities and the flexible functional mapping between the variables of the system. Ensemble learning usually consists of three phases, which are resampling, generation of sub-models and pruning and ensemble integration. Resampling, dealing with the generation of subsets of data from the original dataset, is usually the primary character behind a key-ensemble model. The generation of sub-models defines the process of selecting some suitable regression models for the system. To prune the sub-models, it determines the most suitable ensemble configurations and the structures of sub-models. Finally, ensemble integration is a special method, which transforms or chooses the estimates coming from the members, creating the ensemble estimate. In the literature, the most common ensemble learning frameworks are Bagging, Stacking and Boosting (Alobaidi et al. 2018).

The bagging ensemble, which is proposed by Breiman (1996), is one of the most effective techniques for increasing the predictive accuracy of individual classifiers and reducing the variance. In order to create multiple versions of classifiers, Bagging ensemble uses bootstrap samples created by sampling uniformly instances from training datasets. The final results of the different versions of the classifiers are combined by simple voting to obtain a general estimation. In case of regression, the average estimate is given as a result (Pham et al. 2017).

Stacking, which is the technique presented by Wolpert (1992), involves the creation of linear combinations of different estimators to obtain generally improved results. It consists of two stages. In the first stage, different models such as J48, Naïve Bayes and RF are learned according to the dataset and their output help create new datasets. In the second stage, the dataset is used with the learning algorithm to obtain the final output.

Boosting is another type of effective ensemble methodology proposed by Freund & Schapire (1997) based on the learning in sequence. Learning is primarily done on the completed dataset and then on the resultant data obtained from the last learning performance. The main point is to change the weights of each training data point, explicitly. The obtained weights are increased for the misclassified sample data points. In a similar way, they are decreased for the correctly classified sample data points (Verma & Mehta 2017).

Random forest regression (RFR) is a tree-based technique containing the stratification or segmentation of the predicator space into a series of simple regions (Wu et al. 2017). In other words, it is a nonparametric ensemble learning method gathering the results obtained from many individual decision trees. Overfitting is prevented by growing each tree using a bootstrap sample and by selecting from a random subset of variables at each split. Observations, which are not involved in a sample of tree because of the bootstrapping procedure, are called out-of-bag (on average about 36%). They serve as the test set of the tree and they are used in measuring the prediction error. The importance of an interest predictor can be estimated with permutation, by randomly shuffling its values in the out-of-bag samples and comparing the final prediction error to the error obtained prior to the shuffle. The importance estimate formed in this way is called a VIMP and it includes the effects of all interactions, because it removes the effect of predictor on the choice of other variables deeper in the tree (Van der Meer et al. 2017).

XGBoost is a machine learning algorithm proposed by Chen & Guestirn in 2016. Boosting Tree algorithms are based on a decision tree known as a classification and regression tree (CART) (Dong et al. 2020).

Hyperparameters are parameters that are determined initially before the learning process starts in a machine learning model. The values of these parameters do not change when the learning process is over. Learning rate, epoch, etc., parameters can be given as examples.

Hyperparameter values given by default in machine learning models do not guarantee the best performance (Schratz et al. 2019). Therefore, tuning hyperparameter values can greatly affect the performance of the model (Mantovani et al. 2016). The large amount of hyperparameters of the models makes it almost impossible to manually adjust these values. For this reason, many hyperparameter tune methods are available and used in the literature. The most used methods are GridSearch and RandomSearch.

GridSearch creates a new model by trying all possible combinations from the given collection of values for each hyperparameter and returns the hyperparameter combination that provides the highest accuracy. The problem with this method is that the process takes a long time when there are too many hyperparameters and values to try. This causes the method to run very slowly.

In the RandomSearch method, N combinations determined from each hyperparameter value collection are randomly selected and return the hyperparameter combination that provides the highest accuracy. With this method, a search can be made much faster and with an accuracy close to grid search.

The study is conducted for the Göksu River in Turkey, with the aim of improving the accuracy of streamflow value predictions for water resource planning and disaster mitigation against events such as droughts and floods. The complex nature of river basin flows, influenced by basin characteristics (e.g., topography, vegetation), meteorological factors (e.g., precipitation, evaporation, infiltration) and human activities, necessitates advanced modeling techniques. In recent years, deep and machine learning methods have garnered attention due to their ability to effectively model complex and nonlinear processes. Daily flow values of the Göksu River spanning the years 1990–2010 constitute the dataset for this study. This study focuses on estimating daily streamflow values using deep and machine learning techniques, specifically LSTM, RFR and XGBoost approaches. These methods are chosen for their capabilities in capturing temporal dependencies, handling nonlinear relationships and providing accurate predictions. Each model is trained on a portion of the dataset and hyperparameter optimization is performed to enhance their predictive performance. In the modeling stage, data of three flow observation stations on Göksu River was used. The input parameters are flow data of Kırkkavak and Hamam stations while the output parameter is flow data of Karahacılı station. The various models were developed with seven different input combinations. It was used the previous 1-day (Q_t−1), 2-day (Q_t−2), 3-day (Q_t−3), 4-day (Q_t−4) and 5-day (Q_t−5) flow values of Karahacılı station, and daily flow values of Kırkkavak (Q_(1719)t) and Hamam (Q_(1720)t) stations as input parameters in modeling. The total number of data used in the modeling stage is 7,662. In the models, 80% of the total data (1990–2005 years) is allocated to the training set and the remaining 20% of the total data (2005–2010 years) is used in the testing set. Then, the whole data are normalized from 0 to 1 with the help of MinMaxScaler class of sklearn library. In the LSTM, RFR and XGBoost models, python programming language, numpy (URL-1 2018a), pandas (URL-2 2018b), sklearn (URL-3 2018c) and keras (URL-4 2018d) libraries are used. For LSTM, RFR and XGBoost methods, the best model structure was determined belong to each input combination. The hyperparameters selected for the best models are summarized in Table 1.

As mentioned above, LSTM, RFR and XGBoost models have been developed by various input combinations. The performance of the models is assessed using key metrics such as root mean squared error (RMSE), mean absolute error (MAE) and coefficient of determination (R²). These metrics provide insights into the accuracy and reliability of the models' estimations. RMSE, MAE and R² values of LSTM, RFR and XGBoost models are given in Tables 2–4, respectively. Examining LSTM models in Table 2, it was shown that the higher R² values than 0.8 in developed models are obtained except model 6. Model 1, having Q_t−1 parameter, has the highest R² value. The RMSE, MAE and R² values of model 1 are 28.68 m³/s, 12.04 m³/s and 0.853 for the testing set, respectively.

Examining RFR models, model 5 has the best R² value (Table 3). R² values for training and testing sets of this model have been found as 0.918 and 0.871, respectively. All RFR models showed high performance except the model 6 as in the LSTM models.

When Table 4 was examined, model 5 showed better performance than other XGBoost models. RMSE values of the model having the lowest error values were obtained as 20.47 and 26.38 for training and testing sets, respectively. The R² value of the model is 0.871 for the testing set. In all methods, it was seen that the R² value of model 6 having inputs Q_(1719)t and Q_(1720)t increased by adding Q_t−1 to these input parameters in model 7.

Accurate estimation of streamflow values is necessity for early disaster management in many parts of the world. However, it is very difficult to precisely predict the flow when streamflow measurements cannot be made. The slightest error in streamflow estimations can cause serious damage. Therefore, reliable flow estimation plays an important role in the planning of water resources. For these reasons, various flow estimation methods have been developed. The use of deep and machine learning in the development of estimation models has become inevitable, when it is thought that the number of data collected with the developing technology is increasing rapidly nowadays. The fact that studies on deep and machine learning continue with increasing momentum also supports this idea. For this reason, deep and machine learning was used in this study to estimate the flow of the Göksu River. Various LSTM, RFR and XGBoost models were developed by different input combinations. The hyperparameter tuning of the developed models was made and the best model structure was determined. When the models are examined, it is seen that the XGBoost and RFR models perform close to each other, while the success of this model is more remarkable because the XGBoost model has lower error values for the testing set. The outcomes of this study underscore the successful application of deep and machine learning algorithms for accurate streamflow estimation. The XGBoost model showcases its potential in handling the complexities of the Göksu River's flow patterns. The findings contribute to the advancement of streamflow prediction methodologies, benefiting water resource management and disaster preparedness.

Onur Özcanoğlu, who was one of the potential authors of the manuscript, passed away during the preparation of this manuscript. The other authors are grateful to him for his great contribution to his paper.

Data cannot be made publicly available; readers should contact the corresponding author for details.

The authors declare there is no conflict.