A WaveNet-based convolutional neural network for river water level prediction

The water level is an important hydrological feature of rivers, and accurate water level prediction (WLP) is crucial to flood control, shipping and water supply planning and management (Deng et al. 2021). However, variations in the water level are highly nonlinear due to various factors, such as rainfall, runoff, topography, water conservancy projects, and human activities, which increase the difficulty of accurate prediction (Lai et al. 2013; Zhang et al. 2016).

The WLP models mainly include hydrologic models based on physical processes and machine learning (ML) models based on a data-driven approach (abbreviations: physical models and ML models). Physical models can obtain the meticulous simulation of water level processes by establishing basic equations to express the interaction mechanism between variables (Lai et al. 2013). However, the establishment of physical models requires a wealth of professional knowledge, a large number of basic data and physical parameters, the modelling process is complex, difficult and time-consuming. ML models do not need to understand the mechanism of physical systems, and they predict the water level by directly detecting correlations between variables and realizing the mapping of inputs to outputs. The data that ML models require are easy to obtain, and the modelling process is relatively simple (Yin et al. 2021).

Various ML models were used in hydrological prediction tasks. For example, in Ahmed et al. (2022), six different ML algorithms (i.e., linear regression (LR), interaction regression (IR), robust regression (RR), stepwise regression (SR), support vector regression (SVR), boosted trees ensemble regression (BOOSTER), bagged trees ensemble regression (BAGER), XGBoost, tree regression (TR), and Gaussian process regression (GPR)) were developed to predict the river's water level, on a daily basis based on collected data from 1990 to 2019 which were used to train and test the proposed models. They found that the GPR model was capable of predicting the water level of the river with high precision and less uncertainty. In Latif & Ahmed (2020), (2023) and Latif (2023), the LSTM model was used to predict the daily streamflow of the Kowmung River at Cedar Ford in Australia, daily reservoir inflow of the Dokan Dam in Iraq and daily pan evaporation at Sydney airport in Australia, respectively. The results showed the LSTM model outperformed other conventional ML models (e.g., random forest (RF), tree boost (TB), multilayer perceptron neural network (MLP-NN), and boosted regression tree (BRT)). Huang et al. (2021) built three ML models, namely, an artificial neural network (ANN), a nonlinear autoregressive model with exogenous input (NARX) and GRU to simulate the daily Poyang Lake level from 2003 to 2016. They found that ML models with historical memory (i.e., the GRU model) were more suitable for simulating the Poyang Lake level under the influence of the Three Gorges Dam. In Ho et al. (2022), the LSTM model was proposed to predict short-term water levels in tidal sluice gates from 6 to 48 h ahead in the Bac Hung Hai irrigation system in Vietnam. The findings of this study highlighted the performance of LSTM models in providing high-accuracy short-period water level forecasts for areas near estuaries. Kima et al. (2022) used ML models such as gradient boosting (GB), support vector model (SVM), and LSTM to predict the flood water level of the Heungcheon bridge station which was located in the downstream of Bokha bridge. Rainfall, water level, and discharge data of Bokha bridge station from 2005 to 2020 were collected, especially the collected rainfall data were classified into 53 rainfall events using Interevent Time Definition (IETD) analysis of rainfall data. The LSTM model showed the best predictive power and was selected as the optimal model for real-time floodwater level forecasting in this study. Zhang et al. (2018) built four different neural networks to predict the water level of the combined sewer overflow structure in Norway, through a comparison of other different neural networks (e.g., multilayer perceptron (MLP) and wavelet neural network (WNN)), the LSTM and GRU presented superior capabilities for multistep-ahead time series prediction. In Cai et al. (2021), a GRU model was built for groundwater level simulation in 78 catchments in the central-eastern continental United States, the results showed that the GRU model performed better in regions where hydrogeological properties could promote more effective responses of groundwater to external changes. The above shows that the LSTM and GRU models in recurrent neural networks (RNNs) can usually obtain better prediction performance. However, they still have some shortcomings that need to be improved. For example, their internal recurrent connections required processing inputs according to time sequence, resulting in models that could not be trained in parallelism and increased training time (Fan et al. 2021). Furthermore, they consumed additional memory to hold long-term information. For practical problems with very large datasets, the LSTM and GRU models are relatively low in terms of cost-efficiency.

A convolutional neural network (CNN) is a lightweight structure that has unique advantages in capturing spatial dependencies of input data (Collado-Villaverde et al. 2021). Nevertheless, its performance on time series regression tasks is poor (Yan et al. 2020). A new convolutional structure named WaveNet, proposed by DeepMind in 2016, has performed well in sequential analysis problems (van den Oord et al. 2018). It combines the advantages of dilated causal convolutions, residual, and skip connections. It not only receives nonfixed-length inputs but also learns complex long-term dependencies of sequences.

Compared with RNN models, WaveNet has a smaller number of parameters, large receptive fields, and good parallelism. It has good application prospects in solving sequence analysis problems with large datasets (Borovykh et al. 2019; Rizvi et al. 2021). However, WaveNet was initially proposed for audio generation tasks, and it cannot directly address the time series regression problem. Many researchers have improved it and achieved better prediction performance than LSTM or GRU in the prediction tasks on the power load (Wang et al. 2021), traffic flow (Zhang et al. 2021a, 2021b), and air quality (Benhaddi & Ouarzazi 2021) datasets. However, to the best of our knowledge, WaveNet has not been improved for WLP tasks.

This study aims to develop a highly efficient WLP model of the convolutional structure. A more lightweight CNN, named WaveNet-based convolutional neural network (WCNN), was built by improving WaveNet on the basis of WLP characteristics. The GR in China was selected as the study area, the 1/2/3 days ahead water levels were predicted for the Waizhou gauging station (abbreviation: Waizhou station), and the long-term sequence data of gauging stations could be used for verification. To make full use of the ability of the WCNN to extract spatial information, the water level and discharge sequences of three upstream gauging stations were used as auxiliary input features. Two RNN models (LSTM and GRU) were established, and the performance of the WCNN model was evaluated by comparing their results on the test set.

WaveNet is a deep network model proposed by DeepMind for generating raw audio waveforms that have large and flexible receptive fields and good parallelism, and can capture long-term dependencies of sequences (Rethage et al. 2018; van den Oord et al. 2018). WaveNet can be extrapolated to datasets that are outside of audio, and many achievements have been made in research on time series prediction (Luo et al. 2021; Rueda et al. 2021; Nie et al. 2022).

The number and size of the convolution kernels for all residual blocks in the WCNN were the same, which ensured that all residual blocks outputted a uniform shape. As shown in Figure 5, in the first residual block, the predicted variables X and condition A are based on a convolution operation and ReLU function to obtain channel U containing temporal and spatial features. The calculation formula is shown in Equation (3). Subsequently, the TE block is utilized to learn the global temporal information of the channel U to recalibrate the convolutional channel features, and the calculation formula is shown in Equation (4). The output of the TE block is fed into the backbone network by multiplying it with U, and then, it is added to the inputs to obtain the final output of the residual block z₁, as shown in Equations (5) and (6). The other residual blocks receive z_k−1 and output z_k, and the calculation formulas are as follows in Equations (7)–(10).

The WCNN inherits the advantages of WaveNet. It has a lightweight structure, good learning ability and parallelism, and can solve network degradation (vanishing/explosion gradient) problems. It differs from the original WaveNet in the following three points:

• (1)

  Replace the gated activation function in the residual structure with the ReLU function. It can learn nonlinear correlations of sequences effectively, making the WCNN more applicable to 1-D sequential regression problems. This replacement has been proven to be effective in nonstationary and noisy time series forecasting tasks (Borovykh et al. 2019). In addition, the application of the ReLU function can also reduce the complexity of the model and training time.

• (2)

  Replace the dilated convolutional structure with a TE block. A TE block based on an SE block is proposed to learn the long-term temporal dependencies of sequences. It obtains global temporal information by capturing the information between time steps and feeds back this information to the convolutional channel. Thus, the network can select key temporal features for mapping.

• (3)

  For application to 1-D sequential regression studies, the softmax distribution of the output layer is replaced with a linear function.

LSTM and GRU are widely used ML models, and have already been applied to various WLP tasks (Zhang et al. 2018; Ren et al. 2020; Noor et al. 2022). Thus, LSTM and GRU were chosen as the baseline models in this study. LSTM solves the short-term memory and vanishing gradient problems by setting three gate (e.g., forget, input, and output gate) structures and two hidden states (e.g., short-term and long-term state). The specific structure and calculation formulas of LSTM are shown in Gers et al. (1999). GRU is a simplified version of LSTM that combines the two state variables in the LSTM unit into one. It controls the forget gate and input gate by a single gate controller (Géron 2019).

The hyperparameters have a great influence on the prediction performance of the model. The basic network architecture of WCNN is different from that of LSTM and GRU models, thus, its hyperparameters are also different. The network hyperparameters of the WCNN model include the residual block number, kernel size and filter number of the convolutional layer. The network hyperparameters of LSTM and GRU models are the recurrent layer number and neurons number in each layer. The hyperparameters of the three models in the training stage are the same, which are optimizer, epochs, batch size and learning rate. For the above hyperparameters, the grid search method was used to determine the optimum value. This method selected one value from the value range of each hyperparameter of the model and combined the parameters to build the model, and then the training data were used to validate it. The optimal hyperparameter combination was determined by comparing the loss between the predicted value and the measured value on the validation set under different parameter combinations. The range of hyperparameters and the optimal parameter configuration of the three models are shown in Table 2. After optimization by the grid search method, the optimal parameter configuration of the three models is as follows: the WCNN model consists of four residual blocks. The filter number and kernel size of each residual block are 40 and 5, respectively. The LSTM and GRU models consist of two recurrent layers, the number of neurons in the first and second layers are 100 and 50, respectively. The dropout rate is 0.2. For the hyperparameters of models in the training stage, the optimal parameters of the three models are the same. Adam is selected to be the optimizer, the batch size is set to 100 and the timestep is 10. The initial learning rate is set to 0.001, which is reduced by 50% while the validation loss does not decrease in 20 epochs for scheduling.

The number of training parameters is an important measure of model training efficiency. The fewer the training parameters, the shorter the training time of the model. The number of training parameters of WCNN, LSTM and GRU under the same input conditions identified in Section 3.2 was calculated, which were 32,937, 73,851, and 55,851, respectively. The WCNN model had the fewest parameters, and the LSTM model had the most parameters.

The WCNN, LSTM and GRU models were built to simulate Z_wz for 1/2/3 days ahead, and the results are shown in Table 3. Compared with the LSTM and GRU models for 1-day ahead prediction of total water levels, the MAE and RMSE of the WCNN models are improved by 26.30 and 34.98% and by 19.97 and 24.93%, respectively. The NSE and R² are both improved to 0.998 on the complete test set.

The proposed WCNN model was compared with LSTM and GRU models in the WLP of l/2/3 days ahead of the Waizhou station in GR, China. The result shows that the best prediction with the fewest training parameters was achieved in the WCNN model, which was consistent with the findings of variant models of WaveNet proposed by researchers (Benhaddi & Ouarzazi 2021; Zhang et al. 2021a) in air quality prediction and other sequence modelling tasks, respectively. The WCNN model can be generalized to other rivers with different climate conditions. Because the inputs of the model (i.e., water level and discharge data of the gauging stations) reflect the characteristics of the weather conditions, the data of the predicted station and its upstream stations are easy to obtain.

On the other hand, according to the propagating physical characteristics of rivers, the water level and discharge sequences of gauging stations were selected to be the model inputs. The reasonable input features were selected by calculating the MIC of variables, referring to the studies of Lu et al. (2021) and Zhang et al. (2021b). The influence of input features on the prediction results of models was quantified. The results showed that adding variables of high correlation with predicted variables to model inputs could improve prediction performance. But with the increase of input features, the increase rate of prediction accuracy gradually slowed down, and the training time of models also increased. Therefore, when selecting input features, it was necessary to comprehensively consider the relationship between the time cost caused by increasing input features and the degree of performance improvement. When setting the model parameters, the grid search method was used to optimize the parameters and determine the optimal values. The influence of the main hyperparameters of the WCNN model, such as input length, layer number, kernel size, and filter number, on the prediction results was analyzed and discussed in section 4.3. At present, there are various methods for hyperparameter optimization, such as Random Search, Bayesian Optimization, and Genetic Algorithm. These methods will be selected and adopted for our future research.

To improve the effectiveness of WLP models under the conditions of large datasets, a new CNN model named WCNN was proposed by adapting the WaveNet architecture in the audio generation domain. It was applied to predict the water level at the Waizhou station of the GR in China. The main conclusions are as follows:

• (1)

  For l/2/3 days ahead WLP of the Waizhou station, the WCNN, LSTM, and GRU models all had good predicted accuracy. Moreover, the WCNN showed better prediction performance than the baseline models of LSTM and GRU. Compared with the other two models, the model structure of WCNN was lighter, the training parameters were minimal, and the model had good parallelism, which could significantly reduce the training time of the model. It is a more efficient method for WLP.

• (2)

  It is recommended to select the water level and discharge of upstream stations that are highly correlated with the prediction station as input features to improve the prediction accuracy. However, when the input features increase to a certain amount, the increase rate of the prediction accuracy gradually slows down, and the training time of models also increases. The time and data costs associated with the addition of input features should be considered in combination with the incremental performance improvement.

This work was supported by the National Key Research and Development Programme of China (2021YFD1700802).

Data cannot be made publicly available; readers should contact the corresponding author for details.

The authors declare there is no conflict.