The exploration of a Temporal Convolutional Network combined with Encoder-Decoder framework for runoff forecasting

The Temporal Convolutional Network (TCN) and TCN combined with the Encoder-Decoder architecture (TCN-ED) are proposed to forecast runoff in this study. Both models are trained and tested using the hourly data in the Jianxi basin, China. The results indicate that the forecast horizon has a great impact on the forecast ability, and the concentration time of the basin is a critical threshold to the effective forecast horizon for both models. Both models perform poorly in the low ﬂ ow, and goodwell in the medium and high ﬂ ow at most forecast horizons, while it is subject to the forecast horizon in forecasting peak ﬂ ow. TCN-ED has better performance than TCN in runoff forecasting, with higher accuracy, better stability, and insensitivity to ﬂ uctuations in the rainfall process. Therefore, TCN-ED is an effective deep learning solution in runoff forecasting within an appropriate forecast horizon.


INTRODUCTION
Runoff forecasting is of considerable significance to water resources management. Accurate runoff forecasting can guide the hydraulic engineering construction, reservoir operation, flood control, drought relief, and navigation.
According to the extent of physical principles, models for runoff forecasting can be divided into two categories: process-driven models and data-driven models (Yuan et al. ). Process-driven models represent a specific physical process employing experimental formulas before inputting data. Because of the valuable interpretability of process-driven models, they have been widely used by hydrologists (Beven et al. ; Ren-Jun ; Douglas-Mankin et al. ; Wang et al. ). However, given the uncertainty of the hydrological process and the limitations of artificially constructed process-driven models, the parameters and simulation process are challenging to represent hydrological phenomena fully. Due to processdriven models' physical meaning, researchers without a professional background cannot improve the models, resulting in slow model iteration and difficulty in introducing new technologies. With the improvement of data availability and quality, data-driven models can substitute or supplement to process-driven models for runoff forecasting (Yuan et al. ). Data-driven models focus on the optimal mathematical relationships between a forecast object and a predictor, without considering the physical mechanism (Adamowski & Sun ). Therefore, data-driven models are highly transferable. Therefore, sequential regularities contained in a time series can be learned in a more comprehensive, fine-grained way than using a single network (Bian et al. ).
The objective of this study is to explore the ability and stability of TCN and the integration of TCN with Encoder-Decoder architecture (TCN-ED) for runoff forecasting with multi-step ahead times. To achieve this goal, the remainder of this study is organized as follows. The study area, hydrological data, and the data preprocessing method are given The Qilijie station with a drainage area of 14,787 km 2 is selected for this study (Jie et al. ). The primary soils in this area are red, yellow, and paddy soils. The regional climate is dominated by southeast Pacific Ocean and southwest Indian Ocean subtropical monsoons and partly influenced by regional landforms (Tang et al. ). The catchment is moist and rainy, with the mean annual rainfall from 1,800 to 2,200 mm, most of which occurs from March to September. The map of the study catchment is shown in Figure 1. In order to ease the negative effect of the different scales of data on models' learning ability, the normal standardization is applied to data preprocessing, which is defined as follows.
where X Ã (t) is the normal standardization for input data in the tth time. X(t) is the input sequence, and X and σ are the mean and standard deviation of the input sequence, respectively.

Temporal Convolutional Network
The TCN, based on a 1D convolutional network, is a generic network structure for sequence modeling (Liu et al. ).
With the empirical study demonstrated by Bai et al. (), TCN has been proved superior performance for sequence modeling tasks to LSTM.
In order to meet the requirements of sequence modeling tasks, TCN utilizes causal convolutions. Therefore, outputs are only influenced by present and past inputs in each layer (Moor et al. ). In addition, TCN uses a 1D FCN structure, whose convolution layers have the same size as RNNs by adding zero paddings (He & Zhao ).
In order to increase the receptive field with less compu- If the observation at the time step t is represented by X t , the sequence to sequence forecasting problem is to predict the next sequenceX tþ1 , . . . ,X tþk whose length is k, according to the previous j observation X tÀjþ1 , . . . , X t (Shi et al.
The encoder is to refine the information contained in the input and fix it into a context value.
Then, the decoder decodes the context value and outputs the final prediction.
TCN combined with Encoder-Decoder model TCN-ED is proposed in this study, which is compared with the TCN model to verify their ability in runoff forecasting.
The precipitation, evaporation, and runoff for the first 48 h are used as input to both models to forecast the runoff for As shown in Figure 2, the sample enters the temporal block through 1 × 1 convolution. Each temporal block contains two layers of the causal convolutions. The zero paddings are added to make the output length between the layers the same, and the rectified linear unit is used to activate the output. The input of each temporal block is added to the output after 1 × 1 convolution with reference to Bai et al.  VE are expressed as follows: whereQ is the predicted runoff, Q is the observed runoff, and Q is the average of the observed runoff. Both indicators range from negative infinity to 1, where 1 represents a perfect fit

Evaluation and comparison of models performances
In order to compare the performance of TCN and TCN-ED in the learning and forecasting phase, the minimum, mean, and maximum NSE and VE values averaging over 24 forecast horizons of each model in the training and testing stages with 40 rounds are shown in Table 1.
As can be seen from For evaluating the performance of each model intuitively and holistically, Figure 3 shows Gaussian kernel density estimation (GKDE) for all NSE and VE values of each model in the training and testing stages, respectively. For the GKDE curve, the accuracy when the density peak appears is the mode accuracy of the model, the sharpness of the curve  represents the concentration of the accuracy. If the GKDE curve is on the right of the X-axis and the shape is sharp, it means that the model has stable and high-precision results.
As shown in Figure 3,  The above phenomenon shows that TCN may overfit in the training stage, resulting in weaker transferability than TCN-ED whose GKDE curves change less between the training stage and the testing stage.

Models performance with multi-step ahead times
The models during different forecast horizons is closely related to the concentration time of the basin, which needs to be further verified by more cases in more basins. It is evident that TCN-ED's forecast accuracy and stability are higher than those of TCN at almost each forecast horizon, respectively, especially for short forecast horizons up to t þ 12 and long horizons close to t þ 24. In the process of the Encoder-Decoder, the encoder simulates the process of reading and preprocessing in the brain. The context value with a specific length symbolizes the formed memory. The decoder represents the phase when combining known memory and new information to react by the brain. Compared with ANNs whose network layers are directly stacked, the Encoder-Decoder ANNs are not only more conducive to our understanding of the learning process but also make the tensor transmission between network layers more efficient and stable. Therefore, TCN-ED has higher forecast accuracy and stability than TCN at almost every forecast horizon. For TCN and TCN-ED, in order to keep the output length consistent, the zero paddings are added, resulting in the redundant information which interferes with causal convolutions more seriously during the short forecast horizons. In comparison to TCN, TCN-ED's context value which has been refined already ease the problem caused by redundant information and make TCN-ED more advantageous during the short forecast horizons. In addition, the context value is the output of the last time step of the encoder, so the learning memory is time-sensitive, and TCN-ED is superior obviously to TCN during long horizons close to t þ 24.
In flood control forecasting, more attention is paid to the model's ability to forecast large-scale floods. Therefore, the best model results of TCN and TCN-ED are used to evaluate the largest flood event, whose peak is maximum among 19 flood events in the testing stage at forecast horizons t þ 6, t þ 12, t þ 18, and t þ 24, as shown in Figure 5.
The largest flood event, which is considered moderately hazardous, has a maximal flow peak reaching 7,043 m 3 /s in the study period, and the accumulated precipitation in the basin during the flood rising period achieves 134 mm. It can be found from Figure 5 that because there are multiple rain peaks, the forecast of rising limb and flood peaks by both models are unstable, causing the hydrograph jagged. This situation becomes more and more evident as the forecast horizon increases. In terms of peak time, within t þ 12 forecast horizons, the forecast peak time of each model is basically the same as the observed one, but at the t þ 24 forecast horizon, the models' forecast peaks appear significantly later.
At the t þ 6 forecast horizon, the forecast runoff curves of TCN and TCN-ED both fit the observed runoff curve well, which reflects the excellent forecast ability of TCN and TCN-ED at the short forecast horizon. However, TCN's forecast peak flow is later than the observed peak flow, which will put more pressure on flood control. In contrast, the forecast results of TCN-ED are more practical. At the t þ 12 forecast horizon, it is obvious that TCN-ED has higher accuracy in the peak flow forecast than TCN, as

Robustness of the models based on regression analysis
In order to explore the robustness of the models, the relationships between the observation and predictions of the 40 rounds are drawn at the t þ 6, t þ 12, t þ 18, and t þ 24 forecast horizons in the testing stage, which are displayed as scatter plots with the KDE curves in Figure 6.  Second, the performance of each model during various forecast horizons in the testing stage is comprehensively compared. Third, the best model results at the forecast horizons t þ 6, t þ 12, t þ 18, and t þ 24 are used to analyze the model's forecast ability for the maximum floods. Finally, the robustness of the model is explored by the relationships between the observation and predictions of the 40 rounds at the t þ 6, t þ 12, t þ 18, and t þ 24 forecast horizons. The major findings of this study are summarized as follows.
1. The forecast horizon has a significant impact on the forecast ability of TCN and TCN-ED. In this study, NSE and VE values of both models are high and stable within t þ 12 forecast horizons. As the forecast horizon increases after the t þ 12 forecast horizon, NSE and VE values decrease rapidly, indicating that the forecast ability of the models becomes poor. Since the concentration time of the study basin is about 12 h, it can be inferred that the concentration time is a critical threshold to the effective forecast horizon for both models, which needs to be further demonstrated in more basins.
2. Both models perform poorly in the low flow section, and good in the medium and high flow sections during most forecast horizons, while it is conditional to the forecast horizon in forecasting peak flow for both models.
Whether this rule is universal or not also needs to be further verified in more basins, which will also help to improve the forecast ability of ANNs in runoff forecasting.
3. In general, TCN-ED has better performance than TCN in runoff forecasting in this study. TCN-ED shows better