Cascade-based multi-scale AI approach for modeling rainfall-runoff process

Conversion of rainfall to runoff, according to the laws of gravity, vivifies earth, replenishes groundwater, keeps rivers and lakes full of water, and varies the landscape by the action of erosion. The benefit of rainfall-runoff (R-R) modeling as a role of science is providing information for engineers and decision makers to manage, protect, and enhance water resources. Large uncertainties and high non-linearity of the R-R barricade the process-based modeling and seek a black box relationship between driving and resultant variables. So, various black box methods such as artificial intelligence (AI) models have been already presented for R-R simulation (e.g. Sun et al. 2014; Yaseen et al. 2015; Chadalawada et al. 2016; Kwin et al. 2016; Nourani 2017). Among such AI models, Artificial Neural Network (ANN), Extreme Learning Machine (ELM) and Support Vector Machine (SVM) as black box approaches have revealed their capabilities in modeling and forecasting non-linear hydrological processes in general and R-R modeling in particular (e.g. Okkan & Serbes 2012; Raghavendra & Deka 2014; Taormina & Chau 2015; Afan et al. 2016; Gizaw & Gan 2016; Hosseini & Mahjouri 2016; Humphrey et al. 2016; Lima et al. 2016; Yaseen et al. 2016; Yin et al. 2017). On the other hand, available models often rely on empirical algorithms to represent processes that are not well defined, making multi-resolution wavelet analysis appealing, in which long term, high, or low frequency details can be made and used to deep understanding of hydro-environmental processes. The Wavelet Transform (WT) as a data pre-processor presents better interpretation of the process by decomposing non-stationary time series into sub-signals at different scales (levels) and can clarify spectral and temporal information of the signal by combination with ANN, ELM and SVM (WANN, e.g. Nourani et al. 2014; Gorgij et al. 2016); Wavelet-ELM (WELM, e.g. Deo et al. 2016) and Wavelet-SVM (WSVM, e.g. Nourani & Andalib 2015; Kim et al. 2017) models for hydro-environmental processes simulation. In this study, WANN, WELM and WSVM models are utilized for Multi-Station (MS) modeling of R-R. However in AI models, maybe there is not a paramount relation between inputs and outputs. So, dominant input selection is a main challenge of time series modeling, especially in WANN, WELM and WSVM. For this aim, in this study two pre-processing approaches of Self-Organizing Map (SOM) based clustering and mutual information (MI) concept are employed to extract main features and inputs of the WANN, WELM and WSVM methods. SOM as an effective clustering method recently has been used in various hydro-environmental studies (e.g. Kalteh et al. 2008; Adeloye & Rustum 2012; Nourani & Parhizkar 2013; Kar et al. 2015; Chang et al. 2016; Li et al. 2018). On the other hand, MI as a nonlinear measure of data substance can be a robust scheme in necessity of the choice of dominant inputs among numbers of wavelet-based sub-signals (Nourani et al. 2015). Altogether, due to the significance of the model's inputs to acquire the general pattern of R-R process, participation of watershed's effective data has the paramount importance. So, considering both spatial and temporal variations of runoff as a cascade-based MS can elevate prediction of runoff in watersheds. Although MS models were used in some hydrologic research (e.g. Turan & Yurdusev 2009; Nourani & Komasi 2013; Lee & Resdi 2016), to the best of our knowledge there was no study using a MS framework considering seasonality or periodic properties and its influence on the runoff transmission of the watershed for patterning the R-R process. Accordingly, this paper aims to not only predict the outlet runoff values of Little River Watershed (LRW) but also to simulate the runoff time series via a cascade manner at particular points inside the watershed. Therefore, R-R MS modeling is considered in cases where the inside runoff of the LRW is necessary to be predicted. In cascade modeling, the upper sub-basins runoff time series are applied to predict the interior sub-basins runoff, and consequently, central sub-basins are attended in the LRW outlet runoff prediction. Therefore, the MS model can provide a promising platform regarding the runoff amount in LRW critical places. Hence, two different scenarios are considered for R-R MS modeling to identify an appropriate strategy in hydro-environmental studies. In the first scenario, R-R process Markovian property is suggested as the MS model base, where rainfall antecedent and sub-basins runoff time series are shared (Nourani & Komasi 2013). Furthermore, MI non-linear feature extraction criterion, which is a more appropriate measure compared to the Correlation Coefficient (CC) linear measure, is used for the suitable inputs selection of the Least Square SVM (LSSVM), Feed Forward Neural Network (FFNN), and ELM models to prevent from the laborious process of trial-error for input selection where FFNN is the most commonly used AI model in hydrology, LSSVM uses the concept of SVM classifier as a pre-processing tool which sometimes can lead to more accurate results and ELM as a newer generation of AI models were employed and compared in this study, of course other AI-based models (e.g. genetic programming, Ravansalar et al. 2017) can also be applied. In the second scenario, the seasonality-based or multi-scale property of the R-R process would be considered, where the sub-basins runoff time series using the WT are decomposed at an appropriate level to clarify temporal and spectral time series information. Consequently as a new feature extraction approach, both SOM and MI are used respectively for homogeneous sub-signals clustering and choosing the proper agents of clusters, to be fed into LSSVM, FFNN and ELM models for the LRW MS runoff modeling.

The southeast USA is considered as an authoritative region from agricultural, social, and economical points of view, because of its rapidly increasing population which will increase environmental stress and water demand to currently degraded or stressed ecosystems. The LRW covers 334 km² and approximately 30% of the northern half and 40% of the southern half has been occupied by agricultural cropland including eight sub-basins of extent increasing from 2.62 to 334 km² (Bosch et al. 2004, Figure 1(a)). The observed rainfall and runoff time series from the sub-basins of LRW (Figure 1(a)) considered in this research ranging from January 1990 to December 2012 were compiled and recorded (ftp://www.tiftonars.org/databases/LREW). Table 1 shows the used data statistics and Figure 2 indicates the runoff time series and recorded rainfall at the LRW outlet. The first 75% of total data (01/Jan/1990–02/Apr/2007, 6,301 days) were applied for the training and the remaining 25% data set (03/Apr/2007–31/Dec/2012, 2,100 days) used for verification purposes. In this way, higher values of maximum and standard deviation were considered in the training data set, due to the fact that the AI models, LSSVM, ELM and FFNN, can present accurate predictions for unseen data if their interpolator systems are familiar with the same patterns. To speed up training systems, the input and output data were normalized before entering into the training step.

Due to the non-linear influential dynamical parameters in the R-R process, the LSSVM, FFNN, and ELM nonlinear AI-based models via two various scenarios were proposed in this research to predict runoff in the LRW outlet and some interior points. In a black-box modeling task of R-R, in a lumped routing manner, the flow is predicted at river specific locations (at sub-basin outlets). In this regard, as can be seen in Figure 1(b), a storage-routing strategy in a reservoir form (sub-basins) network (cascade) has been used in this research. This storage (reservoir)-routing approach is commonly applied in the watersheds for lumped flow routing (e.g. Boyd et al. 1979). The purpose of the proposed MS model was the runoff prediction of the watershed by relevant sub-basins data as AI models inputs to determine the runoff values at some points within the watershed. Therefore, it would be possible to determine runoff variations in the watershed by MS modeling. The proposed MS runoff modeling (a storage-routing approach) is shown in Figure 1(b) for the LRW. In Figure 1(b), the circles illustrate sub-basins which transform the rainfall to runoff, and squares indicate the sub-basins not only transforming but also transmitting upper sub-basins runoff to the down sub-basins. In the suggested LRW MS modeling through a cascade manner, the sub-basin I runoff was predicted according to the sub-basins M, K and J runoff time series. In the next step, the sub-basin F runoff was predicted by the sub-basin I data and in the final step, the LRW outlet runoff at station B could be determined using the sub-basins F, N and O data. Therefore, the MS runoff modeling pattern of the LRW was completed using three AI modeling methods as a reservoir cascade. Two various scenarios of Markovian and seasonality-based (multi-scale) applied for LRW MS runoff modeling are explained in the following sub-sections (also see Figure 3).

In this scenario, the seasonality of R-R process of the LRW formed the MS model basis. The focus of the second scenario was on applying proper sub-basin dominant frequencies to remove redundant data.

To select the suitable inputs of AI models with regard to the target in a non-linear process, it is necessary to use a robust supervised tool. To this end, an entropy-based feature extraction tool of MI was utilized which is briefly described below.

Whereas redundant information can create problems in the training step of the AI models, it is important to identify the inputs which have similar information value by clustering them to utilize just one member of each cluster as the representative of all members, so this procedure can help to use just one signal instead of many similar signals. According to this, SOM, a powerful soft computing plan that performs a high-dimensional distribution regular mapping onto an orderly low-dimensional grid, was used in this study. Subsequently, SOM can change nonlinear, complicated relationships to simple geometric ones while retaining the data topology formation (Kohonen 1997).

In the simulation step, three non-linear FFNN, ELM and LSSVM data-driven models were utilized in various types of AI models, where the first is a common method in hydro-environmental modeling, the second is a new neural network approach and the third is currently gaining more attention. The aforementioned data-driven models are briefly explained below.

FFNN is known as the prominent kind of ANN, which is utilized in most research for examining different hydro-environmental processes. Three-layer FFNNs provide a general framework for representing nonlinear functional mapping between a set of input and output variables. The specific equation for one target variable of a three-layer FFNN was presented by Kim & Valdes (2003).

ELM is an elevated system of the single layer FFNN architecture, but is different from the typical neural network where tuning of parameters of the feed-forward networks (hidden layer biases and input weights) is not necessary. It should also be noted that ELM tends to remove some drawbacks of FFNN as over-fitting, local minima and unfit training.

For AI modeling (e.g. FFNN, ELM and LSSVM), the codes were developed in the MATLAB^® environment (MathWorks 2010).

The results of the proposed R-R MS modeling using LRW sub-basins information for two different scenarios are investigated in the following sub-sections. In this way obtained results of FFNN, ELM and LSSVM models are compared via the proposed scenarios.

In data-driven FFNN, ELM and LSSVM models, the suitable detection of inputs has a crucial role in improving the model's performance in both calibration and verification steps; also, it prevents over-training of the model. For this purpose, dominant inputs selection and their appropriate lag times for prediction of runoff values of the sub-basins was carried out by MI (Equation (9)). Table 2 shows the sensitivity analysis results via MI in selecting dominant inputs for MS predictions of runoff. Table 2 presents the outcomes of MS for interior sub-basins (at stations I and F) and outlet of the LRW (station B). Furthermore, by fitting different FFNN, ELM and LSSVM models to all sub-basins of the LRW (one AI model for each sub-basin, separately), it will be possible to update the runoff time series in the future, if necessary. On the other hand, for more understanding of the R-R process and evaluating the inputs of the black box MS models, conversion of rainfall to runoff was surveyed within the LRW. Since the LRW is a relatively small catchment, average rainfall of sub-basins is almost symmetric. So, outlet runoff would be affected by land use/cover. To this end, a reliable identification of hydrologic reaction to soil physical characteristics, such as saturated hydraulic conductivity, bulk density, and moisture retention, will elucidate the outlet runoff value of sub-basins. These characteristics can impact sub-basins hydrology by influencing pathways and transformation rates of rainfall to runoff networks (Price et al. 2010). In general, infiltration rate and rainfall absorption of soils from highest to lowest are forested, pasture, cropland and urban, where forested soils indicated significantly lower bulk densities and higher infiltration rates, and water holding capacities, than cropland and pasture soils. Via the LandSat scene, the LRW land cover was grouped into five categories of cropland, wetland/water, pasture, forested and urban (Figure 5). Considering Figures 1(c) and 6, high elevation lands have mostly forested cover whereas the lowlands are covered by pasture and cropland. After considering the general conditions and land cover/use arrangement of the LRW, upstream sub-basins M, K, J and I should have lower values of runoff per area compared to downstream sub-basins N and O. However, based on the statistics and land cover/use classification of the LRW sub-basins, soil type relation with outlet runoff per area is not observed in practice (Figure 5 and Table 1). So, it could be deduced that the inconformity between outlet runoff values per area with land cover/use classification has an anthropogenic source. As in sub-basins with high pasture and cropland, water consumption is high which reduces outlet runoff from these sub-basins whereas regularly sub-basins with high pasture and cropland with lack of forested areas should allocate less runoff absorption value per area to themselves due to more bulk densities, less infiltration rates, and water holding capacities than forested land. In addition, the MS models input selection using MI was compatible with the LRW geomorphology. In the first model, regarding the sub-basin I runoff prediction, the sub-basins K and J runoff values at time t with no delay were chosen as the inputs due to the high slope and short distance of the river channels between sub-basins K and J outlet stations with the station of I. However, sub-basin M runoff with a 1 day delay was chosen by MI (Table 2). In the second model, sub-basin I runoff with 1 day lag was chosen to model the sub-basin F runoff (Table 2). In the third model, due to the long channel length and low channel slope between F and B stations, runoff data with 2 days lag were selected for the model. Moreover, close to the LRW outlet, sub-basins O and N, shared their runoff with no lag for the runoff predictions at station B (Table 2). Comparison of the obtained results for runoff values predicted by MS models (see Table 2) revealed that although FFNN, ELM and LSSVM models could exhibit acceptable results, ELM could lead to more accuracy than FFNN and LSSVM models. An appropriate data pre-processing scheme such as the method proposed in scenario 2 could enhance the modeling performance. The following sub-section presents the result of MS modeling via scenario 2.

For a better R-R process understanding, the time series clarifying in both spectral and temporal terms would be helpful. Therefore, to provide such clarifications and to extract seasonal multi-scale properties, WT was attached to the MS model to enhance the accuracy of R-R modeling. Appropriate MW selection would provide a challenge in hybrid wavelet-AI modeling as the modeling outcomes may be influenced by the applied type of MW. The main goal of WT is to find the similarity between the used wavelet prototype and analyzed time series and because of the coif2 MW structure, which is mainly the same as the runoff signal compared to other MWs like Daubechies-2, 5 (db2, db5) (Figure 7), it could well capture the features of signal and was chosen as MW for the runoff time series decomposition in this research. Moreover, the suitable decomposition level in the wavelet-AI models is mainly achieved by sensitivity analysis. Whereas the LRW R-R process could have various seasonality patterns, the decomposition level 8 contains eight details such as 2¹-day mode, 2²-day mode, 2³-day mode (nearly weekly), 2⁴-day mode, 2⁵-day mode, 2⁶-day mode, 2⁷-day mode and 2⁸-day mode (approximately annually) and one approximation signal as suitable level for the observed LRW runoff time series decomposition. The following step in the second scenario would be feeding the AI models using the obtained sub-signals (details and approximation) after the time series decomposition via the wavelet. However, such numerous sub-signals consideration (decomposition at level 8 would give eight details and one approximation sub-signal, and in total nine runoff sub-signals for each station) as the AI models inputs and redundant data may prompt network over fitting, divergence, poor precision and obscurity. Consequently, for the optimization of input layer of the models and improving the rate of model training and performance, it was attempted to select the main sub-signals of the decomposed runoff time series of the sub-basins to be imposed to AI models. Dominant input variables selection could be an efficient alternative for the conventional method of trial and error in complicated hydro-environmental systems. In the second proposed scenario, SOM and MI hybrid was applied to pick dominant seasonalities. After the sub-basins runoff multi-scale sub-signals extraction, spatio-temporal clustering was performed using SOM (e.g. Nourani & Parhizkar 2013). The clustering results for the decomposed time series of LRW can be seen in Table 3. The decomposed runoff sub-signals were classified into six groups for sub-basin I and B modelling, and four groups for sub-basin F. The clustering outcomes by unsupervised SOM indicated that mainly the sub-signals were grouped according to their frequency scale, whereas low frequency sub-signals and approximation were grouped in the same groups and high and low frequencies were separated. In addition to the clustering (by SOM), MI was employed to pick dominant sub-signals from each cluster. Similar to the first scenario, the sub-signals from each cluster which had higher non-linear correlation (high MI) with the main time series (target) were consequently entered into the AI models. Table 4 presents the dominant inputs of MS model in the second scenario picked by MI. In the first model, for sub-basin I runoff prediction, sub-basin J runoff time series participated with four sub-signals, i.e. 2⁸-day, 2³-day, 2²-day and 2¹-day modes, and station I rainfall time series at time t. It is worth mentioning that sub-basin J includes more runoff and a larger area in comparison with sub-basins M and K. As the second effective area, sub-basin K in producing sub-basin I runoff, shared its approximation sub-signal and 2⁷-day mode. It can be noted that the furthest sub-basin M with runoff poverty due to huge forested lands was not desired in the runoff prediction of sub-basin I. In the second model, for the sub-basin F runoff prediction, the approximation, 2⁸-day, 2⁷-day and 2³-day, modes of sub-basin I as well as station F rainfall time series at time t were chosen as inputs. In the third model, for sub-basin B runoff prediction, sub-basin F as a sub-basin including higher runoff and far from the LRW outlet, and sub-basins O and N with low runoff values per area and close to station B, shared their suitable sub-signals as inputs. It can be observed that according to the sub-basins properties, high and low frequencies from the near sub-basins and the far sub-basin respectively were chosen as the inputs using the suggested SOM-MI method. The MS models results through LSSVM, FFNN, and ELM methods are shown in Table 4 for scenario 2. The comparison of results showed that LSSVM has slightly more accuracy than FFNN, this is perhaps because firstly FFNNs often converge on local minima rather than global minima, and secondly FFNNs often overfit if training goes on too long, meaning that for any given pattern, an FFNN might start to consider the noise as part of the pattern. Also, superiority of ELM was proved to LSSVM and FFNN models due to presenting good generalization performance. For the second scenario, the observed and predicted runoff value in the LRW outlet obtained via ELM model can be seen in Figure 8.

The MS runoff predictions for the LRW indicated differences between two proposed scenarios of Markovian and seasonality-based modeling (Figure 9, Table 5). The AI models results indicated that scenario 2 can provide more precision outcomes as far as the R-R process seasonal (multi-scale) pattern was concerned. The justification would be that in scenario 2 the models were fed by dominant pre-processed data, whereas in scenario 1, the worthiness of sub-basins was not compared. Meanwhile, the accuracy of scenario 1 was lost because of the absence of temporal pre-processing and data robust purge. The prediction results of the LRW runoff via ELM can be seen in Figure 9. The second scenario results, employing both spatial and temporal pre-processing methods using feature extraction criterion, were more accurate compared to the first scenario results. Consequently, the wavelet-AI models, using SOM and MI to capture the space and/or time variations in the process, are a suitable and reliable MS runoff modeling method. After decomposition of time series by wavelet, the sub-basins multi-scale sub-signals entered the SOM and the sub-signals with similar properties were grouped in one cluster. Subsequently, using a suitable feature extraction criterion (e.g. MI), effective sub-signals from each cluster were picked, and those reflecting the runoff process dominant seasonality were imposed into AI models. In this way, just dominant information was fed to the models. To assess the suggested MS method, the station B runoff was also predicted using just rainfall and runoff time series antecedent of this station, where up to 3 days' lag time was considered through the trial-error process (i.e. ⁠) as inputs. Such antecedent modeling and MS modeling results via both scenarios can be observed in Table 5. According to the obtained assessment criteria shown in Table 5, it is evident that the cascade-based MS method has more accuracy in comparison with the lonely station B modeling. Moreover, Figure 9 shows a comparison between the computed and the observed runoff time series at the LRW outlet. As can be seen, the seasonality-based MS modeling could provide more accurate runoff predictions (especially peak values), as it considers runoff spatial variations within the watershed and various upper sub-basins dominant frequencies contributions. The residuals (error) Probability Density Function (PDF) of the proposed scenarios via ELM model in the outlet runoff prediction can be observed in Figure 10 where the scenario 2 error PDF with lower values of mean and standard deviation is well-proportioned having high errors accumulation in zero values and, regarding the B antecedent, indicates the highest errors standard deviation.

In this study, three AI-models (FFNN, ELM and LSSVM) were applied for LRW cascade-based MS runoff prediction (located in Georgia, USA) via two various scenarios (Markovian and seasonality-based scenarios). Such MS modeling could reveal information about the watershed internal hydrometric stations and LRW outlet. The sub-basins daily rainfall and runoff time series were considered over the study area as the potential inputs and the LRW interior and outlet runoff values were considered as the AI models outputs. In such a cascade manner the output runoff was predicted using data from upper sub-basins. Before modeling, the conversion of rainfall to runoff in LRW was investigated by land cover classification and data statistics. It was deduced that in the sub-basins with high cropland and pasture, water consumption is high which reduces outlet runoff from such sub-basins. In the first scenario, the R-R time series antecedent were used as inputs, but for appropriate input selection, supervised feature extraction criterion of MI was applied to prevent the trial-error process. The MI input selection results indicated conformity with the LRW geomorphology (i.e. the input sub-basins far from the output sub-basin and low slope would have more lagged time in comparison with the high slope or near sub-basins). Considering the second scenario and to improve the first scenario results, data pre-processing using feature extraction methods of WT and SOM-MI led to important hydrological parameters detection which proved helpful in enhancing AI-based MS runoff predictions. The second scenario success was because of the fact that the LRW R-R process obeys from a multi-scale seasonal pattern which can be covered using WT. The scenarios comparison indicated that using WT to capture sub-basins multi-scale features could enhance the model accuracy if it is combined with a promising feature extraction method such as SOM-MI, compared to ad hoc AI-models. Runoff time series of sub-basins were decomposed at level 8, which not only considers the dominant seasonality but also does not mar the clustering. Afterwards, decomposed sub-signals were clustered via SOM, and then the MI feature extraction criterion picked the effective sub-signals of clusters as AI models inputs. Moreover, it was concluded that for the near upstream sub-basins, higher frequency sub-signals participate in LRW runoff predictions, while for far sub-basins it is the low frequency sub-signals. Finally, the comparison of the results achieved by LSSVM, FFNN, and ELM models indicated that ELM had superior efficiency than FFNN and LSSVM as ELM could reveal well generalization efficiency and learning process speed up by simplifying the training step avoiding over-fitting and improving the accuracy of runoff prediction (Deo et al. 2016). For future research, it can be suggested to use and compare the results of the theory-driven (conceptual and physical based) models with the outcomes of the proposed methodology for other watersheds. Moreover, other feature extraction methods and alternative AI models may be examined via the proposed scenarios.