Unveiling flood-generating mechanisms using circular statistics-based machine learning approach without the need for discharge data during inference

Flooding is one of the most destructive natural disasters in the world, which leads to serious fatalities and massive property losses (Jonkman 2005; Dottori et al. 2018; Koç & Thieken 2018). In the context of climate warming, the frequency and intensity of extreme precipitation are expected to increase, thereby increasing the risk of flooding (Kendon et al. 2014; Alfieri et al. 2015; Mallakpour & Villarini 2015; Arnell & Gosling 2016; Esposito et al. 2018; Yin et al. 2018; Hounkpè et al. 2019). The impact of climate change on flood characteristics is strongly influenced by the specific and nuanced details of global changes (Kundzewicz et al. 2019), which encompass a complex interplay of factors such as changes in precipitation patterns, temperature, land use, and soil moisture regimes, among others. Understanding the main drivers of flooding is thus essential for flood hazard mitigation and water resources management (Winsemius et al. 2016; Keller et al. 2018).

Extensive studies have been conducted on changes in the frequency of flood events (e.g., Slater & Villarini 2016; Wu & Qian 2017; Najibi & Devineni 2018). On this basis, there is a growing interest in attributing floods to their causal factors, which is crucial to gaining a better understanding of the underlying physical mechanisms (Neri et al. 2019). The Budyko framework (Budyko 1974) is frequently employed to assess the sensitivity of hydrological components such as discharge, evapotranspiration, and soil moisture to various impact factors (Gudmundsson et al. 2016, 2017). Patterson et al. (2013) used Budyko curves to ascribe changes in streamflow and found that human impacts were equivalent to or even greater than climate impacts over 27% of the South Atlantic basins, USA. Berghuijs et al. (2017) developed an improved Budyko-based method to assess the sensitivity of average annual runoff, and the results indicate that runoff is the most sensitive to changes in precipitation for 83% of land areas of the globe. Milly et al. (2018) revealed that temperature can affect flood peaks through evapotranspiration and the snow-related processes based on extensions of the Budyko water-balance hypothesis. Statistical methods are also used to identify flood-generating schemes in complement with the Budyko framework.

However, the Budyko framework is limited by assuming a linear relationship between precipitation and evaporation, which may not hold true for certain climatic conditions in watersheds. Additionally, the framework only considers mean annual precipitation and evapotranspiration, ignoring their temporal variabilities. To address these limitations, many studies employed various methods, such as incorporating nonlinear relationships into models, considering seasonal changes in water balance, and utilizing the remote sensing data to better capture the spatiotemporal variability. Slater & Villarini (2017) modeled streamflow as a function of precipitation, temperature, antecedent wetness, agricultural land cover and population using the Generalized Additive Model for the U.S. results showed that rainfall is the dominant source of discharge in rain-on-snow events along the West Coast, Cascades and the Sierra Nevada, while snowmelt dominates streamflow in the other regions. Kundzewicz et al. (2019) used the Generalized Extreme Value distribution to assess flood risk caused by rain-on-snow events over the conterminous United States. Circular statistics (Burn 1997) are widely applied to investigate the consistency of hypothesized flood-generating processes and maximum annual flow (Berghuijs et al. 2016; Slater & Villarini 2017; Berghuijs et al. 2019). Process-based models were also adopted to investigate the factors affecting discharge characteristics. For example, Wang et al. (2017) found that the human-induced changes in land use/land cover can alter the pattern and the magnitude of streamflow based on the Soil and Water Assessment Tool (SWAT) model.

Discharge data are indispensable when the aforementioned statistical methods or process-based models are used to identify flood-generating mechanisms. However, it is important to note that such data may not always be readily available. Discharge data are typically obtained from monitoring stations or derived from models, which can limit the spatial resolution of the data. In situations where critical variables are missing, traditional statistical methods are often unable to perform effective analysis. Machine learning algorithms use the power of big data to learn and optimize through automatic learning and iteration, enabling them to evaluate the relationships between various factors and perform effective prediction and analysis even in the absence of certain variables. Parajka et al. (2010) identified the different seasonal characteristics of flood regimes by the k-mean clustering technique using annual maximum of flow and precipitation. Curran & Biles (2021) estimated the metrics of mean monthly streamflow to guide the identification of seasonal flow drivers by principal component analysis. Stein et al. (2021) employed a random forest (RF) model with accumulated local effects to investigate the influence of climate and catchment attributes on the spatial distribution of flood-generating processes.

In this study, we present a machine learning framework based on circular statistics for determining flood-generating mechanisms even in situations with limited discharge data. This study seeks to evaluate the potential of machine learning in identifying flood-generating mechanisms without relying on discharge information, contributing to a more effective and comprehensive approach to understanding and managing flood risk.

We utilized hydrological variables from the Catchment Land Surface Model of the Global Land Data Assimilation System (GLDAS) version 2.0 at the 0.25° spatial scale, including precipitation, soil moisture, and snowmelt to maintain consistency among these variables and their inherent interrelations. Meanwhile, the discharge data is obtained from the Global Reach-scale A priori Discharge Estimates for Surface Water and Ocean Topography (GRADES), as GLDAS does not have this variable. GRADES is a model-derived 0.25° daily discharge database based on the hydrography data from Multi-Error-Removed-Improved-Terrain Hydro. The modeling chain used to derive the dataset includes the Variable Infiltration Capacity land surface model and Routing Application for Parallel computatIon of Discharge river routing model. More detailed information about GRADES can be found in Lin et al. (2019). A 35-year period from 1979 to 2013 was considered to provide a sufficiently long temporal overlap between GLDAS and GRADES.

The definition of flooding occurrence of each year is the maximum annual discharge. Three major mechanisms, namely precipitation, soil moisture and snowmelt, are considered the causes of the flooding occurrence following Berghuijs et al. (2019). For the first mechanism, floods are caused by extreme precipitation events. In other words, the maximum annual flow (MAF) and the largest precipitation event occurred at the same time of the year. Regulated by the second mechanism, the MAF occurred simultaneously with the largest soil moisture event. When the soil moisture is high, even light rainstorms can cause floods. This hypothesis suggests that the antecedent soil moisture storage controlled by seasonal rainfall and evaporation is the primary controlling factor for the streamflow generation in a flood event. Under the last mechanism, the floods are caused by the largest snowmelt event, which means that the occurrence dates for the MAF and the largest snowmelt are the same. The proposed mechanisms serve as the basis for our analyses on the drivers of flood generation.

The term is the date of occurrence of process i in each individual year j; equals 365 days (366 for leap years), and n is the number of years of the data (35 years for this study).

Three different types of flooding occurrences are classified by the calculated relative importance: P type (dominated by precipitation), M type (dominated by soil moisture excess), and N type (dominated by snowmelt). The dominant type is defined by the flood-generating mechanism with the highest relative importance.

Besides the dominant mechanism, other mechanisms may also regulate flood generations for a substantial portion of flooding occurrences, which leads to multiple mechanisms in areas of interest. Therefore, a sub-type flooding classification is needed to better understand the complex hydrological processes. Hence, to classify flood sub-types, we employed the k-means clustering algorithm. The relative importance for each grid will be input into the k-means clustering model. The model is implemented by the Scikit-learn software package to classify flood mechanisms into sub-types. To determine the optimal number of flood sub-types, two contour metrics, Silhouette Coefficient (SC) and Calinski-Harabasz Index (CI), were calculated to evaluate the performance of the clustering. SC measures the average distance of each sample from other samples, while CI assesses the covariance between different classes. In simpler terms, higher SC and CI values indicate that the clustering results are more effective and accurate.

An RF model is trained to predict the flood type. RF is a supervised ensemble learning algorithm to solve problems by creating and combining regression trees (Breiman 2001). It can tackle high dimensional nonlinear relationship (Addor et al. 2018), which is suitable for the investigation of flood generation with other factors. A more detailed description of the RF algorithm can be found in (Wiener 2002). This model has been widely applied into the hydrometeorological field (Turini et al. 2019; Xu et al. 2019; Zhang & Shi 2019).

The inputs of the RF model are set as longitude, latitude, and the occurrence day of the highest precipitation/soil moisture excess/snowmelt of each year during the period of 1979–2013, and the output is the flood type. The selection of the predictor variables is crucial for gaining insights into the factors that contribute to the occurrence of floods. The inclusion of longitude and latitude enables the model to identify the location and its associated flood type with great accuracy. This is because different areas are prone to different types of flooding due to their unique topographical and geographical characteristics. In addition, floods are often caused by excessive water exceeding the holding capacity of soil due to heavy rainfall, rapid snowmelt, and high soil moisture excess. By incorporating the dates on which the highest precipitation, soil moisture excess and snowmelt occurred in the model, it becomes possible to isolate the underlying causes of the flooding event and better predict the expected flood type. However, the original information of the occurrence days of the highest precipitation, soil moisture, and snowmelt during a period need to be transformed before being inputted into the model. These days are represented as numbers between 1 and 365 (366 for leap years) and their statistics are calculated, including minimum, maximum, mean, standard deviation, range, peak-to-peak, and 7%iles from 20th to 80th. Anomaly for each percentile is also calculated by subtracting the mean from the percentile value. Consequently, 20 statistics are generated for each variable of precipitation, soil moisture, and snowmelt. In total, 62 features are fed into the model for training and prediction.

To minimize spatial contiguity between the test and training sets, a 10-fold cross-validation with a longitude-based partition was employed (e.g., areas between 0°E and 36°E for testing and other areas for training) instead of a random split. This method was implemented to ensure that the model's performance accurately reflects its generalizability to regions that are geographically distant from the training data. The dataset is divided along longitude instead of latitude to ensure sufficient samples for each of the flood types in both training and testing datasets and therefore increases robustness of the RF mode. By partitioning based on longitude, the training and testing sets are more representative of the geographical diversity of the region under investigation, thereby potentially increasing the robustness of the findings.

Furthermore, the feature importance of the RF model is calculated based on how much each feature contributes to the reduction in impurity when building the decision trees that make up the forest. When building an RF, each decision tree is constructed using a randomly selected subset of features. This helps to prevent overfitting and increase accuracy. For each decision tree, the importance of each feature is calculated by measuring how much it reduces impurity across all of its nodes. The resulting values are then averaged across all trees in the forest to obtain the final feature importance. This feature importance can be used to identify which features are most important for making accurate predictions with the RF model. This information can be useful for understanding the underlying dynamics of the data being modeled.

Globally, maximum annual precipitation is an important flood driver with the global mean of 0.387 (Figure 2(a)). In this study, a mechanism driver is deemed to be the overwhelming (respectively, insignificant) one if its relative importance score exceeds 0.8 (respectively, falls below 0.2). We adopted the threshold values (0.8 and 0.2) by referring to several studies, including Wang et al. (2021), Kim et al. (2018), Nguyen et al. (2017), and Zhou et al. (2019). Extreme precipitation takes the overwhelming control on flood generation for 12% of the global land area. These areas are primarily located in and near Southeast Asia, Central Africa, Northern South America, Southern North America, and Northwestern North America. In contrast, maximum annual precipitation is not a significant flood driver for approximately one-third of the globe; these areas are mainly located in Europe, the U.S., and Western Asia.

Among the three flood-generating mechanisms, soil moisture excess is the most important flood driver with the global mean of 0.535 (Figure 2(b)). This driver overwhelms flood generation in nearly one-quarter of global land areas, which exceeds the areas overwhelmed by the other two drivers combined. These regions are predominantly located in Western Asia, Northern Asia, Europe, Central America, and Central Africa. Soil moisture excess is an insignificant mechanism for flooding in such areas as Southern Asia, Northeast Asia, Central Africa, and Northwestern North America, which accounts for 20% of global land areas.

Although snowmelt has a low global average value of 0.078 as a flood driver, its significance can vary greatly depending on location and local conditions (Figure 2(c)). Therefore, while it may not be the most significant flood driver globally, it should not be overlooked in specific regions. Flood is overwhelmingly controlled by snowmelt in a few areas, and the relative importance is insignificant in nearly 85% of global land areas. Snowmelt has a negligible impact on the flood formation in low-latitude regions and the Southern Hemisphere. By contrast, it has a more significant influence in the areas with high latitudes and high altitudes, such as Southwestern Africa, along the west coast of the U.S. (the Cascade Range and Sierra Nevada Mountains), Northern North America, and High Mountain Asia.

The above results are similar to the studies of Berghuijs et al. (2019) in Europe, and of Berghuijs et al. (2016) in the United States with some differences. For example, a lower relative importance of snowmelt (0.4–0.5) in Northeastern Europe is found in this study, whereas Berghuijs et al. (2019) concluded that 31% of the catchments in Northeastern Europe are dominated by snowmelt with the relative importance larger than 0.5. The difference may be explained by the following: (1) Data sources such as the GRADES and GLDAS data are used in this study, whereas the European Flood Database (Hall et al. 2015) is adopted in Berghuijs et al. (2019) and (2) we modeled snowmelt using the GLDAS while the snowmelt described in the study by Berghuijs et al. (2019) was defined as the sum of daily liquid precipitation and snowmelt during the melting days.

The evaluated metrics of the RF model for the primary classification are shown in Table 1. Due to the large difference in the proportion of each type, the weighted average values are used to assess the overall performance. The RF model shows a precision value of 0.692 and a recall value of 0.701. The F1-score, the harmonic average of precision and recall also reaches 0.692. Of the three types of mechanisms, the RF model performs the best for the M type followed by the P type and the N type in terms of the F1-score value. It is more difficult to accurately identify the types of P and N compared to the M type due to the lower number of training samples. Nevertheless, the RF model maintains an accuracy of 0.701 for all three types.

Figure 3 shows the spatial distributions of three flood types estimated on the basis of the circular statistics and predicted by the RF model in the 10-fold cross-validation. In general, the RF model can capture the spatial patterns of the three flood types. Some areas identified as M type by the circular statistics are classified as P type by the RF predictions. Another evident discrepancy between the circular statistics-based estimations and the RF predictions is that many areas belonging to the N type in North Asia are misclassified into the M type. Since the RF model does not use the discharge data, it cannot distinguish one type from another, when the conditions of two or three among precipitation, soil moisture, and snowmelt are similar. The meteorological background between adjacent grids usually maintains a high spatial continuity, whereas the discharge characteristic is more affected by the local topography. The predicted flood types from the RF model do not show a high spatial heterogeneity. The model also predicts a larger N-type area than circular statistics-based estimations in Tibet. The complicated topography in this area substantially affects the hydrological process. The limited number of input variables used in the RF model cannot capture such a topographic-induced process, which is partially responsible for the poor model performance. In Eastern Asia, Central Africa, and Northern South America, some areas that originally belong to the M type are classified as P type by the model. This is because extreme rainfall and excess soil moisture in these areas have similar contributions to flooding (Figure 2(a) and 2(b)), which makes the model more difficult to correctly predict the flood type. For example, those areas misclassified as the P type in Northern Africa show the relative importance of extreme precipitation is 0.5–0.6 while the relative importance of soil moisture excess is 0.3–0.5.

Specifically, the P type can be further classified into P-* type, P-M type, and PN-* type. The P-* type indicates extreme precipitation overwhelmingly dominates the flood generation, while soil moisture excess and snowmelt play little role in the process. The areas with this sub-type are rare, and they are mainly distributed in southwest China, north South America and central Africa. The P-M type is mainly distributed in southern Asia, central Africa and northern South America. The floods occurring in these areas are most likely influenced by extreme precipitation and soil moisture excess. It is interesting to find that the P-* type is often surrounded by the P-M-type areas. Different from the P-M type, the PN-* type represents that soil moisture excess accounts for a negligible portion of flooding occurrences while both extreme precipitation and snowmelt play significant roles in flood generation.

The M type can be further divided into four sub-types. The M-PN type indicates that soil moisture excess overwhelmingly dominates the flood with a mean α_m value of 0.938. This flood type is widely distributed over the globe but quite scattered. The M-P type represents that both extreme precipitation and soil moisture excess substantially contribute to the flood generation with a mean relative importance value of 0.729 for the former and a mean value of 0.269 for the later. This type occurs in large swaths of high latitudes in the Northern Hemisphere. It is cold all year round in these areas, and snowmelt has little impact on discharge. Different from the M-P type, the M-N type indicates that snowmelt makes a substantial contribution to flood generation and the mean relative importance of this type is approximately 0.344. This type overwhelms other types in North America and Europe as well as some areas of central and western Asia. The MP-* type represents a similar contribution of soil moisture excess and extreme precipitation to the flood generation but little contribution of snowmelt, and is widely distributed in Northern Africa, central South America, and Australia.

As for the N type, it corresponds to only one type N-PM, showing snowmelt overwhelms flood drivers with a mean α_m of 0.782. The N-PM type represents areas under the primary control of snowmelt and the low influence of precipitation and soil moisture (with a mean value of 0.105 and 0.113, respectively). It should be noted that some overlap was observed between the newly identified clusters and originally three classes. For example, some data points in the newly allocated P-MN category may have originally belonged to the M or N classes. This overlap was not prevented by the lack of inherent structure in the k-means algorithm. Specifically, N type represents the areas with the highest influence from snowmelt no matter how similar its relative importance is to the other two factors. Also, the number of grids with the N-PM type is 5, 050, while the number of the N type is almost doubled.

We also attempt to use the RF model to predict these flood sub-types. Following the same approach as predicting the primary types of flooding, the RF models for predicting sub-types of flooding are also trained using longitude, latitude, and the occurrence day of the highest levels of precipitation, soil moisture, and snowmelt after applying the unordered transformation as the predictors. However, the models do not perform well with 0.467 accuracy and 0.461 F1-score (Table 1) in the 10-fold cross-validation due to insufficient training samples and input features. Similar to the spatial distribution of the three predicted types in Figure 5(b), the sub-types predictions also show a great spatial continuity compared with the scattered distribution in circular statistics-based estimations (Figure 5(a) and 5(b)).

Feature importance is an essential tool in machine learning that helps us understand which variables carry the most weight in predicting the outcome of interest. It measures how much each feature contributes to the accuracy of the model prediction. By understanding feature importance, we can gain insights into the underlying factors that drive the prediction results. In this study, the RF model uses 62 features to predict flooding types as described in Section 3.3 (Figure 6). The results show that geographic information including longitude and latitude are the most important variables, with an importance score exceeding 0.04. This suggests that the location of a particular area can have a significant impact on the likelihood of flooding. Interestingly, the model also finds that low percentile values (i.e., 20th, 30th, and 40th) for the occurrence day of the highest soil moisture and high percentile values (i.e., 60th, 70th, and 80th) for the occurrence day of the highest precipitation are relatively more important variables. Furthermore, the standard deviation for the highest precipitation and soil moisture are also found to be important features in the model. This suggests that variations in precipitation and soil moisture could significantly affect flooding occurrence.

The top-down hypothesis explaining the relative importance of floods provides a simple and repeatable method to decipher the first level of understanding of the different flood-generating mechanisms. The RF model proposed in this study offers a simple and direct way to detect the dominant mechanism. The good performance between the RF model predictions and statistics calculations suggests that the flood-generating mechanism can be predicted without discharge input. Compared with other studies that use circular statistic to learn the process controls on floods (e.g., Parajka et al. 2010; Slater & Villarini 2017), our trained RF model achieves robust results while not relying on the discharge data. This advantage is particularly important in the areas without discharge data. It seems that our finding that soil moisture is the primary factor influencing global flood occurrences contradicts some of the previous studies. For instance, Brunner & Fischer (2022) and Pechlivanidis et al. (2020) concluded that precipitation or snow melt are the main drivers. However, it should be noted that these studies did not consider soil moisture as a candidate factor for flood occurrence, and thus cannot be directly compared to our conclusions. Moreover, Stein et al. (2020) highlighted that the local dominant flood generation processes could be varied for different flooding occurrences. These individual contributions from different processes are difficult for the RF model to predict based on the limited number of input attributes. Therefore, we should keep in mind that the uncertainties exist in determining the dominant mechanism for flooding occurrences with this method, though the overall prediction accuracy is high.

To address the issue of imbalanced data in our study sample, we also attempted to use a resampling method, namely the Synthetic Minority Oversampling TEchnique (SMOTE) algorithm for prediction. The SMOTE algorithm is designed to address the issue of class imbalance in machine learning models. It works by oversampling the minority class (which usually has fewer samples) by generating synthetic data points that are similar to the existing samples. This helps to balance the class distribution and therefore may improve the model performance. However, the results showed that after applying the SMOTE algorithm, the average precision, recall, F1-score, and accuracy values were 0.682, 0.685, 0.683, and 0.685, respectively. These values were slightly lower compared to the corresponding values (0.692, 0.701, 0.692, and 0.701, respectively) in the experiments with the original data. Although the SMOTE algorithm slightly improved the predictions for the P and N classes, it resulted in a significant decrease in the prediction accuracy for the M class. For instance, the F1-score dropped from 0.773 to 0.753. In this particular study, the decrease in accuracy after applying the SMOTE algorithm could be due to several reasons. First, the synthetic data generated by SMOTE may not have captured the underlying patterns and relationships between the input features and the target variable accurately. This can result in overfitting or underfitting of the model, leading to a drop in the accuracy. Second, while the SMOTE algorithm can increase the number of samples in the minority class, it does not necessarily guarantee that these samples are the representatives of the true distribution of the class. As a result, the synthetic data points may introduce noise into the model, leading to a lower prediction accuracy. In such cases, oversampling the minority class can actually harm the overall performance of the model. Therefore, it is essential to carefully evaluate the impact of SMOTE on the dataset and model performance before deciding whether to use it or not.

While our study yielded promising results, it is important to note that the generalizability of our findings to other datasets may be limited. As with any machine learning model, the performance of our methodology may be impacted by the specific features and characteristics of the dataset used. Therefore, it is possible that adjustments to the methodology may be necessary to achieve comparable results on different data sources. To address this potential limitation, future research could focus on the generalizability of our methodology across a wide variety of datasets. This would involve replicating our methodology on different data sources and conducting comprehensive sensitivity analyses to evaluate the robustness of the results. Such efforts could provide valuable insights into the performance and generalizability of our methodology in different contexts. Moreover, due to the relatively coarse spatial resolution, topographical effects are not considered. The contributions from watershed features such as shape and slope are ignored in the implementation. Furthermore, the flood may be driven by mechanisms other than the three drivers investigated in this study. For example, long-term rainfall, glacial outburst (Björnsson 2003) and water resources management activities (such as reservoir operation and irrigation) can also affect flood generation. In addition, flood mechanisms in the same locality may differ depending on the specific events that occur (Bennett et al. 2018; Pendergrass 2018), which is not considered in this study.

This study provides a framework of using machine learning models to identify flood-generating mechanisms. We use circular statistics to estimate the seasonality of extreme precipitation, soil moisture excess and snowmelt. Three dominant flood types (P type, M type, and N type) and eight sub-types are identified. RF models are trained to predict the flood types. This global-scale study indicates that soil moisture excess is the most important driver of annual maximum flooding followed by extreme precipitation and snowmelt. During 1979–2013, nearly 44% of the regions have a late trend in annual flood timing with a mean value of 7 days/decade, while 51% of the regions show an earlier trend with a mean value of 6 days/decade. The RF model can identify the dominant driver of flood generation with an overall F1-score value of 0.692 and accuracy of 0.701. The approach presented in this study provides a means to extend the flood type mapping to other regions at a finer spatial scale where discharge data may not be available, thereby enhancing our understanding of the underlying processes driving flood generation.

This research is funded by the National Natural Science Foundation of China (Nos 52079151 and 52111540261).

All relevant data are included in the paper or its Supplementary Information.

The authors declare there is no conflict.