Flash flood susceptibility mapping based on catchments using an improved Blending machine learning approach

Flash floods are among the most catastrophic hazards that cause extensive damage and disruption to the environment and society (Khajehei et al. 2020). In China, flash floods have caused an average of 356 casualties between 2011 and 2020, representing approximately 60% of flood mortality (Ministry of Water Resources of China (MWR) 2021; Guo et al. 2018a). Flash flood susceptibility mapping is a challenging task due to the complex factors that affect flash flood generation, including catchment properties and rainfall characteristics (Rozalis et al. 2010). The Sendai Framework for Disaster Risk Reduction, signed in 2015 under the leadership of the United Nations Office for Disaster Risk Reduction (UNDRR), prioritizes the strengthening of disaster risk governance to manage disaster risk (UNDRR 2015). One way to strengthen flash flood risk management is to assess and map flash flood susceptibility (Youssef et al. 2016; Ha et al. 2021). The Chinese government has significantly increased investment and attention to flash flood prevention and control. The MWR has conducted the National Flash Flood Disasters Investigation and Evaluation project, which covered 30 provinces, 305 cities, and 2,138 counties. The project has established a database of investigation and evaluation results for national flash flood disasters to provide a solid data foundation for flash flood monitoring and warning systems, disaster management, and mitigation research (Guo et al. 2017).

To produce accurate flash flood susceptibility maps using machine learning methods, reliable historical flash flood points and factors are required (Costache et al. 2020a; Hosseini et al. 2020). Historical flash flood points are usually derived from the survey records of local authorities (Chen et al. 2020). The conditioning factors for flash floods are complex, and there is no agreement on how to apply these factors accurately (Rozalis et al. 2010; Chen et al. 2019). The conditioning factors can be classified into three categories: geometric characteristics (GC), environmental characteristics (EC), and hydrological characteristics (HC). The factors and their corresponding references are shown in Table 1. Among the GC, the topography is closely related to flood susceptibility, mainly due to the elevation and slope (Zhou et al. 2000; Ragettli et al. 2017). Catchment area, catchment flow channel length, and their ratio are considered to have a strong correlation with flash flood hazards in catchments (Fan et al. 2012). EC includes vegetation, soils, and rivers, quantified accordingly as the normalized difference vegetation index (NDVI), topographic wetness index (TWI), and the density of the river network or the distance to rivers (Soulsby et al. 2010; Huang et al. 2012; Reager et al. 2014; Miao et al. 2016; Zhao et al. 2016; Liu et al. 2019). Short-duration heavy rainfall is usually regarded as the direct cause of flash floods, and the related HC (Peak discharges per unit area, curve number, and time of concentration) also have a remarkable influence (Guo et al. 2018b).

Several methods have been utilized to identify areas susceptible to flash floods. Hydrological models have been developed to predict flash flood susceptibility or rainfall thresholds for flash floods (Miao et al. 2016; Nguyen et al. 2016). However, these models require substantial data inputs that are often difficult to obtain (Rozalis et al. 2010; Hapuarachchi et al. 2011). Studies can only be carried out for a single catchment or watershed, and it is difficult to model a larger area (Zhang et al. 2021). The GIS-based spatial analysis approaches have also been used to investigate flash flood susceptibility. These approaches discriminate the susceptibility of different regions by analyzing the spatial heterogeneity and geographical similarity of the conditioning factors (Xiong et al. 2020). These approaches have performed well in a single catchment or watershed, but their performance in large study areas remains a topic of discussion (Abdelkareem 2017; Abdo 2020).

With the development of machine learning, an increasing number of studies apply machine learning to geography and GIS (Gao 2020; Janowicz et al. 2020). Researchers have found that machine learning performs well in solving nonlinear problems, feature selection, and data mining (Liu et al. 2022). A variety of machine learning approaches are widely used to assess flash flood susceptibility, including decision tree (DT) (Tehrany et al. 2013), SVM (Tehrany et al. 2014), KNN (Costache et al. 2019), etc. Cao et al. (2020) incorporated the Pearson correlation coefficient and Geodetector into LR to filter the evaluation indicators and finally map the susceptibility to flash floods in Fujian Province, China (Cao et al. 2020). Chiang et al. (2007) employed the recurrent neural network (RNN) model to study flash floods by combining multiple precipitation data sources (Chiang et al. 2007). Fang et al. (2021) added local spatial information of grid cells to long short-term memory (LSTM) to take advantage of the sequential model to process attribute information and spatial relationships of flash floods.

The performance of a single model is always limited. To improve predictive power, researchers attempt to combine these single models in hybrid and ensemble models. Hybrid ideas are usually divided into two categories.

The first is the hybrid model, which combines completely different, heterogeneous machine learning approaches. Primary prediction models, such as neural networks, including adaptive neuro-fuzzy inference systems, are combined with stochastic optimization algorithms to optimize hyperparameters of neural networks. The resulting hybrid models outperform benchmark models in both convergence speed and prediction results (Bui et al. 2016; Hong et al. 2018; Termeh et al. 2018). Costache et al. (2020b) used the bivariate statistical method to process the data of the input model in advance, then inputted the statistical index as a new predictive variable into the machine learning model (Costache et al. 2020a, 2020b). Although the final predictions are still output by a single model, other approaches are combined into the entire experiment process, which is different from the pure single machine learning models.

The other category is ensemble model, also known as ensemble learning, which consists of homogeneous machine learning approaches, called base learners. Single or multiple base learners are combined into ensemble models according to different strategies to reduce the impact of the bias of base learners and improve predictive power (Choubin et al. 2019). Bagging, boosting, and stacking are the three most commonly used strategies. Bagging has been frequently used in flash flood susceptibility mapping (Chapi et al. 2017; Bui et al. 2019; Chen et al. 2019; Arabameri et al. 2020; Ha et al. 2021). In addition, boosting approaches, including Adaptive Boosting (AdaBoost) (Pham et al. 2020; Ha et al. 2021), Gradient Boosting Decision Tree (GBDT) (Chen et al. 2021), and eXtreme Gradient Boosting (XGBoost) (Chen et al. 2021; Ma et al. 2021), have been shown to perform well in flash flood susceptibility assessment. Stacking and blending have been used in flash flood susceptibility mapping for their superiority, as demonstrated by Yao et al. (2022), but there are relatively few studies associated with it (Yao et al. 2022).

In this study, the Blending approach and an RF-Blending approach are selected as the main approaches of this study. The study is carried out for all the catchments in Jiangxi Province, China. Catchment is used as the unit of study since it may be better to use catchments to map flash flood susceptibility in large areas (Ragettli et al. 2017). SVM, KNN, RF, and LR have all demonstrated good performance in predicting flash flood susceptibility (Tehrany et al. 2014; Costache et al. 2020b; Rahman et al. 2021). They also outperformed most methods in our previous experiments. Since Yao et al. (2022) have used the stacking and blending approach for flash flood mapping in Jiangxi Province and SVM, KNN, and RF were used as base learners and linear regression was used as a meta-learner in their study (Yao et al. 2022). For this reason, SVM, KNN, and RF are selected as the base learners. Linear regression is not suitable as a meta-learner for RF-Blending since the meta-learner still needs to learn the features in the input dataset. A linear model that has performed well in the prediction of flash flood susceptibility is selected as the meta-learner, which is LR.

This paper aims to perform flash flood susceptibility mapping in Jiangxi Province, China, using a novel GIS-based approach combined with the RF-Blending model. Firstly, a literature review of the factors and approaches for flash floods is introduced and used as the basis for the selection of factors and methods for this study. Secondly, the study area and data employed are presented. The RF-Blending approach, the Blending approach, and the benchmark models, as well as the metrics for model performance, are then explained. Finally, the results of the model and flash flood susceptibility maps are presented. The differences in model performance, the distribution of different levels of susceptibility, and the limitations of this study are discussed. The final section summarizes the content and highlights the main finding, which is that the RF-Blending has a good performance in flash flood susceptibility assessment and mapping.

The south and northwest of Jiangxi Province are mostly hilly and at higher altitudes, while the north is relatively flatter and at lower altitudes. The vegetation cover is high in Jiangxi Province, with significantly more vegetation cover in the south than in the north, and less vegetation in the basins such as Poyang Lake. There are more than 2,400 rivers in the province, with a total length of 18,000 km, most of which converge to the Poyang Lake. The five main rivers are Ganjiang River, Xinjiang River, Fuhe River, Xiuhe River, and Raohe River.

Jiangxi belongs to the subtropical monsoon climate zone, with four distinct seasons, a humid and hot summer and a cool winter. The annual average temperature is 19 °C and the average annual rainfall is 1,400–2,200 mm, with uneven distribution between seasons and regions. The northeast of the province receives the highest amount of rain, while the northwest receives the least. The frequency of short-duration intense rainfall in Jiangxi Province is higher in the east and lower in the west, with high-frequency areas distributed in Jingdezhen and Shangrao in the northeast, and low-frequency areas located in Yichun and western Ganzhou. Temporally, rainfall is mainly concentrated in the period from April to September each year, peaking in June, with a clear trend of increasing precipitation frequency year by year (Tang et al. 2018).

The conditioning factors are first selected and the catchment dataset is collected. Subsequently, the training and test sets are preprocessed for normalization and segmentation. The RF-Blending approach used in this paper consists of a three-level structure: the base learners (level-0), the construction of a meta-feature dataset (level-1), and the meta-learner (level-2). Based on the trial-and-error method and literature review, four models are used: RF, KNN, SVM, and LR, where LR is used as a meta-learner and the remains as base learners. The base learners are first fitted on the training set, and output the results from the training and test sets, respectively. Then the obtained predictions are used to form a meta-feature dataset along with the training and test sets, and finally, the meta-learner is fitted on the meta-feature dataset. Considering that only one set of training and test sets are used for model building and performance comparison, to explore the differences between the trained models when different training and test sets are used, and to verify the stability of the models, a ten-fold cross-validation (CV) approach is used to validate their performance and compare the model performance and generalization ability of the original Blending model, the RF-Blending model, and the four benchmark models. Finally, flash flood susceptibility is assessed and mapped for all catchments in the study area. The remainder of this section details the basic definition of each model, hyperparameter tuning, and model evaluation metrics.

The experimentation process, including model training, hyperparameter tuning, and performance evaluation, is implemented using Python and Scikit-learn (https://scikit-learn.org/stable/index.html). ArcGIS was used for data management, processing, and visualization.

The RF-Blending modeling process is as follows.

• 1.

  Train the three base learners with the training set individually.

• 2.

  Input the training set and test set into the trained three base learners to get the predictions.

• 3.

  Combine the predictions obtained in step 2 with the training and test sets to obtain the meta-feature data training set and test set.

• 4.

  Train the meta-learner with the meta-feature data training set.

• 5.

  Input the meta-feature data test set into the trained meta-learner to get the predictions and verify the model performance.

In this study, SVM, KNN, and RF were selected as the base learners at level-0, and LR was employed as the meta-learner.

• (1)

  Logistic regression

KNN is a classical nonparametric machine learning method that determines the category of the samples to be classified according to the categories of the K samples closest to them. However, not all classification problems have actual distance. In most cases, the similarity between samples is used instead of distance. When calculating sample similarity, there are some distance measures to choose from, including Minkowski distance, Euclidean distance, and Manhattan distance. In this study, the Minkowski distance was used. The performance of the KNN model mainly depends on the selection of the K value and distance measure.

• (3)

  Support vector machine

RF is an improvement of the tree-based bagging method in essence. It typically consists of a series of random decision trees. Firstly, the training data and features are selected in a put-back process and divided into multiple independent sub-datasets. These sub-datasets and the corresponding features are then input into the random trees for training. Finally, the predictions from all decision trees are integrated and the final prediction is calculated by voting or averaging. Unlike traditional decision trees, the RF does not overfit the training data since the random trees that comprise it are all independent of each other. Furthermore, it effectively avoids deviations, missing values, and chaotic inputs. The RF is often used as a benchmark model to evaluate the performance of flash flood susceptibility prediction results. For further details of the RF, refer to Costache et al. (2020a) and Breiman (2001).

Hyperparameters are a critical component of machine learning models and significantly affect model training and performance. Typically, a model contains several hyperparameters, and it is necessary to determine their values that achieve optimal performance on a given dataset before training the model. Hyperparameter optimization aims to find the best hyperparameters that achieve the optimal performance. In this study, Bayesian optimization is used to select hyperparameters. First, several points are randomly searched within a given range to fit a surrogate model. Then the best point is selected using the acquisition function, and this new data point is used to optimize the surrogate model. The above steps are repeated and the optimal hyperparameters are obtained after a certain number of iterations. The hyperparameter optimization is carried out using the test set to avoid overfitting. The hyperparameters are listed in Table 3.

Model evaluation is critical for measuring the performance of machine learning approaches. In this study, flash flood susceptibility prediction is solved as a binary classification problem, with two outputs: one representing the probability of a catchment belonging to a positive sample, which is used for flash flood susceptibility mapping, and the other representing whether a catchment belongs to a positive or negative sample, used for model performance evaluation. Classification models typically use accuracy, precision, recall, sensitivity, specificity, and F1-score to represent their performance. To more intuitively reflect the classification performance and not be affected by the threshold used for classification, the receiver operating characteristic (ROC) curve is also commonly used as a model evaluation criterion.

Equation (9) shows that TP is the number of true positive samples, TN is the number of true negative samples, P is the number of positive samples in the input data, and N is the number of negative samples.

In summary, RF-Blending outperforms the other models on the test set, with higher values for accuracy, F1-score, and AUC compared to both Blending and RF, and much higher values compared to the other three benchmarks. While the RF-Blending model is not the highest in terms of precision, specificity, and recall, it is still among the top-performing models. The cross-validation results demonstrate similar characteristics to the test set, with RF-Blending having the highest accuracy, F1-score, and AUC value, followed by Blending and RF. Although RF-Blending is slightly less stable than Blending, its standard deviation is still low and does not significantly impact the model's performance.

As shown in Figure 8(a), flash flood susceptibility is categorized into ten classes, where a higher susceptibility value indicates a higher risk of flash floods in the corresponding catchment. The map shows that areas with high flash flood susceptibility are mainly concentrated in the north, northeast, and southwest of Jiangxi Province, while catchments in the central, eastern, and southeastern parts of the province have a low susceptibility to flash floods. In most parts of Jiangxi Province, the high and low risk areas are distributed in a striated pattern. Comparing Figure 8(b), where flash flood susceptibility is classified into three classes, with Figure 8(c), where it is classified into two classes, there are few catchments with medium susceptibility. This is corroborated by Figure 8(a), where most of the catchments with low susceptibility have susceptibility values below 0.3 and most of the catchments with high susceptibility are greater than 0.9. Different from the catchments with high susceptibility, the catchments with low susceptibility do not exhibit significant clustering, with only a few of them concentrated in the central and eastern parts of Jiangxi Province. Most catchments with low susceptibility are distributed randomly throughout the study area, except in the northeast. In addition, the Poyang Lake has low flash flood susceptibility because it is mostly a lake area itself, but most of the catchments around the Poyang Lake, especially those in the southwest region, have high flash flood susceptibility.

The main objectives of this study are two: the first is to prove the superiority of the RF-Blending model incorporating the original dataset into the meta-feature dataset in assessing flash flood susceptibility, while the other is to assess and map the flash flood susceptibility in Jiangxi Province, China. Further discussion on model performance, the practical implications of the flash flood susceptibility map, limitations of this study, and possible future research directions are presented in this section.

The results show that the strengths and weaknesses of the models vary across the different evaluation metrics. The RF-Blending model demonstrated a clear advantage over the Blending model and the benchmark models in terms of accuracy, F1-score, and AUC (see Tables 3 and 4) on both the training and test set. The CV box plots confirmed this advantage, with the RF-Blending model exhibiting much higher metrics than KNN, SVM, and LR, and slightly higher metrics than Blending and RF. While the RF-Blending model was found to be less stable than the Blending model, it is important to note that the RF-Blending model's meta-feature dataset contains 13 features, while the Blending model only contains 3 features. Furthermore, the standard deviation of the RF-Blending model was smaller than that of the benchmark models. Therefore, the RF-Blending model used in this study outperformed the Blending model and the benchmark models.

The flash flood susceptibility maps and pie charts for Jiangxi Province indicated that most catchments had either very low or very high susceptibility to flash floods. The binary classification approach using 1 and 0 as outcome variables may be responsible for this result, as most catchments predicted to be in the true category had a susceptibility higher than 0.9. Further classification of these catchments is necessary to refine the assessment of their flash flood susceptibility.

The flash flood susceptibility map can be used as a reference for flash flood management in Jiangxi Province. For instance, Jingdezhen, Shangrao, the western part of Ganzhou, and the western part of Jiujiang were identified as highly susceptible to flash floods and thus, should be prioritized for flood management and prevention. The high-risk areas maintained a correlation with the distribution of rivers, and exploring the connection between them could reduce flash flood risk through better management of river channels.

The study acknowledges several limitations, such as the lack of highly accurate spatiotemporal historical flash flood records and the need to consider more catchment factors relevant to flash floods. Additionally, different combinations of machine learning models, including deep learning models, could be explored to improve the performance of the Blending model.

This study demonstrates that the RF-Blending model plays a positive role in assessing flash flood susceptibility in Jiangxi Province, China, outperforming the Blending model and benchmark models. The flash flood susceptibility maps show that about half of the catchments in Jiangxi Province are highly susceptible to flash floods, particularly in the north, northeast, and southwest regions (Jingdezhen, Shangrao, the western part of Jiujiang, and the western part of Ganzhou). These results can both serve as a reference for further research and flash flood management and prevention, ultimately contributing to reducing disaster mortality, the number of affected people, and the direct disaster economic loss in accordance with the Sendai framework.

This work was supported by the National Key R&D Program of China [grant No. 2019YFC1510601].

Data cannot be made publicly available; readers should contact the corresponding author for details.

The authors declare there is no conflict.