## Abstract

Floods and their associated impacts are topics of concern in land development planning and management, which call for efficient flood forecasting and warning systems. The performance of flood warning systems is affected by uncertainty in water level forecasts, which is due to their inability to measure or calculate a modeled value accurately. Predictive uncertainty is an emerging type of uncertainty modeling technique that emphasizes total uncertainty quantified as a probability distribution conditioned on all available knowledge. Predictive uncertainty analysis was done using quantile regression (QR) for machine learning-based flood models – Hybrid Wavelet Artificial Neural Network model (WANN) and Hybrid Wavelet Support Vector Machine model (WSVM) for different lead times. Comparing QR models of WANN and WSVM revealed that the slope, intercept, spread of forecast, and width of confidence band of the WANN model are more for each quantile indicating more uncertainty as compared to the WSVM model. In both models, with an increase in lead time, uncertainty has shown an increasing trend as well. The performance evaluation of inference obtained from QR models was evaluated using uncertainty statistics such as prediction interval coverage probability, average relative interval length (ARIL), and mean prediction interval (MPI).

## HIGHLIGHTS

Predictive uncertainty using quantile regression on two hybrid flood models, i.e. WANN and WSVM, is implemented.

Performance evaluation of quantile regression models is done using certain statistical methods such as average relative interval length and mean prediction interval.

The WSVM model is less uncertain than WANN.

With an increase in lead time, the confidence bandwidth is increased, which shows more uncertainty.

## INTRODUCTION

Floods are one of the most frequent and devastating natural calamities that occur every year in various parts of the world. The flood events and their unfortunate consequences are increasing markedly over the last decades. Flood forecasting is an important measure to reduce risks and vulnerabilities associated with floods. Using accurate and timely hydrological and meteorological forecasts, various national disaster agencies are taking measures to cope with flood events. To facilitate appropriate evacuation plans, flood prevention, and rehabilitation measures, flood forecasting is to be done hourly with sufficient lead time. Hydrological processes control water movements in a hydrological cycle (e.g., groundwater recharge, river flow, precipitation, and evapotranspiration). It is highly stochastic and complex in nature, which is conventionally resolved using hydrological time-series analysis and flood forecasting methods. These complexities, non-linearities, and stochastic behavior among different components in the hydrological process will affect the reliability of physically based models and conventional statistical methods, thus paving the way for different sources of uncertainty. For real-life flood forecasting systems, hydrological forecasts are inevitably very uncertain (Beven & Binley 1992; Gupta *et al.* 1998; Refsgaard *et al.* 2007).

Real-time-based flood forecasting and warning systems are intended to give responsible authorities, property owners, and residents near floodplains response time for a flood threat before the violation of critical threshold as a mitigation measure for reducing further consequences. Real-time flood forecasting systems are currently operational in many major parts of the world, including England and Wales where the National Flood Forecasting System (NFFS) is used by the Environment Agency (Werner *et al.* 2009). For controlling the flood risk and flood damages, nonstructural flood risk management (e.g. lifting of an existing structure to a higher elevation, relocation, filling the basement with the addition of main floor, acquisition, wet floodproofing, dry floodproofing, land use regulations, flood emergency preparedness plans, and flood warning system) is used, which demands a reliable assessment of predicted events and its certainty analysis. The performance of flood management systems is mainly affected by uncertainties in different predictands such as discharge, water level forecast, etc., that results in reducing the potential in eradicating flood risk, and also, releasing unwarranted action calls by institutional decision-makers and false alarms issued by safeguarding operational users. Rational decision-making (for flood navigation, warning, or reservoir systems) requires that the total uncertainty about a hydrological predictand (such as runoff volume, river stage, or discharge) is quantified in terms of a probability distribution, conditional on all available knowledge and information, and that hydrological knowledge is typically embodied in a deterministic catchment model (Krzysztofowicz 1999). Uncertainty results from the lack of knowledge or the inability to accurately calculate or measure a modeled value, which can lead to differences between the modeled and true values of a variable (Gouldby & Samuels 2009). Some sources of uncertainty include inherent uncertainty in observation and measurement, meteorological forecast, and initial and boundary conditions such as uncertainty in the future boundary conditions of temperature, evaporation, and precipitation, which can be from numerical weather prediction models, model schematizations, parameter uncertainty, and model structure uncertainty. To reduce these sources of uncertainty, a wide range of research has been performed over the past decades. Predictive uncertainty is the probability of any future (real) value, conditional upon all information and knowledge, available up to the present. Predictive uncertainty emphasizes that it is the uncertainty around the prediction, which is being described or quantified rather than validation uncertainty or model uncertainty. Predictive uncertainty is the expression of a subjective assessment of the probability of occurrence of a future (real) event that is conditional upon all knowledge available up to the present (the prior knowledge) and the information that can be acquired through a learning inferential process (Rougier 2007). Probability forecasts can then be used to take a risk-based decision, where the consequences of possible outcomes can be weighted by their probability of occurrence function (Raiffa & Schlaifer 2000; Todini 2007). Also, depending on these consequences, decision-makers can set a threshold of probability against which to decide, thus choosing an appropriate balance between false alarms and missed floods.

Predictive uncertainty emphasizes such a total uncertainty, which is quantified as a probability distribution that is conditional on all available knowledge and information, up to the present (Todini 2008). Predictive uncertainty considers uncertainty related to predictions that are quantified or described rather than model uncertainty, parameter uncertainty, or validation uncertainty, which are defined as the ability of a model to reproduce reality (Todini 2008; Klein *et al.* 2016). Research studies done on predictive uncertainty estimation suggests various benefits for end users (Krzysztofowicz 2001; Collier *et al.* 2005; Verkade & Werner 2011; Ramos *et al.* 2013; Dale *et al.* 2014). A major benefit of the concept of predictive uncertainty is the radial change of model forecast from a deterministic threshold paradigm to a probabilistic threshold paradigm, which is defined as the ‘probability of flooding’ in terms of different probability levels (10, 30, 50%, etc.) rather than deterministic threshold values. Therefore, the critical threshold will not be based on different water levels (warning level, alert level, and flooding level) but on various flooding probabilities. In this study, the main motivation is to estimate uncertainty in flood forecasting models using quantile regression (QR). Flood forecasting models adopted for assessing predictive uncertainty are the Hybrid Wavelet-ANN model (WANN) (Alexander *et al.* 2018) and the Hybrid Wavelet-SVM model (WSVM) (Shada *et al.* 2022), which were developed in the Achankovil River basin in Kerala for different lead times – 1, 3, and 6 h using QR. QR (Koenker & Bassett 1978; Koenker & Hallock 2001; Koenker 2005) describes a full probability distribution of the variable that is the predictand, which is conditional on one or more predictors. Coccia & Todini (2011) observe that the usefulness and performance of QRs depend on the assumed patterns in quantiles; for example, lack of linear variation of the error variance with the magnitude of the forecasts hinders reasonable estimation of the quantiles, especially for high flows/water levels. López López *et al.* (2014) applied QR to predict the quantiles of environmental variables (water level) rather than the quantiles of the model error, and the four different configurations of QR are compared and extensively verified. The performance of QR models was evaluated using various statistical measures. These are, namely, prediction interval coverage probability (PICP) (Shrestha & Solomatine 2006), mean prediction interval (MPI) (Shrestha & Solomatine 2006), and average relative interval length (ARIL) (Jin *et al.* 2010). PICP has also been used by other authors (Laio & Tamea 2007) as an important performance measure to estimate the accuracy of probabilistic forecasts. The main aspect of this study is to assess the uncertainty in terms of probability density, from which uncertainty bands can be created, which showcase the deviation of predictions of discharge, water stage, etc., from the observed values. Uncertainty can also be expressed in terms of the probability distribution function or probability density, which is conditional upon forecasting model predictions.

## METHODOLOGY

*et al.*2018) and the Hybrid WSVM model (Shada

*et al.*2022) for the identified flood events. To evaluate predictive uncertainty, QR was adopted. QR (Koenker & Bassett 1978; Koenker & Hallock 2001; Koenker 2005) is a regression technique for estimating the quantiles of a conditional distribution. Performance evaluation of uncertainty estimation techniques is carried out using several statistical measures such as PICP, MPI, and ARIL. Detailed methodology is explained in the below sections.

### Predictive uncertainty

Predictive uncertainty emphasizes such a total uncertainty, which is quantified as a probability distribution that is conditional on all available knowledge and information. Predictive uncertainty considers uncertainty related to predictions that are quantified or described rather than model uncertainty, parameter uncertainty, or validation uncertainty. Predictive uncertainty is defined as the probability of occurrence of a future value of a predictand (such as water level, discharge, or water volume) that is conditional on prior observations and knowledge, as well as on all information one can obtain on that specific future value, which is typically embodied in one or more hydrological/hydraulic model forecasts (Krzysztofowicz 1999).

#### Quantile regression

*Q*is the number of quantiles

*τ*(

*τ*ɛ[0,1]). Here, we consider

*Q*= 20 and

*τ*ɛ {0.05,0.1,0.15,…,0.95} and φ

_{t}denotes continuous distribution.

*t*considered and for every quantile

*τ*, there is a linear relationship between observation O and water level forecast P.where a

_{t,c}and b

_{t,c}are the slope and intercept of this regression. These parameters can be found by the process of linear programming by minimizing linear regression and by finding the sum of residuals:where o

_{i}and p

_{j}are the jth sample pairs from j samples. ρ is the quantile regression function for

*τ*th quantile.where

*ɛ*

_{t,j}is the difference between observation o

_{t,c}and QR estimation (a

_{t,c}P

_{t}+ b

_{t,c}).

This technique can be described for the complete conditional distribution of dependent variable *O* for each quantile *τ*. Quantreg package (Koenker 2013) in R programming software (R Core Team 2013) is used to carry out quantile regression mainly in Equation (4). A graphical overview of the selected quantiles and these plots are being analyzed in the Results and Discussion section.

### Performance methods

There are several statistical measures of uncertainty to evaluate the performance of quantile regression for different flood models. Some of them are given below.

#### Prediction interval coverage probability

*α*. It is an important measure of uncertainty performance as it indicates several observations falling in between prediction intervals.where

#### Mean prediction interval

#### Average relative interval length

_{.}

## STUDY AREA AND DATA SOURCES

^{2}, and the river is 128 km long (Prasad & Ramanathan 2005; Dhanya 2014). The Western Ghats form the eastern boundary of the basin, while the Arabian Sea forms the western boundary. Like all river basins in Kerala, the Achankovil basin can also be divided into three physiographic zones based on elevation, namely the lowlands, midlands, and highlands. The study area lies upstream and is subjected to flash floods during the rainy season.

### Data sets and sources

Data sets and their sources are summarized in Table 1.

S. No. . | Data . | Source . | Types . |
---|---|---|---|

1 | Rainfall | Dept. of Water Resources, Govt. of Kerala | Hourly rainfall data from Jan. 2011 to Dec. 2015 |

2 | Water levels (stage) | Dept. of Water Resources, Govt. of Kerala | Hourly water level data from Jan. 2011 to Dec. 2015 |

3 | Land use map | Land Use Board | Land use |

4 | Soil map | Soil Survey & Soil Conservation Department | Soil map |

5 | Satellite images | USGS Earth Explorer website | SRTM DEM resolution – 30 m |

S. No. . | Data . | Source . | Types . |
---|---|---|---|

1 | Rainfall | Dept. of Water Resources, Govt. of Kerala | Hourly rainfall data from Jan. 2011 to Dec. 2015 |

2 | Water levels (stage) | Dept. of Water Resources, Govt. of Kerala | Hourly water level data from Jan. 2011 to Dec. 2015 |

3 | Land use map | Land Use Board | Land use |

4 | Soil map | Soil Survey & Soil Conservation Department | Soil map |

5 | Satellite images | USGS Earth Explorer website | SRTM DEM resolution – 30 m |

## RESULTS AND DISCUSSIONS

### Input data

The water level was considered as a predictand for obtaining an uncertainty model. The hourly water level at the Konni gauging station from the period 2011 to 2015 was mainly used as the observed water level. The forecasted water level corresponding to different observed water levels was obtained from the developed Hybrid WANN (Alexander *et al.* 2018) and Hybrid WSVM (Shada *et al.* 2022) models.

### Flood event identification

After a primary analysis of water level data and model outputs, 19 flood events (E1–E19) were identified for the period from 2011 to 2015. The flood events were distinguished by fixing a threshold of 2 m in terms of river stage, which is based on the overflow characteristics of the river bank. Here, a flood is defined as one that starts with an increase in surface runoff, reaching a peak and its attenuation (Alexander *et al.* 2018).

As a comparison of the uncertainty of the two models, Hybrid WANN and Hybrid WSVM models are made of four validated flood events, which are common for the two developed models (Alexander *et al.* 2018) & (Shada *et al.* 2022) that were considered. Details of the flood events considered are represented in Table 2.

Events . | Start . | End . | Water level (m) . | |||
---|---|---|---|---|---|---|

Date . | Time (h) . | Date . | Time (h) . | Mean . | Max . | |

E5 | 12/11/2015 | 1 | 17/11/2015 | 15 | 2.21 | 2.75 |

E12 | 22/7/2013 | 15 | 27/7/2013 | 16 | 2.42 | 2.80 |

E13 | 4/8/2013 | 11 | 7/8/2013 | 18 | 3.17 | 4.36 |

E15 | 19/10/2013 | 9 | 21/10/2013 | 5 | 2.68 | 3.71 |

Events . | Start . | End . | Water level (m) . | |||
---|---|---|---|---|---|---|

Date . | Time (h) . | Date . | Time (h) . | Mean . | Max . | |

E5 | 12/11/2015 | 1 | 17/11/2015 | 15 | 2.21 | 2.75 |

E12 | 22/7/2013 | 15 | 27/7/2013 | 16 | 2.42 | 2.80 |

E13 | 4/8/2013 | 11 | 7/8/2013 | 18 | 3.17 | 4.36 |

E15 | 19/10/2013 | 9 | 21/10/2013 | 5 | 2.68 | 3.71 |

### Hybrid machine learning models

Hybrid WANN (Alexander *et al.* 2018) and Hybrid WSVM (Shada *et al.* 2022) were developed in the Achankovil River basin in Kerala for hourly flood forecasting for different lead times −1, 3, and 6 h. Both models performed satisfactorily and were assessed using standard performance rating criteria. The evaluation of model performance was done using several statistical techniques such as percentage deviation in peak flow (Dev), the coefficient of determination (*R*^{2}), Nash–Sutcliffe coefficient (NSC), root-mean-square error (RMSE), and the time difference to peak flow (Dep), which yield the quantitative assessment of the predictive ability of the model. Based on previous studies (Alexander *et al.* 2018; Shada *et al.* 2022), it was observed that WSVM performed better in flood prediction and more accurately estimated peak discharge magnitude and time of peak than WANN. From the performance measurement in Table 3 and flood hydrographs of both hybrid models, it is evident that both models perform efficiently in comparison to observed data, but the uniqueness and comparison among the models are not properly evident.

Performance measures . | Lead times . | |||||
---|---|---|---|---|---|---|

1 h . | 3 h . | 6 h . | ||||

WSVM . | WANN . | WSVM . | WANN . | WSVM . | WANN . | |

E5 | ||||||

RMSE (m) | 0.02 | 0.03 | 0.05 | 0.04 | 0.09 | 0.12 |

R^{2} | 0.99 | 0.98 | 0.92 | 0.97 | 0.76 | 0.60 |

NSC | 0.99 | 0.97 | 0.92 | 0.96 | 0.76 | 0.54 |

Dev (%) | −0.81 | 1.82 | 1.57 | 1.81 | 0.6 | −0.38 |

Dep (h) | 1 | 0 | 2 | 0 | 5 | 5 |

E12 | ||||||

RMSE (m) | 0.02 | 0.04 | 0.04 | 0.06 | 0.07 | 0.09 |

R^{2} | 1.00 | 0.97 | 0.98 | 0.96 | 0.93 | 0.86 |

NSC | 1.00 | 0.97 | 0.98 | 0.94 | 0.92 | 0.86 |

Dev (%) | −0.22 | 0 | −0.81 | 0.73 | 1.55 | 1.87 |

Dep (h) | 0 | 0 | 2 | 0 | 8 | 0 |

E13 | ||||||

RMSE (m) | 0.24 | 0.12 | 0.45 | 0.18 | 0.59 | 0.5 |

R^{2} | 0.93 | 0.98 | 0.76 | 0.9 | 0.55 | 0.6 |

NSC | 0.93 | 0.97 | 0.73 | 0.94 | 0.47 | 0.46 |

Dev (%) | 0.17 | −4.05 | 5.35 | −6.97 | 8.27 | −13.80 |

Dep (h) | 0 | 1 | −2 | 1 | 0 | 0 |

E15 | ||||||

RMSE (m) | 0.07 | 0.15 | 0.22 | 0.15 | 0.38 | 0.37 |

R^{2} | 0.99 | 0.98 | 0.87 | 0.93 | 0.59 | 0.59 |

NSC | 0.99 | 0.97 | 0.86 | 0.93 | 0.56 | 0.59 |

Dev (%) | 0.95 | −1.76 | 2.69 | −4.85 | 0.66 | 6.46 |

Performance measures . | Lead times . | |||||
---|---|---|---|---|---|---|

1 h . | 3 h . | 6 h . | ||||

WSVM . | WANN . | WSVM . | WANN . | WSVM . | WANN . | |

E5 | ||||||

RMSE (m) | 0.02 | 0.03 | 0.05 | 0.04 | 0.09 | 0.12 |

R^{2} | 0.99 | 0.98 | 0.92 | 0.97 | 0.76 | 0.60 |

NSC | 0.99 | 0.97 | 0.92 | 0.96 | 0.76 | 0.54 |

Dev (%) | −0.81 | 1.82 | 1.57 | 1.81 | 0.6 | −0.38 |

Dep (h) | 1 | 0 | 2 | 0 | 5 | 5 |

E12 | ||||||

RMSE (m) | 0.02 | 0.04 | 0.04 | 0.06 | 0.07 | 0.09 |

R^{2} | 1.00 | 0.97 | 0.98 | 0.96 | 0.93 | 0.86 |

NSC | 1.00 | 0.97 | 0.98 | 0.94 | 0.92 | 0.86 |

Dev (%) | −0.22 | 0 | −0.81 | 0.73 | 1.55 | 1.87 |

Dep (h) | 0 | 0 | 2 | 0 | 8 | 0 |

E13 | ||||||

RMSE (m) | 0.24 | 0.12 | 0.45 | 0.18 | 0.59 | 0.5 |

R^{2} | 0.93 | 0.98 | 0.76 | 0.9 | 0.55 | 0.6 |

NSC | 0.93 | 0.97 | 0.73 | 0.94 | 0.47 | 0.46 |

Dev (%) | 0.17 | −4.05 | 5.35 | −6.97 | 8.27 | −13.80 |

Dep (h) | 0 | 1 | −2 | 1 | 0 | 0 |

E15 | ||||||

RMSE (m) | 0.07 | 0.15 | 0.22 | 0.15 | 0.38 | 0.37 |

R^{2} | 0.99 | 0.98 | 0.87 | 0.93 | 0.59 | 0.59 |

NSC | 0.99 | 0.97 | 0.86 | 0.93 | 0.56 | 0.59 |

Dev (%) | 0.95 | −1.76 | 2.69 | −4.85 | 0.66 | 6.46 |

It is difficult to give inference regarding which model is better than the other due to similar standard performance values, and thus, a need for a detailed uncertainty study is required in those aspects. QR is mainly incorporated to estimate conditional distribution quantiles, which is a rather conditional mean, while considering outliers.

### QR uncertainty models

*τ*∈ {0.10,0.50,0.90}. The selected quantiles are denoted in the graph as the 50th quantile (Least Absolute Deviations (LAD) – blue color), 10th quantile (bottom line – green color), and 90th quantile (upper line – red color). QR models for different lead times and flood events E5 and E13 for WANN and WSVM are shown in graphs, in Figures 4 and 5. Similar results were seen for other events also. In all the scatter plots, a strong correlation between the observed and forecasted water levels was evident. Comparing QR models of WANN and WSVM, in most cases, the slope, intercept, and spread of forecast of the WANN model are more for each quantile, indicating more uncertainty compared to the WSVM model. For both WANN and WSVM models, with an increase in lead time, an increase in uncertainty is observed, which is evident with the spread of prediction interval width. Analyzing the slope and location of quantiles indicates that in most cases, the spread is small at a low predicted value and increases with an increasing value of the forecast.

### Hydrographs

### Performance evaluation using uncertainty statistics

Flood event . | Models . | Lead time (h) . | Uncertainty statistics . | ||
---|---|---|---|---|---|

PICP . | MPI . | ARIL . | |||

Event 5 | Wavelet-ANN Model | 1 h | 1 | 0.249 | 0.106 |

3 h | 1 | 0.344 | 0.146 | ||

6 h | 1 | 0.719 | 0.309 | ||

Wavelet-SVM Model | 1 h | 1 | 0.167 | 0.074 | |

3 h | 1 | 0.278 | 0.114 | ||

6 h | 1 | 0.559 | 0.228 |

Flood event . | Models . | Lead time (h) . | Uncertainty statistics . | ||
---|---|---|---|---|---|

PICP . | MPI . | ARIL . | |||

Event 5 | Wavelet-ANN Model | 1 h | 1 | 0.249 | 0.106 |

3 h | 1 | 0.344 | 0.146 | ||

6 h | 1 | 0.719 | 0.309 | ||

Wavelet-SVM Model | 1 h | 1 | 0.167 | 0.074 | |

3 h | 1 | 0.278 | 0.114 | ||

6 h | 1 | 0.559 | 0.228 |

## CONCLUSIONS

QR models of Hybrid WANN and Hybrid WSVM for different lead times were done. The inference obtained from QR models was validated statistically using uncertainty statistics such as PICP, MPI, and ARIL, and a similar conclusion was obtained.

There was a strong correlation between observed and forecasted water levels. Comparing QR models of WANN and WSVM, in most cases, the slope, intercept, and spread of forecast of the WANN model are more for each quantile, indicating more uncertainty compared to the WSVM model. In both models, with an increase in lead time, the spread of the forecast width of prediction is increased, which shows more uncertainty with the highest lead time. Analyzing the slope and location of quantiles indicates that in most cases, the spread is small at a low predicted value and increases with an increasing value of the forecast. The confidence bandwidth for the WANN model is wider than the WSVM model, which indicates an increase in uncertainty. In both models, with an increase in lead time, the standard error shows an increasing value, and thus, the confidence interval and uncertainty also show an increase.

To evaluate and compare the performance of QR models, certain validation methods such as PICP, MPI, and ARIL were done. As all the observation pairs were within the prediction limits, PICP was 1 for all cases. So, further analyses using MPI and ARIL were required. MPI and ARIL results showed a similar variation as obtained from QR models. The MPI of individual models shows an increase with the increase in lead time, and thus, an increase in uncertainty is evident. MPI and ARIL values were less for the WSVM model for most of the flood events and the corresponding lead time when compared to the WANN model, which showed that the WSVM model is less uncertain compared to the WANN model. While calculating the percentage increase of MPI and ARIL values of the uncertainty models of WANN and WSVM models, it shows an increase of 30–50% more uncertainty for the WANN model than the WSVM model, with the percentage getting reduced with the increase in lead time.

It can be concluded that the WSVM model is less uncertain and has a better performance than the WANN model, based on reliability and accuracy.

## DATA AVAILABILITY STATEMENT

Data cannot be made publicly available; readers should contact the corresponding author for details.

## CONFLICT OF INTEREST

The authors declare there is no conflict.