The UK standard for estimating flood frequencies is outlined by the flood estimation handbook (FEH) and associated updates. Estimates inevitably come with uncertainty due to sampling error as well as model and measurement error. Using resampling approaches adapted to the FEH methods, this paper quantifies the sampling uncertainty for single site, pooled (ungauged), enhanced single site (gauged pooling) and across catchment types. This study builds upon previous progress regarding easily applicable quantifications of FEH-based uncertainty estimation. Where these previous studies have provided simple analytical expressions for quantifying uncertainty for single site and ungauged design flow estimates, this study provides an easy-to-use method for quantifying uncertainty for enhanced single site estimates.
Bespoke bootstrap methods for quantifying uncertainty for ungauged and enhanced single site FEH design flow estimation.
Comparison of flood estimation uncertainty across catchment types.
Simple equations to derive variance and standard error for enhanced single site design flow estimates.
describe bootstrap methods for estimating fse for the FEH08 SS, UG and ESS cases;
use the methods of point one to apply and compare the uncertainty across these cases at sites considered suitable for pooling (National River Flow Archive);
compare the uncertainty across different catchment types; and
develop a simple expression for quantifying uncertainty for the ESS estimates.
The paper has four main parts: firstly, a summary of the data used in the study; secondly, a section describing the fse and the bootstrap methods applied to estimate it for the SS, UG and ESS cases. This method section also details the approach to compare fse across catchment types and the analytical expression for calculating fse in the ESS case. Thirdly, a summary of all the results is provided, and lastly, some concluding remarks.
STUDY AREA AND DATA
The annual maximum gauged flow data used for this study is from the National River Flow Archive (NRFA) peak flow dataset version 9. The NRFA collates, quality controls, and archives hydrometric data across the UK, including networks operated by the main UK Measuring Authorities; the Environment Agency (England), Natural Resources Wales, the Scottish Environment Protection Agency and for Northern Ireland, the Department for Infrastructure – Rivers. The locations of the 545 gauging stations used in the study are shown in Figure 1. They are all the sites in the NRFA that are considered suitable for pooling. Figure 2 provides a histogram of the AMAX sample sizes (record lengths in years) with some accompanying statistics. Table 1 provides a summary of the frequency curve parameters across all the sites used in the study.
QUANTIFYING UNCERTAINTY OF DESIGN FLOWS
BOOTSTRAPPING TO APPROXIMATE THE SAMPLING DISTRIBUTION
Bootstrapping, introduced by Efron (1979), and often applied in flood frequency analysis since (Zucchini & Adamson 1989; Faulkner & Jones 1999; Burn 2003; Hall et al. 2004; Burn et al. 2007; Bomers et al. 2019), is a class of resampling in which multiple additional samples of the same size are derived by drawing randomly with replacement from an original sample and estimating the statistic on each (or the model parameters), thus approximating the sampling distribution. Bootstrapping is used throughout this study as a non-parametric method for estimating uncertainty from an approximated sampling distribution. The fse can be calculated with bootstrapping as follows (example for the single site).
Randomly select from the sample with replacement N times (where N is the samples size)
Repeat step 1 M times to create M bootstrapped samples
Calculate the design flow for each of the M samples to approximate the sampling distribution of the design flow
Create log-transformed residuals by subtracting the log mean of the sampling distribution from each of the log-transformed design flows
- 5.The fse is then the exponent of the standard deviation of the log-transformed residuals (Equation (3)).
Faulkner & Jones (1999) and Burn (2003) applied a balanced resampling approach where, in place of steps 1 and 2 above, the AMAX series is repeated M times, permuted and split into M samples. The balanced resampling ensures that each value in the AMAX series appears an equal number of times in the union of resampled datasets. For this study, balanced resampling was compared with random resampling by calculating the single site QMED fse applying both methods to each of the 545 AMAX series twice (providing four distributions of fse). A Kruskal–Wallis test found no significant difference between the four fse samples (p-value = 0.86). To maintain intersite correlation within the pooled group, Burn (2003) also applied vector resampling, whereby each year across the AMAX series in a pooling group, is resampled together. Intersite dependence within a pooling group increases the uncertainty in comparison to the same pooling group with no dependence. This is because there is less information when deriving the L-moment ratios. Conversely, calculations of uncertainty would appear to decrease. To see why we can take an extreme example where every site in a pooling group is perfectly correlated. The L-CV and L-SKEW would be the same for each site (there would be less variance), but the sample size has remained the same. The standard error is a function of variance and sample size and is decreased by a reduction in variance or a larger sample size. Therefore, a pooling group with perfectly correlated sites provides no more confidence in our growth curve estimation than one of the sites alone, but an estimation of uncertainty would appear to decrease in comparison. Maintaining intersite dependence within the bootstrapping procedure underestimates the uncertainty for groups with dependence, as does non-vectorised bootstrapping. Fundamentally, intersite dependence undermines the benefits of RFA, hence Hosking & Wallis (1997, p. 8) list independence at different sites as an assumption of the index flood-based RFA. Thankfully, no bias is caused by a violation of this independence assumption and no two AMAX samples within a pooling group will be perfectly correlated. Therefore, the addition of a further site (assumed to have the same scaled distribution as the subject site) to a pooling group, will always provide useful information. However, it would be sensible to consider highly dependent catchments when forming the pooling group, because replacing a catchment with high dependence with an independent site will reduce the uncertainty. In this study, random resampling has been applied and the estimation of fse error assumes intersite independence.
THE METHOD FOR UNGAUGED CATCHMENTS
This approach to approximating the ungauged QMED estimate sampling distribution is used as part of the process for calculating fse for UG pooling analysis. For calculating the fse for a UG pooled estimate, the following steps were applied (where N equals 500):
Bootstrap each site of the pooling group individually to create N new samples of the same size
Create N new pooling groups from the bootstrapped samples of each site
Undertake a weighted (based on pooling group weightings) random selection of a single site from each of the N pooling groups and calculate the growth factor for each
Sample from the QMED sampling distribution (Equation (5)) N times
Multiply the results of step 4 by the results of step 3 to derive N estimates which approximates the sampling distribution
Apply Equation (3) to the results of step 5 to derive the fse.
Step 3 is of note because it is used rather than taking the weighted average growth curve for each pooling group. If this was done, due to the more homogenous weighting in the UG case (because there is no at-site data) there would be less variance in the growth curve estimate between the bootstrapped UG pooling groups than in the ESS case. We have more confidence when there is at-site data, hence the individual growth curves make up the sampling distribution in this method as opposed to the sampling distribution of the weighted mean growth curve. It is considered that this approach takes a better account of the wide range of possible growth curves that could be considered the ‘true growth curve’ (see Figure 4) in the UG case.
For the ungauged pooled estimates (of the 545 catchments detailed above – assumed ungauged), the default FEH08 pooling groups were used and if the site was urban, an urban adjustment was applied to the ungauged QMED estimate and growth curve (Wallingford HydroSolutions 2016). Where a donor or two donors were applied, the closest rural sites (Bayliss et al. 2006) were used. The single donor method was that of Environment Agency (2008) and the two-donor method was that of Kjeldsen (2019). The GLO distribution was used for estimating the growth curve (Equation (1)) with the L-moments method and UG weighted L-CV and L-SKEW (for more detail, see Environment Agency 2008).
METHOD FOR GAUGED CATCHMENTS
The approach for the ESS case is more straightforward and the following steps were applied (where N equals 500).
Bootstrap each site of the pooling group individually to create N new samples of the same size
Create N new pooling groups from the bootstrapped samples of each site
Estimate N growth curves, one for each pooling group using the ESS weighting method
Bootstrap the at-site sample N times and calculate the median for each sample to approximate the gauged QMED sampling distribution
Sample from the QMED sampling distribution N times
Multiply the results of step 5 by the results of step 3 to derive N design flow estimates
Apply Equation (3) to the results of step 6 to derive the fse.
For the pooled ESS estimates (of the 545 catchments detailed above), the default FEH08 pooling groups were used. If the gauged catchment was urban, it was included in the pooling group and deurbanised before an urban adjustment was made to the final growth curve (Wallingford HydroSolutions 2016). The GLO distribution was used for estimating the growth curve (Equation (1)) with the L-moments method and ESS weighted L-CV and L-SKEW (for more detail, see Environment Agency 2008).
AN EASY-TO-USE EQUATION FOR ESS VARIANCE
COMPARISON OF RESULTS ACROSS CATCHMENT TYPES
A comparison of uncertainty across small catchments (<25 km2) (Faulkner et al. 2012), permeable catchments (BFIHOST > 0.65) (Faulkner & Barber 2009) and urban catchments (URBEXT2000 > 0.03) (Bayliss et al. 2006) were undertaken to determine if there are significant differences between them and catchments which are larger, primarily rural, and are not considered permeable. To ensure that the sample size did not influence the comparison, the following steps were taken to approximate the sampling distribution of median fse for comparison (using catchment size as an example):
Split the gauges into small and large catchments
List the sample sizes for the small catchments
Find, for each sample size in step 2, the large catchments with the same sample size
Create a sample of large catchments, the same size as the small catchments sample (N), by randomly sampling from the results of step 3. List the fses from the sample of large catchments
Repeat step 4500 times and derive a median fse for each of the 500 samples (providing the sampling distribution for the large catchments)
Resample with replacement N*500 fse from the small catchment fses and calculate the median for each of the 500 samples (providing the sampling distribution for the small catchments)
Compare the two distributions.
These steps were undertaken to compare the UG, SS and ESS 100-year fse across the different catchment types.
RESULTS AND DISCUSSION
Results for ungauged uncertainty
Table 2 provides the resulting fse for ungauged QMED estimates and the mean fse for the UG pooled estimates of longer return periods.
Results for gauged uncertainty
Table 3 provides six number summaries of the ESS fse estimated across the 545 sites suitable for pooling for a range of return periods (T).
Figure 5 provides a comparison between the mean single site fse and the mean ESS fse across a range of return periods.
As can be seen in Figure 5, the benefits of ESS analysis over SS increases significantly after the 20-year return period. That is on average, sites with very few years of data would benefit at shorter return periods.
Figure 6 provides a boxplot comparison of fse across the 545 sites for the two-donor UG case, the SS and the ESS case. The mean 100-year fse for SS, ESS and UG with 0, 1 and 2 donors are 1.16, 1.12, 1.61, 1.57 and 1.55, respectively.
Result and example of use for an easy-to-use equation for ESS variance
The inputs to Equation (11) are estimated from the single site case (QMED and L-CV). The ESS bootstrapped variance and the modelled variance (Equation (11)) were compared with the SS variance equation provided by Kjeldsen (2021) across the 545 gauging stations considered. Figure 7 provides a summary of the results. It was found that those with higher variance for modelled and bootstrapped ESS than for SS, tended to have negative L-SKEW.
Figure 8 provides a scatter plot of the square root of the modelled and bootstrapped variance (i.e. the standard error) for which the R2 value is 0.9. Unfortunately, the results show some heteroscedasticity, which suggests the estimates of ESS standard error is more varied for larger flows, although the proportional error is not correlated with the magnitude of the flow estimate (R2 = 1.38 × 10−5). The results plotted in Figure 8 are across all return periods modelled (5, 10, 20, 50, 100, 200, 500 and 1,000).
As a comparison, the single site estimate of variance (Kjeldsen 2021) for the same AMAX (the L-SKEW is 0.1429, and β is 0.18133) is 933. ESS bootstrapped intervals were derived 30 times and the mean upper and lower 95% intervals were 156 and 217 m3 /s. When using Equation (11), more scrutiny should be applied where the AMAX statistics fall outside the range of those in Table 1. Although the author considers Equation (11) to be readily applicable, given the dispersion of the estimates as flows increase, it is acknowledged that the model could be improved. Consideration of weights in the pooling groups and the individual variance of estimates from pooling group members may be fruitful in this regard. Such investigation may also provide a variance estimator for individual ungauged pooling design lows as a function of pooled parameter estimates.
Results of uncertainty across catchment type
Figures 9–11 provide boxplot comparisons of the resulting sampling distributions of the 100-year median fse. Figure 12 provides a comparison of uncertainty for the observed QMED across the catchment types. Figure 13 provides a boxplot comparison of proportional error for the QMED regression equation with no donors (QMEDcd).
There are a few results that may be of surprise, such as the observed QMED uncertainty in non-permeable catchments being greater than that of permeable (Figure 12, left plot). This is probably due to the lack of variance around the 2-year flow where many permeable catchments see a relatively uniform distribution except for a few outliers. For example, gauging station 42010 on the River Itchen at Highbridge & Allbrook Total, where the baseflow index is 0.96. The AMAX sample is shown in Figure 14. The median can be relatively stable in such catchments. Conversely, the estimation of QMED using the QMEDcd equation in permeable catchments has a greater variance in the proportional error (Figure 13, left plot). This is probably because the calibration for the QMED equation was undertaken on catchments that were mostly non-permeable.
The increased uncertainty in rural catchments when compared with urban in the ESS case (Figure 11, right plot) appears to be due to the fse of the observed QMED in rural catchments being greater (Figure 12, right plot). Why this is the case, is not clear. It may be due to control structures, flood management schemes and abstractions for urban areas that reduce the variance in flow even at the 2-year level.
Bootstrap methods, bespoke to the FEH08 pooling procedures, have been detailed and applied to the SS, UG and gauged pooling (ESS) cases, to derive fses for design flow estimates. The resulting fse estimates have been compared across the FEH08 cases (SS, UG and ESS) and across catchment types for all sites considered suitable for pooling (while accounting for sample size). Building on the work of previous studies, it was established that there is a need and a trend for providing easy-to-use equations for estimating uncertainty for design flow estimates. Given the complexity of the FEH procedures, such equations have been a long time coming since the FEH of 1999. From the initial work of Kjeldsen (2015), Dixon et al. (2017) established quadratic expressions for fse estimates in the case of UG pooling groups. Here, updated versions have been established applying an approach that specifically considered the uncertainty associated with each pooling group. Kjeldsen (2021) established an easy-to-use equation for the SS case, assuming a GLO distribution. This SS variance equation was adapted here for use with design flow estimates in the case of gauged pooling groups (ESS). For the quantification of uncertainty, there is now an analytical and easy to apply set of equations for single site, ungauged and gauged FEH-based design flow estimation. Quantification of uncertainty, and importantly, simple to apply quantification of uncertainty, for estimated design flows, will enable flood risk management authorities and engineers to make better decisions when considering flood risk management infrastructure.
The author would like to thank all those responsible for the availability of the National River Flow Archive. The author would also like to thank the two anonymous reviewers for their helpful comments.
DATA AVAILABILITY STATEMENT
All relevant data are available from an online repository or repositories (https://nrfa.ceh.ac.uk/peak-flow-dataset).