Flood inundation forecasts using validation data generated with the assistance of computer vision

Forecasting ﬂ ood inundation in urban areas is challenging due to the lack of validation data. Recent developments have led to new genres of data sources, such as images and videos from smartphones and CCTV cameras. If the reference dimensions of objects, such as bridges or buildings, in images are known, the images can be used to estimate water levels using computer vision algorithms. Such algorithms employ deep learning and edge detection techniques to identify the water surface in an image, which can be used as additional validation data for forecasting inundation. In this study, a methodology is presented for ﬂ ood inundation forecasting that integrates validation data generated with the assistance of computer vision. Six equi ﬁ nal models are run simultaneously, one of which is selected for forecasting based on a goodness-of- ﬁ t (least error), estimated using the validation data. Collection and processing of images is done of ﬂ ine on a regular basis or following a ﬂ ood event. The results show that the accuracy of inundation forecasting can be improved signi ﬁ cantly using additional validation data.


INTRODUCTION
Forecasting real-time flood inundation is challenging due to the lack of validation data and high-computational time required by two-dimensional (2D) inundation models for producing flood inundation maps. Thus, researchers have focused on using alternatives to 2D inundation models. A straightforward approach is to generate a large database of inundation maps, using either 2D inundation models (Disse et al. ) or historical satellite images (Bhatt et al. ), and create rules to select the most likely inundation map, based on forecasted discharges or flood stages (Bhola et al. ). However, the uncertainty associated with this approach is too large (Henonin et al. ). Another alternative is the use of surrogate models (Bermúdez et al. ) that replace expensive 2D inundation models with data-driven models or more simplified model structures (Razavi et al. ).
Inundation models are available with various levels of simplification (Néelz & Pender ; Bach et al. ).
A widely used model is a diffusive wave model that simplifies full dynamic equations to reduce the computational time (Leandro et al. ). These models are suitable when inertial terms are not important, which is often the case for flood inundations in urban areas (Martins et al. ). Inundation models are typically calibrated, often using Manning's coefficient, to reproduce a set of observations, e.g. water levels, inundation extent. This coefficient represents the resistance to flood flows in the model domain. Various studies point out that inundation models can be very sensitive to these coefficients, which leads to a higher degree of uncertainty (Oubennaceur et al. ). Despite uncertainties, a single calibrated model is used in operational forecast applications (Henonin et al. ) instead of using multiple models in forecasting mode.
Validation of the inundation forecasting is essential to evaluate its accuracy and predictive capabilities. However, spatial and temporal flood validation data in urban areas are scarce (Leandro et al. ). Fortunately, recent developments in technology and crowdsourcing have led to new sources of data. A few researchers have used remote sensing data to validate inundation maps with satellite images (Poser & Dransch ; McDougall ). There have also been attempts to gather crowdsourced hydrological measurements using smartphones and to develop a low-cost, practical method of data collection that can be used to predict floods (Kampf et al. ).
Computer vision algorithms, such as edge detection and image segmentation, have been used to extract information from images (Zhai et al. ) and have been applied to many new areas of research (Uma et al. ). For instance, Jaehyoung & Hernsoo ()  Hence, there remains a need to use this validation data in improving the forecasting and establishing a back communication from crowdsource to the inundation forecasts.
In this paper, we present a methodology that integrates additional validation data, which are extracted from an image with the assistance of a computer vision algorithm.
The main focus is to improve the accuracy of the inundation forecasting by using water levels obtained from images, which are collected on a regular basis or following a flood event. The methodology is tested on three historical flood events and is applied to the city of Kulmbach, Germany.

STUDY SITE AND DATA Kulmbach
The present study is in the city of Kulmbach (Figure 1), which is located in Upper Main river catchment in the north-east of the Free State of Bavaria in Southern Germany. The city has around 26,000 inhabitants. With a population density of 280 inhabitants per km 2 in an area of 92.8 km², it is categorized as a great district city.
Traditionally, it has been a manufacturing base for the food and beverage industry. On 28th May 2006, up to 80 L/m 2 intense rainfall occurred and within a few hours all the streams and rivers were filled (Tvo ). The incident prompted decision makers to revisit the flood protection measures for the city.

Hydrological data
Three hydrological events are used to assess the methodology. The hydrographs of the events upstream of the city at gauges Ködnitz on the river White Main and Kauerndorf on the river Schorgast are presented in Figure 2. Hydrological measurement data for the events were collected by the Bavarian Hydrological Services.
The winter flood in January 2011 (event I) was one of the largest in terms of its magnitude and corresponded to a discharge of 100-year return period at gauge Kauerndorf and 10-year return period at gauge Ködnitz (Figure 2(a)).
Intense rainfall and snow melting in the Fichtel mountains caused floods in several rivers of Upper Franconia. Within 5 days, two peak discharges were recorded. The first one occurred on 9th January and the second peak measured 5 days later on 14th January caused even higher discharges and water levels. The maximum discharge of 92.5 m³/s was recorded at gauge Kauerndorf and 75.3 m³/s at gauge Ködnitz. Agricultural land and traffic routes were flooded, but no serious damage was reported. In Kulmbach, a dyke in the region of Burghaig was about to collapse due to the large volume of water. The Water Management Authority opened the weir in Kulmbach which saved potential damages (Hof ).
Events II and III that occurred on 13th April 2017 and 7th December 2017 respectively, were of relatively smaller magnitudes as compared to event I and corresponded to a discharge of the lowest value of a year (MNQ) and the arithmetic mean (MQ) respectively (Figure 2(b) and 2(c)).
During these events, the water was contained well within the floodplains and thus, no inundation was recorded in the urban area.

Measured water levels and available images
The images and water levels were collected in three phases.
In the first phase (event I), the Water Management Authority in Hof, Germany collected data during the winter flood and recorded water levels at eight bridges in Kulmbach. Figure 1 shows the location of bridges and The water levels were measured using a levelling instrument, Ni 2 (Faig & Kahmen ). The instrument was used due to its availability and high accuracy, therefore associated uncertainties were not evaluated in this study. The For the second phase (event II), images were taken to increase the computer vision data set ( Figure 4). For the third phase (event III), both images and water depths were recorded ( Figure 5). During the event, the water surface heights were recorded using an electrical contact gauge, which is a measuring tape connected to an electric sensor used to detect water depth in tanks. The heights were measured from the tops of the bridges and converted to water levels using the reference dimensions of the bridges.
Event III was used in validating the 2D inundation model.

Topography and land use
The quality of inundation maps mainly depends on the topography of the study area. Topography data for this study were provided by the Water Management Authority, Hof. In the digital elevation model, the terrain is determined by airborne laser scanning and airborne photogrammetry, whereas the river bed is mostly recorded by terrestrial survey (Skublics ).
The land use of the model domain generally consists of agricultural land, specifically floodplains and grasslands, and covers up to 62% of the total model area. Water bodies make up to 7% and include river channels and lakes. The urban area covers around 26% and includes industrial, residential areas and transport infrastructure, whereas the forests form barely 5% of the total area.

METHODOLOGY
This section briefly describes the methodology used for flood inundation forecasts, next the 2D inundation model     it is assumed that the inertial terms are less than the gravity, friction, and pressure terms. Flow movement is driven by barotropic pressure gradient balanced by bottom friction (Brunner ). The equations of mass and momentum conservation are as follows: where H is the surface elevation (m); h is the water depth (m); u and v are the velocity components in the x-and y-directions respectively (ms À1 ); q is a source/sink term; g is the gravitational acceleration (ms À2 ); c f is the bottom friction coefficient (s À1 ); R is the hydraulic radius (m); |V| is the magnitude of the velocity vector (ms À1 ); and M is the inverse of Manning's n (m (1/3) s À1 ).
The model was set up for the city of Kulmbach using the gathered data and Table 1

Computer vision
The work flow of the computer vision algorithm used to estimate water level is shown in Figure 7. restricting the drawn edges to coincide with the edges on the bridge, both in horizontal and vertical directions. The distance between the water surface and reference level (c) in m was obtained using Equation (5): where a_pixel and c_pixel are the pixel distances of the bridge slab and water surface in the image. Their ratio was calibrated for each image using many iterations by manually detecting the edges. In this approach, ten iterations for each image were used to calibrate the ratio. The water level in m asl was calculated as the difference of b and c.

Model selection
For the real-time forecasting, n þ 1 number of model parameter sets were selected from 1,000 uniformly distributed parameter sets based on the sum of the absolute error between the simulated and the measured water levels at eight sites (Figure 1). The goodness-of-fit (e) was calculated using Equation (6), which returns an array of 1,000 values.
The values were sorted and n þ 1 least errors were selected for the inundation forecast: where r is the number of models, p is the number of sites, M i is the measured water level and S i (r) is the water level of the rth model at the ith site.

Comparison of inundation maps comparison
For evaluating the performance between predicted and reference inundation extents, Fit-Statistic (F ) was used. It is widely used for cell-based models (Moya Quiroga et al. ). It varies between 1 for a perfect fit and 0 when no overlap exists. It is defined as in Equation (7): where shows very large differences. The root-mean-square error (RMSE) was also calculated for the assessment between the selected and calibrated models. It is calculated using Equation (8) for flooded cells: where n is the number of flooded cells, m i and s i are the water depths in the calibrated and selected models, respectively.

Calibration and validation of the HEC-RAS 2D model
The water levels measured in event I were used to calibrate the model parameters. Table 2  but it over-predicted the water levels by 0.10 m.
The maximum inundation focussed on the eight sites is shown in Figure 8 for the three events. In event I, the flood-

RESULTS OF FLOOD INUNDATION FORECASTS WITH COMPUTER VISION
This section presents the water levels extracted from the images using computer vision and the models selected for flood inundation forecasting.

Water levels obtained by computer vision
A computer vision algorithm was applied to the images collected from three different events at eight sites in the city of Kulmbach. The images that were suitable for computer vision are shown in Figures 9-11. Images of event II were used as the reference images and events I and III as the target images. The water levels obtained from the algorithm were compared with the measured water levels.
A box plot of the pixel distance ratiodivision of the distance between the water surface and the height of the referenced object in pixels (c_pixel), and the   respectively. Thus, the water surface height (c) was calculated in m using Equation (5). Furthermore, the mean of the c was converted to water level. The difference of the water levels between the measured and predicted by computer vision for the seven images is shown in Table 3. As mentioned before, no measurement was performed for event II, hence the difference cannot be calculated.

Flood inundation forecasting
The total number of models to be simulated in real-time is restricted by the computational resources. Given a large infrastructure, a large number of models can be realized with this methodology, however, in our case, existing resources limited the number of models to six (1 þ 5). To conclude, out of 1,000 models, we selected six models that produced the least error in the water levels. As mentioned above, the sensitivity analysis was performed for a single event (event I) based on pre-determined ranges of the 2D inundation model parameter. Figure 13 presents the six models that return the least error in the water levels at the eight sites. The radar plot shows the variability of Manning's M for each land use class. It is evident from the figure that the parameter space is different in each model, which results in different output. The output of the models is presented in Table 4, which shows the difference between the measured and the simulated water levels resulted from the six models for event I. A threshold value of ± 0.15 m is used for highlighting the differences in the model results.
To select the most suitable model out of the six, water levels obtained using computer vision are used as the validation data. The goodness-of-fit (Equation (6)) is calculated for the three events for the six models and one least error model is selected for the real-time forecast for each event.
If there were no validation data, inundation maps of the calibrated model (M Cal ) would have been used as the final forecast.
To assess the difference between the calibrated and selected models, goodness-of-fit Fit-Statistic (F) in percentage and root-mean-square error (RMSE) in m (Equations (8) and (9)) is presented in Table 5. For events I and III, the    The spatial distribution of the error for event I is shown in Figure 14.

Computer vision
For event I, only the images of sites 2, 5, and 6 were amenable for analysis using computer vision. The images from   Therefore, the uncertainty in the computer vision-based water level was estimated at ±0.13 m. In some cases, the side surface used as the reference for drawing the edges is  A reasonable match was found in the measured and the computer vision water levels on five images: sites 2, 5, and 6 in event I and sites 2 and 6 in event III. The water levels predicted from computer vision for event II (Figure 10(a) and 10(b)) were 1.12 and 0.92 m for sites 2 and 6, respectively.
In the absence of measured water levels, the calibrated HEC-RAS model results at those sites, 1.2 and 0.65 m, can be used as good estimates to evaluate the performance of computer vision. The image for site 2 is more in line with the requirements than the image for site 6. This has potentially resulted in better results for site 2 than site 6. If the images are captured as per the requirements, computer vision has the potential to be a good validation tool for flood inundation forecasting.
One of the limitations of these methodologies (as in

Flood inundation forecasting
It can be seen from the model parameter distribution ( Figure 13) that six different sets of parameters were selected based on the least error. Equifinality can be observed from Comparison between the 2D model and computer vision was carried out on the available sites (see Table 4).
For event I, the model M 3 was selected based on the least error ( These examples show that the inclusion of computer vision can produce changes in the forecasted inundation extent. In this study, we assumed that computer vision was the prevailing source of accurate data.

CONCLUSIONS
We present a methodology for real-time flood inundation forecasts incorporating additional crowd-sourced validation data generated with the assistance of the computer vision algorithm. Six 2D diffusive wave models (HEC-RAS 2D) are run in parallel. The selection of models used for the inundation forecasting is based on 1,000 models run for a single event. In this study, validation of the methodology is carried out using three events on eight sites located in the Kulmbach inner city. Model selection (one out of six) for flood forecasting is based on the least error using computer vision at available sites. The computer vision algorithm is used to estimate the water levels of the images that meet the requirements of the proposed guidelines. The algorithm uses specific features, such as bridges and water surfaces, to estimate water levels in the images.
Since the procedure is not fully automated, we suggest collecting images on a regular basis or following a flood event for model selection.
The major advantage of the forecast framework is its fast integration of more complex models will become feasible.
In addition, analysing additional model outputs, such as flow velocities and hazards, should improve the existing forecasting framework by incorporating flood risk assessments.