Calibration and sensitivity analysis of a novel water flow and pollution model for future city planning: Future Urban Stormwater Simulation (FUSS)

In the fast-changing world of today it is becoming increasingly difficult to predict the future pressures on urban water management. Highly dynamic and rapid changes in urban form, land-use, and human activities in urban catchments (i.e., city development), coupled with changing rainfall patterns and intensities, are making it difficult to reliably predict a single future to determine the most appropriate urban water practices. Rather, there is an ever-changing, wide array of possible scenarios (Goonetilleke et al. 2005). While significant work has been done in trying to model future climate change on stormwater quantity using long historical rainfall records (Nguyen et al. 2020), stormwater pollution prediction has been lacking, due to a high dependency on uncertain future city planning. Apart from simple mapping tools of potential hazards from diffuse pollution (Mitchell 2005), currently there are no models that can explore the impact of urban planning strategies on water pollution emissions across multiple spatial and temporal scales.

The complex nature of future city planning needs to account for a multitude of stakeholders with different priorities, so tools that are addressing this problem need to be flexible, offering multiple scenarios for pre-set criteria. To accomplish this in a reasonable time, tools and models need to be fast and usually simplify urban form, while keeping a relatively high level of model reliability. Recently, a novel tool has been proposed called UrbanBEATS, which utilizes a spatial urban planning model and abstract urban form characteristics for existing and new areas (Bach et al. 2020). This is done with high accuracy and limited input data, including readily available maps such as land-use, elevation and population density. However, while it explores future urban development well, currently it does not offer prediction of stormwater flow and pollution transport. While water and pollution transport models in urbanized catchments have been either very detailed, accounting for most processes and requiring significant input data (Mannina & Viviani 2010), or end-of-catchment, ignoring pollution distributions and changes throughout the catchment (McCarthy et al. 2011). If we want to model the behaviour of future cities, simplified models are needed that can accurately predict water and pollution transport throughout the catchment spatially and dynamically, without the need for significant input data. While urban drainage tools have been heavily focused on build-up/wash-off processes (Al Ali et al. 2018; Zhang et al. 2019a), they are often unreliable in urban catchments, due to significant pollution from varied human activities (Shi et al. 2019), during both dry and wet events.

The aim of this work is to propose a novel Future Urban Stormwater Simulation (FUSS) model, which utilizes a previously developed urban planning algorithm (UrbanBEATS) to dynamically assess pollution changes in urban catchments. By using minimal input data and adding stochastic point-source pollution to the traditional build-up/wash-off approach, along with the conceptual model, this study highlights the calibration and sensitivity analysis of hydrological (flow) and pollution modules across a range of common stormwater pollutants. The work further explores the use of the model in future urban stormwater prediction, suggesting possible improvements and development directions.

Flow and pollution models, as part of the Future Urban Stormwater Simulation (FUSS) framework, have been applied and calibrated in the Dandenong Creek (DC) catchment in Melbourne, Australia. The catchment area is approximately 8,062 ha, with a mix of residential, commercial, industrial and green spaces. The measurement campaign for model calibration and validation was conducted at three different sites across the catchment, capturing areas that have dominant residential/commercial (Ringwood – RG), and industrial (Old Joes Creek – OJC) land-use, with the final site at the lowest end of the catchment (DC) capturing a mixed land-use. Detailed, 6-minute measurements have been conducted for flow (industrial standard HACH flowmeters with compatible submerged AV probes and American Sigma 900 autosamplers) and rainfall (Melbourne Water weather station at Heathmont in the middle of the catchment), with five continuous water quality samplings during rain events, between May 2019 and April 2020. To account for point-source pollution due to human activity, a high density of low-cost sensors and dead-end ultrafiltration (DEUF) samplers were deployed across OJC catchment to collect an average of 160 dry weather events across all pollutants for industrial and 79 events for residential/commercial land-use. Water quality analysis was conducted for total suspended solids (TSS), total nitrogen (TN), total phosphorous (TP), lead (Pb) as a representative heavy metal, and Escherichia coli (E. coli) for pathogenic pollutants, using standard methods.

The FUSS model (including both flow and pollution modules) was designed to couple outputs of the UrbanBEATS tool (Bach et al. 2020) with the US EPA's Storm Water Management Model (SWMM) (US EPA 2015). Using land-use, population and elevation maps, UrbanBEATS delineated the catchment into 500 × 500 m blocks (25 ha). Urban catchment delineation and its water flow directions were determined by the D8 algorithm (O'Callaghan & Mark 1984); flow paths are post clean-up, favouring blocks that have water bodies and the neighbours with the lowest elevation. The urban form, effective percentage of land use, and the calculation of topographic information (including slope, infiltration, imperviousness, block size and area) were then linked with the simulation tool Stormwater Management Model (SWMM), which we selected as the simulation engine for the flow and water quality model (Figure 1). The UrbanBEATS' user-defined blocks were considered as the sub-catchments in SWMM to allow flow to propagate downstream to the outlet following the continuity equation along with Manning's equation: ∂d/∂t=i-f-q; where i is inflow from precipitation, f is infiltration and q is the overland runoff (evaporation was assumed negligible to reduce input dependency). Due to the simplified nature of the model, and its need to be fast and robust, pits and pipes were not introduced, instead the upstream sub-catchment (block B1 in Figure 1) creates runoff onto the virtual copy of the next downstream sub-catchment (block B2-virtual). These virtual blocks replace the non-existing drainage system, and they retain the original blocks' slope and size, but are 100% impervious and do not receive any rainfall or contribute to pollution accumulation (only transport). To account for disconnected impervious areas, in addition to standard SWMM parameters, sub-catchment depression storage (pervious/impervious), Manning's coefficient (pervious/impervious) and infiltration rate (maximum/minimum), effective impervious factor (EIF) was added, which presents a percentage of impervious area of the block that is directly connected to drainage. This allows the model to dynamically respond to changes in effective imperviousness of the catchment (which will be altered depending on land-use change resulting from the urban growth model) and can be readily calibrated for specific case study.

The water quality model used SWMM's pollution algorithm with linear build-up (⁠⁠), with maximum build-up (B_max) and rate constant (K_B) as calibration parameters, and exponential wash-off function (⁠⁠), with wash-off coefficient (K_w) and exponent (N_w) as calibration parameters (US EPA 2015). The q is the runoff per unit area and is the total mass of pollutant build-up. In addition to the wet-weather flow resulting from the rainfall/runoff process, the stormwater quantity and quality models also aim to simulate the dry-weather component coming from accidental spills, illegal discharges or sewer cross-connections into the stormwater drainage network. The observed flow and pollution concentrations during dry periods were used to create frequency and intensity (flow and concentration) distributions (log-normal), which were stochastically sampled to generate daily human-activity point-source input on each ‘real’ sub-catchment in both flow and pollution models (Figure 1). The distributions were developed for both residential/commercial and industrial land-uses, assuming undeveloped and green spaces are not contributing point-source pollution (no agricultural land use exists in the study catchment).

Calibration and sensitivity analysis of the flow model was done using a Monte-Carlo approach with 10,000 simulations and Nash-Sutcliffe as a criterium function (NSE). Both calibration (using 8 months of continuous data) and the validation (using the remaining 3 months) have been performed on all three catchments (DC, RG and OJC), with NSE calculated only across wet weather events to remove dry weather bias. Relative volume error (RVE) has been used to understand the difference between model total volume prediction and the measured runoff volume. The pollution model was calibrated on the DC catchment using 5,000 Monte-Carlo simulations, and Nash-Sutcliffe for model fit estimation. Four pollution sampling runs (with 48 measurement points) were used for model calibration and one sampling run (with 12 measurements) for model validation. Due to its stochastic nature, 100 iterations were assessed using the optimal parameter sets (for each pollutant), to understand model uncertainty.

The model sensitivity to a high initial number of flow model parameters was assessed across the DC, RG, and OJC catchments. Table 1 shows that overall, only two parameters were found to be sensitive to model performance. The n_imp is strongly negatively correlated with model performance, followed by EIF. Both are considered highly sensitive parameters across all three catchments, with a significantly higher absolute correlation coefficient and lower p-values than the other five parameters. The importance of Manning's n for impervious surfaces in the SWMM model has been widely demonstrated in the existing literature with different study objectives (Ballinas-González et al. 2020). However, n_imp values are similar across the catchments and show lower sensitivity across the lower range (<0.0075, Figure 2). This was expected since the model utilizes the entire block width for runoff, which would result in low velocities under higher coefficient values. In addition, EIF is also an important parameter from the conceptualization of the stormwater network as shown in Dotto et al. (2012). It varies amongst catchments significantly (Figure 2) due to the diverse drainage layout. For catchments with large values (even >1, e.g., OJC) it highlights high drainage connectivity and potential for dry weather discharge. Hence, EIF can be sensitive to future scenarios where land use may change.

The best set of calibration parameter values (Table 2) can provide reliable continuous simulation by producing 0.63 < NSE < 0.76 across long-term calibration (8 months), and 0.57 < NSE < 0.84 during validation (4 months) (catchment dependent). This represents good performance given common interpretations of the NSE in hydrologic model assessment, especially considering the use of long-term continuous data (Hossain et al. 2019). In addition, the model used a very fine simulation time step (6 minutes), which was also found to negatively affect NSE performance (Pontes et al. 2016). In our model, the aggregated hourly and daily results are far better when compared with the 6-min outputs (e.g., NSE_{6 min->1 hour} = 0.68–0.80 and NSE_{6 min->1 day} = 0.80–0.88 in the calibration period). The model also shows a good estimation of the overall stormwater volumes (across both dry and wet periods, Table 2), with some underprediction in OJC during calibration (18%), and overprediction in DC during validation (12%). This is likely due to the seasonality of rainfall patterns in Australia, and measurement uncertainties in OJC.

Figure 3 shows the performance of the observed and modelled hydrographs, across calibration and validation. Although a few peaks may be over- or underpredicted due to the reduction in spatial resolution through sub-catchment aggregation, the hydrograph shape and volume are adequately simulated. Overall, even though the rainfall/runoff module is simple and does not require lengthy estimation of SWMM input data, model performance is comparable with the other studies; for example, an hourly SWMM model by Mancipe-Munoz et al. (2014) achieves NSE = 0.61–0.72 with physical parameters carefully computed from advanced remote sensing techniques; the model performance of EHSMu (Cristiano et al. 2020), an ecohydrological urban runoff model that considers the soil dynamics, is 0.61–0.72 in terms of NSE at the hourly time step.

General pollution model performance across the calibration was found to be good, with excellent fits for typically particulate pollutants like TSS, TP and Pb, while TN and E. coli were more challenging. Table 3 highlights calibrated parameter sets for each pollutant, showing good performance across TSS, TP and Pb over the sample-based NSE assessment of 48 measurements (0.50, 0.61, and 0.48 respectively), and even better performance when event-based assessment is considered (0.86, 0.96, and 0.94 respectively). This suggests accumulation of these pollutants is well represented with a continuous, 6-minute linear buildup model (see Supplementary material, Figure S1), while pollution transport in the catchment positively correlates with catchment flow. This is further supported by Figure 4, where 95th percentiles of the 100-fold optimal parameter simulation (due to model stochasticity) for TSS, TP and Pb follow well most measured data points, with some measurement peaks underestimated. The trend in model uncertainty due to stochasticity is to flatten the peaks, aiming at the higher accuracy of event-based prediction. Due to the study catchment size and complexity, it is likely that some of the poorly estimated peaks (like the second wet weather event) might have been accumulated from the human activity in the catchment (e.g., construction).

TN concentration calibration showed low NSE values (−0.29) across sample-based calibration, but event-averaged calibration highlighted good model estimates (NSE = 0.89, Table 3). The sources of nitrogen in urbanized catchments are time-varied and, as well as stormwater runoff, may come from point-source polluters such as leaking septic tanks, sewer cross-connections and emergency sewer overflows, which the time of occurrence stochastic model component cannot accurately predict, but event average is targeted. Hence, when looking at individual measurements in Figure 4, it can be observed that the start and end of events (intersection between shaded areas) are typically modelled poorly due to significant point-source, dry weather contribution to the TN pollution (Shi et al. 2019), whose trends could not be predicted by the daily stochastic component of this model. Due to overlapping sources such as TN and higher measurement uncertainty, the model performance for E. coli was consistently poor (McCarthy et al. 2011), with the highest sample- and event-based NSEs of −0.58, and −1.81 respectively. While the model seemed to perform well across events 1 and 4 (Figure 4), the sudden spike of E. coli during the second event was not captured, suggesting it might have been a point-source, with model over-compensation during the third event. Hence, the highly dynamic nature of microbial contamination in these urban catchments could present an issue for reliable model performance.

The validation of pollution model was somewhat lower, with only Pb showing good model fit (NSE_max = 0.3). This is likely due to uncertainty in pollution accumulation and extreme weather patterns in Australia in late 2019 and early 2020 (bushfire season), when the validation data set was gathered, suggesting further data is needed for better model validation. Generally, the pollution model was able to follow trends quite accurately even across finer timesteps, with the stochastic component of the model adding robustness. If further accuracy is required, a better understanding of dry weather pollution trends is required. However, since the FUSS model does not aim for high accuracy within every wet weather event, but rather to simulate long-term catchment pollution, the results across most pollutants (other than E. coli) were found to be satisfactory for future management scenario assessment.

The initial testing results of both flow and pollution modules of the novel FUSS model concept showed a promising step towards application in future urban development assessment. The distributed and temporally adaptable (exploration across different simulation time-steps) nature of the model allows for flexible interrogation of pollution emissions across different size catchments. Once calibrated, model results allowed generation of distributed pollution heatmaps (Figure 5), where flow and accumulation of the pollution can be both visually and numerically assessed for each scenario. However, the main benefit of the simplified FUSS methodology is that by using UrbanBEATS as a future urban development engine, which considers how human activity and land-use change in the catchment without any additional input data, FUSS can create a pollution distribution for a future scenario within seconds (for similarly sized catchments), allowing for multi-scenario future assessment. In addition to human activity change, using future rainfall generated patterns and distributions (Zhang et al. 2019b), the effect of future climate can also be considered.

To fully achieve modularity and seamless simulation of the effect of future urban development and climate on the pollution propagation, detailed FUSS model testing is required on ‘pure’ land-use catchments, where residential, commercial, industrial, green (and/or other land-uses) dominate the landscape. This would ensure more reliable land-use-based calibration, that can be applied to other catchments without prior need for calibration. However, to overcome data scarcity, running the model uncalibrated (but using pre-calibrated parameter sets) can still reliably show relative pollution changes throughout the study catchment when changing land-use. Additionally, while this study used 500 × 500 m block resolution, finer (200 × 200 m) or coarser resolutions could be assessed, depending on the size of the study catchment (Bach et al. 2020).

This work highlighted calibration and sensitivity analysis of the new flow and pollution modules as part of the Future Urban Stormwater Simulation (FUSS) framework, which utilizes a simplified catchment urban form (land-use, population, elevation) to explore temporal and spatial flow and pollution dynamics for future urban pollution assessment. While the flow model, with only one significant calibration parameter (EIF), achieved an excellent fit to measured values in a continuous rainfall simulation, the pollution model was more variable. TSS, TP and Pb showed high model efficiency, while TN was predicted well only across event-based assessments. Using the proposed framework, future work will aim to further create land-use dependent model parameter sets, to achieve flexibility for model application across varied urban catchments. Furthermore, with greater availability of dry weather pollution data, the stochastic component of the pollution model could be further improved for more accurate intra-event prediction.

This project is funded by the Australian Research Council (ARC), Linkage Project LP160100241, titled ‘Advancing water pollution emissions modelling in cities of the future’, EPA Victoria, Australia, Melbourne Water, Australia and Know City Council, Australia. We also appreciate the help and support from Caroline Carvalho, Trish Grant, Paul Leahy, Sam LeRay, and Daniella Gerente.

Data cannot be made publicly available; readers should contact the corresponding author for details.