Feasibility of using smart meter water consumption data and in-sewer flow observations for sewer system analysis: a case study

The dry weather flow (DWF) describes the flow in the sewer system during periods without rain. This flow consists of wastewater from, for example, households, industry and institutions, as well as groundwater infiltration and rain-induced infiltration into the sewer that can take place for days and weeks after the end of a rain event due to the long transportation time of the stormwater through soil.

Accurate knowledge of the sources of DWF as well as an estimation of its spatial distribution can be used for a wide range of management objectives:

• -

  Estimation of total suspended solids, chemical oxygen demand, organic matter, drugs and nitrogenous pollution loads if the DWF estimation is supported by compound measurements or estimations for different consumer types (Métadier & Bertrand-Krajewski 2011; Schilperoort et al. 2012; Plósz et al. 2013).

• -

  Optimised dimensioning (Cole & Stewart 2013) and operation (Brito et al. 2017) of sewer systems including control during dry weather with focus on energy minimisation and minimisation of sewer overflow during rain events. This could both be based on water quantities and estimated water quality, cf. the first bullet. Both types of control may be carried out using model predictive control (Lund et al. 2018).

• -

  Optimised dimensioning and operation of wastewater treatment plants (WWTPs) (for example, chemical dosing) by knowing how much of the inflow to the plant originates from households, from industry and from groundwater, since these fractions will determine the expected pollutant concentrations (Nguyen et al. 2018), cf. the first bullet.

• -

  Asset management (Djebbar & Kadota 1998; Brito et al. 2017) based on, for example, knowledge of leaking sewer pipes.

• -

  Improved heat recovery from sewage systems (for example, Abdel-Aal et al. 2018) and better planning of decentralised wastewater reuse (Elías-Maxil et al. 2014) by using the estimation of the wastewater flow distribution to pinpoint optimal heat pump locations and optimal wastewater reuse locations, respectively.

In residential areas, the wastewater flow is dominated by the citizens' behaviour. In upstream parts of the system or in small systems, the flow is intermittent (i.e. occurring at irregular intervals) and the detailed flow dynamics may not be captured if too large a sampling interval is used (Butler & Graham 1995; Elías-Maxil et al. 2014). The DWF of smaller areas is thus, in general, more difficult to estimate (Djebbar & Kadota 1998). Further downstream, the aggregation of different dynamic inputs from many small upstream sources results in a change in the nature of the wastewater flow, which forms a less dynamic pattern (Butler & Graham 1995). The DWF from a catchment has previously been studied by a range of authors. For example, Métadier & Bertrand-Krajewski (2011) analysed DWF data and recognised different flow patterns depending on the weekday and date, but also found a relatively large variation within each pattern; Djebbar & Kadota (1998) estimated DWF peaks and average DWF using a neural network model based on the land use and population; and Brito et al. (2017) fitted a partial least-squares model to DWF data to estimate the DWF in situations with missing data. All of these methods may be used to establish DWF patterns but will not give a real-time picture of the DWF. Such real-time information could be obtained from in-sewer measurements, but flow sensors are often scarcely distributed, since it is both expensive and impractical to cover a large urban drainage system (Djebbar & Kadota 1998). Highly spatially distributed real-time DWF information is therefore currently not realistic to obtain using only in-sewer observations.

Water supply and urban drainage systems are intrinsically linked, since most of the consumed water ends up in the sewer system. The wastewater flow can thus be approximated by estimating the water consumption. Butler & Graham (1995) and Elías-Maxil et al. (2014) used a questionnaire and a probabilistic model to estimate the water consumption. Both studies subsequently modelled the resulting flow in the sewer system to obtain a spatial and temporal distribution of the wastewater flow. However, these methods only provide generalised flow patterns. Contrarily, smart meters measure the real-time water consumption in each household every hour or more frequently, and are increasingly implemented as part of the digitalisation of the water sector, mainly for billing and leakage detection purposes (Boyle et al. 2013; Monks et al. 2019). Data collected from the water supply system should be pre-processed before use including validation and re-estimation of missing and invalid data (Kirstein et al. 2019). After this, the smart meter data can potentially be used to estimate the wastewater flow with a high spatial and temporal resolution. Furthermore, knowledge of the wastewater flow component can, in comparison with in-sewer flow data, be used to estimate the addition or loss of water through infiltration and exfiltration and thus contribute to closing the water balance of the urban drainage system. The concept of applying smart meter data to estimate wastewater flows has been mentioned in the recent literature (Cole & Stewart 2013; Nguyen et al. 2018; Monks et al. 2019). Zhang et al. (2021) used a combination of pressure and flow data from the water distribution system, smart meter data and a water distribution network model to successfully link water consumption with sewage flow. In this effort, they scaled the water consumption based on the sewage flow observations and thus did not take, for example, infiltration/exfiltration and the uncertainty of flow measurements into account.

This study investigates the hypothesis that smart meter consumption data can be used directly to estimate the magnitude, timing and spatial distribution of wastewater flow, without the need of a water distribution network model. This hypothesis is tested using data from the city of Elsinore, Denmark. We validate the smart meter data by comparing the data with observations from the waterworks outlet, observations from the WWTP inlet, and annual water consumption data from a year prior to the installation of smart meters. The smart meter data are also routed through a 1D hydrodynamic urban drainage model for simulating the flow dynamics. We compare the smart meter data and the simulated flow to in-sewer flow measurements from five locations in Elsinore city centre. This comparison forms the basis for assessing anomalies, including unreliable data and other DWF components than wastewater. The comparison is undertaken by using both quantitative methods and logical reasoning to evaluate the wide range of possible sources of uncertainty that arise when employing large amounts of data in a real full-scale urban catchment study.

The city of Elsinore, Denmark, is located 30 km north of Copenhagen and has 47,000 inhabitants. It covers an area of 18 km² and has a dense medieval centre, surrounded by suburbs developed mainly in the period 1930–1970. The potable water is supplied by four waterworks and the city contains one water tower. The drainage system is predominantly a combined system that leads most of the water to a centralised WWTP. The average yearly precipitation in the city is 670 mm. The upstream parts of the city have a low degree of imperviousness (20–40%), whereas the downstream part containing the city centre is more paved (imperviousness of 60–80%). The utility company has installed permanent smart meters in all the consumers' homes, and in-sewer sensors have been installed for a temporary monitoring campaign in the sewer system, which enables the type of investigations performed in this study. Table 1 explains the main characteristics of the wide range of different data sources that form the basis for this study, which focuses on three week-long periods in October and November 2018 and February 2019.

Around 19,000 MULTICAL21 smart meters with a temporal resolution of 1 h and a measurement uncertainty of up to ±5% (Kamstrup 2019) are permanently installed in households and industries for billing purposes (Figure 1). In the urban area, the meters are placed inside the households. The smart meters cover the entire city, except 14 consumers who still have manually read meters. There are no unmetered consumers. Around 2,000 of these smart meters are situated in the catchment upstream of the in-sewer sensors (Figure 1), which is the main area of interest in this study. Contractor 1 undertook the installation of the smart meters and provided data from three separate weeks in 2018 and 2019 (Table 1). The raw smart meter data are accumulated volume readings. Due to the various data transmission and collection mechanisms of smart meter data, the raw data arrive at non-uniformly distributed time steps (for example, Kirstein et al. 2021). Contractor 1 filled the data gaps primarily using linear interpolation to obtain a uniform interval between flow values of 1 h. Kirstein et al. (2021) showed that this interpolation procedure sufficiently represents the total consumption within an area as long as a great proportion of the individual consumers' raw smart meter data are not missing and if the raw data resolution is less than 2 h. Contractor 1 furthermore reported the quality of the data. The time periods in Table 1 were selected based on the availability of good quality data in periods without rainfall creating direct runoff in the sewer system.

Hourly outflow data from the four waterworks supplying clean water to the city were obtained from MAGFLOW flowmeters type MAG3100 Water with an uncertainty of less than 5% (Siemens 2020) for specific days in each time period (Table 1). The utility estimated one of the waterwork outlets and the outlet from the water tower based on other observation points, meaning that the actual uncertainty is higher than the declared uncertainty for the instruments.

Furthermore, Contractor 2, who maintains the utility's hydraulic water distribution network model, provided a database with the annual water consumption readings for each individual consumer in Elsinore from 1 January to 31 December 2012. This database contains readings from manually read water meters (read once a year for billing purposes) and is the newest available independent source of water consumption data from before smart meters were installed in Elsinore. In Denmark, it is legally required that each consumer has a water meter at home, and these meters are owned by the utility that performs random control samples to ensure that the meters provide accurate and unbiased readings. The high Danish water price means that most people and the utility are very aware of large changes in the water bills, which further increases the credibility of the consumption data. The 2012 data have a declared uncertainty of ±2% and are thus the best estimation of the annual water consumption we can get that is independent of the smart meters.

Additionally, the utility provided the total amount of distributed drinking water per year in 2012 and 2018 and per month in 2018 and 2019. The utility also provided a list of known pipe bursts in the three time periods.

Utility Elsinore has divided the city into wastewater catchments depending on the layout of the sewer system. These catchments, together with asset data, are described in the municipal wastewater plans, which in detail explain how wastewater and rainwater flows are and will be managed. The asset data have been used as the basis for a detailed 1D hydrodynamic model of the sewer system, which is constructed in MIKE URBAN and maintained by Contractor 3.

Contractor 4 installed level and velocity sensors in Elsinore city centre, as part of a measurement campaign running from May 2018 to June 2019, in order to obtain data for calibration of the model. The sensors were installed at five locations: East Upstream (EU), East Downstream (ED), Central Downstream (CD), West Upstream (WU) and West Downstream (WD) (Figure 1). These abbreviations will be used both to refer to the sensor locations and to their upstream catchments. The level and velocity observations stem from pressure and Doppler sensors, respectively. The uncertainty of the resulting flow data for the given sensors is estimated to be around ±12% (Franck, T. 2019, Personal Communication, BlueNorth). Two of each sensor types were installed at each of the five locations to ensure that data were present even in case of failure as well as for validation purposes. The contractor also combined the level and velocity data with the geometry of the pipes (multiplying the velocity with the wetted area based on the geometry) to obtain flow observations. The pipes are circular in four of the locations (WU, ED, CD and WD), while one location has an egg-shaped geometry (EU).

Daily inflow data were also available for the WWTP (Table 1). These were aggregated from an ultrasound inlet flow sensor with an uncertainty of ±10 m³/h (Laursen, B. 2019, Personal Communication, Utility Elsinore). Furthermore, the utility and Contractor 4 provided information on last calibration dates for the WWTP inlet sensor and the in-sewer sensors, respectively (see Table 1).

Utility Elsinore supplied data from 18,804 meters, of which 18,449 were georeferenced. About 338 of the remaining 355 meters were successfully referenced using the QGIS tool ‘MMQGIS’. We manually verified that the 17 non-referenced meters are located outside the upstream area of the in-sewer sensors and thus outside the main area of interest. Due to the general data protection regulation, we could not obtain water consumption data on household level for the 18,787 georeferenced meters, and they should thus be aggregated into groups. It is important that all smart meters within a group discharge water to the same part of the sewer system. Thus, we initially based the groups on the catchments from the Elsinore wastewater plans. 20% of the 18,787 meters were discarded because they did not discharge wastewater to the WWTP used in this study, leaving 15,011 meters. It was important to get an accurate description of the flow dynamics in the area upstream from the in-sewer sensors, and we therefore further manually sub-divided the groups in this area from 19 groups in the wastewater plans to 111 groups with a total of 2,015 meters and between 2 and 38 meters in each group (see the group divisions in Figure 1). This division was based on the general flow paths in the sewer system (described by the asset database). The overall result was 15,011 meters distributed in 245 groups containing each up to 869 meters.

We provided a list with the meters in each group to Contractor 1 who returned the aggregated smart meter water consumption for each group in hourly time steps. 2.4% of the 15,011 meters were missing in the files provided by the contractor, of which 45 were located in the area upstream of the in-sewer sensors. These meters were missing because they had either not yet been replaced with smart meters (which are at most 14 meters) or because they were older meters no longer in use. The total water consumption for these 45 meters was estimated to be around 9,500 m³ in 2018 (corresponding to a yearly average of 0.3 L/s), which is less than 2% of the water consumption in the area. The final result included 14,694 smart meters in 245 groups, of which 1,970 smart meters belonged to the 111 groups in the area upstream of the in-sewer sensors.

Furthermore, Contractor 1 returned the aggregated consumption data from all smart meters receiving water from the Elsinore Waterworks.

The applied MIKE URBAN model (supplied by Contractor 3) simulates the flow in the sewer system pipes by solving the full 1D Saint-Venant equations. It can furthermore simulate the behaviour and flows through the most important structures and elements in the system such as basins, overflow weirs, pumps and gates. The typical application for this sort of model is to simulate stormwater flows to assess the hydraulic capacity of the system, but it is also capable of simulating flows under dry weather conditions, as in this study. The applied model has a drainage area of 386 ha and contains 1,959 nodes of which seven are basins, as well as 1,986 links, 30 pumps, 33 weirs and one orifice. All asset attributes have been derived from physical properties, such as pipe lengths, slopes and pipe materials. There are no other water sources included in the model than the wastewater flow, which in this study is taken directly from smart meter data. During dry weather, none of the overflows are activated and the only sink is thus the WWTP. The model is thus well suited for routing the wastewater from the consumers to the WWTP and thereby taking into account the time dynamics of the system. These time dynamics have not been calibrated for the dry weather situation.

To simulate the wastewater flow, Q_sim, with the MIKE URBAN model, we identified the model entry (boundary) node for each of the 245 smart meter groups. The aggregation of a set of smart meters into one group will cause some water consumption to be delayed and some to arrive too fast to the sewer system, depending on the consumers' distance to the entry node in the sewer system model. Thus, we decided to add the aggregated smart meter data to the node in the model that was closest to the geometrical centre of each group. The error introduced here will be in the form of a time displacement of the wastewater inflow, and the magnitude of this error is expected to be minor due to the limited size of the smart meter groups. We found the node nearest to the centre of the shape of each group with the Distance Matrix tool in QGIS. The M1D API (DHI 2019) in C# was hereafter employed to automatically use the 245 water consumption time series files as boundary conditions for the 245 localised entry nodes in the MIKE URBAN model. Hereafter, we used C# to run the model, which allowed us to calculate the simulated wastewater flow in the entire sewer system model, including in the five sensor locations (Q_sim,loc). The model was run with adaptive time steps between 5 and 60 s (but saving the results in 1-min time steps), and it took less than 5 min to run one day of simulations on a Windows server 2012 with an Intel^® Xeon^® CPU E5-2670 v2 @ 2.50 GHz processor with 20 cores and 20 logical processors when the runoff file was generated previously. The setup used here could thus be applied in a real-time setting.

Q_SM and Q_sim may differ from Q_obs if the system is affected by other water inflows and outflows than the smart meter-measured water consumption or due to erroneous data.

These residuals may both be positive and negative, and exhibit constant, diurnal or seasonal variations. They may also vary according to the outside temperature, previous rainfall and pipe geometry. Table 2 lists three overall possible reasons for deviations as well as expected residual patterns. Some of the uncertainties can be quantified, such as the declared uncertainty from the measurement devices (Table 1). The uncertainty associated with the equipment installed in situ may, however, be higher than declared by the manufacturer, which can be difficult to quantify without comparing the data with other, independent data sources. The remaining potential uncertainties listed in Table 2 are also difficult to quantify without additional data, since these are often highly contextual and case-specific. One would thus need to make additional independent measurements in the case area for the same time period as the in-sewer observations were performed. Even here, however, the actual corresponding uncertainty would be difficult to quantify. Complex systems, such as the one presented in this study, are often characterised by higher-order levels of uncertainties rather than pure statistical uncertainty, and in this case pragmatism is called for (Warmink et al. 2010). To prevent the pitfall of underestimating the uncertainty by including only the uncertainties that can be quantified explicitly (Warmink et al. 2010), the potential causes for deviations between the smart meter and in-sewer data are evaluated based on their effect on the residual pattern calculated by Equation (5). The three sub-sections below elaborate on the three deviation types individually.

Most of the consumed water ends up in the sewer system either directly, such as water used for showering, or delayed, such as the water used for washing clothes. However, some consumed water will end up elsewhere, such as water that evaporates during cooking or from wet laundry or water that infiltrates in the soil during car washing or gardening; the latter is assumed to be season dependent and may even infiltrate into the sewer system as discussed later. In Denmark, a conservative estimate is that at least 86% of the used water is discharged to the sewer system (HOFOR 2016). This is in alignment with results from, for example, Zhang et al. (2021). Since more water is used during the day than at night, the residuals will be larger during the day and thus exhibit a diurnal pattern. In coastal cities like Elsinore, discrepancies may also arise if ferries and trains take clean water on board in the city for kitchens, toilets and cleaning but discharge the foul water somewhere else or at another time of the day. This may lead to irregular residual patterns.

Leakage may occur in the supply pipe after the smart meter location, which means the metered consumption would not enter the urban drainage system and the observed sewer flow would be smaller than the simulated flow. If the smart meter in this case is placed at the property boundary, the water from a leakage in the supply pipe between the property boundary and the house is likely to infiltrate into the soil, a situation hard to diagnose. If the smart meter is instead placed inside a household, leakage after the smart meter is likely to lead to building damage which is easier to diagnose.

Exfiltration from the sewer system may occur when there are cracks and fractures in the pipes and the groundwater level is below the sewer system. The groundwater level is generally lower in the summer than during winter, meaning that the exfiltration rate is expected to follow a seasonal pattern with peaks in the summer.

Q_sim only contains data from consumers that have smart meters installed. Some consumers may, however, have older meters requiring manual readings. In fact, large consumers will often have such meters installed. Such a deviation would exhibit a diurnal pattern unless missing industrial users consume water, for example, during the night. Some of the water discharged to the sewer system may furthermore originate from somewhere else. This may be the case for, for example, ferries and trains that take on board clean water when they set out and discharge it through the sewer system at their destination. This may result in irregular residual patterns.

Buildings and construction sites that are located partly underground may need to pump away groundwater and possibly intruding seawater if they are located on the coast. This water may be discharged into the ocean, infiltrated somewhere else or discharged to the sewer system. Such pumping would most likely occur both day and night leading to a constant deviation. If the groundwater level is only an issue in the winter, the residual pattern will be seasonal. Sea-level variations may, however, also affect the groundwater level and thus impact the amount of pumping.

Rainwater may be used for toilet flushing and laundry, and this will increase the flow in the urban drainage system compared with the recording by the smart meters. Since more water is consumed during the day, the residual pattern would follow a diurnal pattern. An increase in water consumption from the water distribution system would be seen again when the rainwater tanks are empty.

In winter, precipitation falling as snow can cause a delayed runoff into the sewer system. This may increase the measured flow in the urban drainage system. The temperature determines when the snow melts, and the residuals will thus neither follow a constant, diurnal or seasonal pattern.

Infiltration into leaky sewers can arise from either groundwater, rain, groundwater pumped from perimeter drains around buildings and construction sites, or leaking water supply pipes:

• -

  Groundwater levels change slowly over the course of the year, and groundwater infiltration thus exhibits a seasonal change in the infiltration rate, which may also be affected by the sea water level.

• -

  Rain-induced infiltration increases the sewage flow only after rain events and slowly decreases with time.

• -

  Infiltration stemming from the pumping of groundwater exhibits a constant or seasonal pattern (see ‘Pumping of groundwater to the sewer system’).

• -

  Leakage from the water distribution system into the urban drainage system is expected to be a function of the pressure in the water distribution system. If the system pumping is pressure controlled or if the pressure fluctuates due to variations in demand, the leakages could, to some extent, display a diurnal pattern. In Denmark, the water supply pipes in general have too large diameters due to former expectations of growth in the water consumption and requirements of being able to supply fire hydrants. Therefore, the pressure drops in the distribution systems are small, and temporal variations in demand will only have a minor effect on the pressure in the pipe system. The residuals would thus predominantly be constant over the day.

Some sewer system flow observations, like the ones in Elsinore, are obtained by multiplying the velocity with the wetted area of the cross-section of the pipe. If sedimentation is present at the location of the sensor, the water level will rise and lead to a larger calculated than actual flow. The residuals will vary in size depending on the geometry of the pipe. Usually sensor companies will try to avoid installing flow sensors in pipes with a lot of sediments, but sedimentation is a dynamic process that may change after the installation of the sensor.

Deviations may occur due to erroneous smart meter observations. In general, individual meter errors are not likely to have a noticeable impact on the simulated results since each consumer only uses a minor fraction of the water in each sub-catchment, and suspicious individual meter errors would due to the high water price in Denmark be noticed during the billing process. A general bias of the smart meters could, however, impact the results. Depending on whether this bias is proportional or additive, it may both result in a diurnal pattern or a constant deviation. Furthermore, poor transmission conditions can result in even larger uncertainties, and wrong mappings of the geographical location of the smart meters will also cause errors.

The conceptualisation of the urban drainage system layout is essential to make the correct coupling between the smart meter data and in-sewer sensors. If one or more catchments are falsely connected to the sewer system upstream of the in-sewer sensors, the simulated flow would be larger than the observed. Contrarily, missing connections in the asset database would lead to a smaller simulated than observed flow. This deviation would follow a diurnal pattern.

In-sewer observations of wastewater velocity, level and flows may be erroneous due to turbulence, presence of solids, aggressive environment (Hager 1994), low water depth (Larrarte et al. 2008), or poor maintenance and calibration. The residuals may both follow a diurnal pattern or be a constant offset. Normally, flow observations are assumed to have uncertainties up to 20% (Bertrand-Krajewski et al. 2003), leading to a diurnal residual pattern.

Here, we (1) assess if smart meter data can be used to estimate the wastewater component of DWF, (2) identify anomalies, i.e. other sources to the DWF and erroneous data, by comparing the simulated wastewater component with the observed in-sewer flow, and (3) discuss the added value from using a hydrodynamic 1D model.

The smart meter data can be used to estimate the wastewater component of the DWF if the smart meter data set is trustworthy.

Figure 2 compares the total outflow from the waterworks and water tower, Q_obs,WW, to the aggregated smart meter data from the whole of Elsinore, Q_SM,Elsinore. First, it is noticed that the consumed water exceeds the total outflow from the waterworks for a short period in October. Data about the outflow were missing for this time period, and the period can thus not be used to assess the validity of the smart meter data. Looking at the other two time periods, the smart meters registered on average 14–17 L/s (15–17.5%) less water than Q_obs,WW. This deviation could be due to leakage in the water distribution network, errors in the waterworks' flow sensor (higher than the <5% reported uncertainty, potentially due to the partial estimation of the total waterwork outflow) or errors in the smart meter data set. The smart meter data set may be erroneous due to meter errors (higher than the 5% reported uncertainty), data transmission errors, faults in the data handling or missing consumers. The latter is, however, not assumed to be an issue, since all consumers are metered and less than 0.1% of the consumers still have manually read meters. The data in Figure 2 thus do not give a clear indication of the validity of the smart meter data.

We further evaluated the completeness and quality of the smart meter data set by comparing the daily inflow to the WWTP, Q_obs,WWTP, with the simulated inflow to the WWTP, Q_sim,WWTP, using all smart meters in the WWTP catchment (Figure 3). The model was here used as an easy way to include the flow only from smart meters within the WWTP catchment. Since Figure 3 shows daily values and the model does not affect the time dynamics on this time scale, the results for the summed smart meters would be almost the same as the simulated (apart from insignificant differences due to numerical issues). Q_sim,WWTP is around 1.5 times that of Q_obs,WWTP in October and November. This corresponds to, on average, more than 1,500 m³/day, which seems an unlikely amount of exfiltration or an unrealistic volume of consumed water not ending up in the urban drainage system. In February, however, Q_sim,WWTP and Q_obs,WWTP are very similar. Q_sim,WWTP is in the same range as in October and November, but Q_obs,WWTP has increased notably. This cannot be explained by the calibration of the WWTP inlet sensor since this was last calibrated in March 2018. Neither can it be explained by an increase in the outflow from the area with in-sewer observations shown in Figure 3 (sensor location WD). Here, the average daily observed flow is actually larger in October and November than in February, and this is not explained by the smart meter data. The sensed area, however, only partially covers the WWTP catchment (see Figure 1). The deviation may thus instead be due to increased groundwater infiltration in February (winter) in the unsensed part of the WWTP catchment, but it may also be due to erroneous WWTP inlet observations above the reported uncertainty (Table 1).

Since the 2012 consumption data are independent from the smart meter data, we assume that the completeness and quality of the smart meter data set can be assessed by comparing these with this data (see Figure 4). We have added the declared uncertainty as error bars to the figure. For the water consumption from 2012, we have furthermore adjusted this uncertainty to reflect the annual and seasonal variations. From 2012 to 2018, there was a total decrease in the outflow from the waterworks and water tower of 8%, which fits well with a general decrease in water consumption both in Elsinore and in Denmark in general (DANVA 2017; Elsinore Municipality 2019), and the error bars are thus shifted downwards with 8% of the registered consumption value. This means that the error bars reflect the 2012 data's expected ability to predict the 2018 conditions. Furthermore, the monthly average outflow from the waterworks and water tower ranged between −10 to +11% compared with the mean outflow in 2018 and 2019. In October, November and February, the water consumption was specifically 2, 4 and 10% higher than the mean. We have therefore conservatively added a 10% uncertainty to all three time periods on top of the declared uncertainty of 2%. We have not added error bars to the simulated wastewater flow, ⁠, since this is simply a post-processing of the smart meter data, ⁠. These two differ from each other due to water generated in empty pipes of the model for the sake of numerical stability and because is affected by the routing time in the sewer system. Overall, the smart meter data and the consumption data from 2012 match well, taking the uncertainties into account. Figure 4 also shows a large difference between the 2012 water consumption data and ⁠, which is larger than the declared uncertainties. Consumers that have private wells will neither be in the 2012 consumption database nor have smart meters installed. However, there are no private or commercial wells from where the water is subsequently discharged to the sewer system in the area upstream of the in-sewer flow sensors (Elsinore Municipality 2019; Pratt, A. 2019, Personal Communication, Elsinore Municipality). Overall, the comparison with the 2012 water consumption data indicates that the amount of unaccounted for consumers and uncertainty related to the smart meters, data transmission and data handling (Table 2) is limited, meaning that the smart meter data set is complete and sufficiently correct to represent the wastewater component of the DWF.

Figures 3 and 4 show that there is a discrepancy between the smart meter-based wastewater flows, Q_sim and Q_SM, and the observed in-sewer flow, Q_obs, which is so big that it exceeds the combined, declared uncertainty of the smart meter and in-sewer sensors (Table 1).

To assess possible anomalies, we look at the mass balances for the five sub-catchments for the three time periods (Equation (3)). From Figure 5, it is evident that there are no clear patterns in which catchments are losing or gaining water in the three periods. Trusting the observations, almost 2,000 m³ more water leaves ‘ED’ than enters it during October (negative mass balance), which could indicate a large external water inflow such as, for example, infiltration. From the percentage (Equation (4)) shown in Figure 5, it is clear that this possible external inflow would be the main contributor of the outflow stemming from the sub-catchment. This trend corresponds poorly with the results for November and February that show a small positive mass balance for ‘ED’. The natural variations in groundwater levels and soil moisture that could explain month-to-month variations in infiltration would give the opposite pattern, since the summer of 2018 was very dry followed by a wet winter. ‘CD’ displays some of this expected behaviour, but it would be premature to draw any conclusions from this considering the large unexplainable variation for the remaining sub-catchments.

Figure 6 shows the time dynamics of Q_sim, Q_SM and Q_obs, and is arranged to resemble the flow through the sewer system as conceptualised in Figure 1. The flow is naturally smallest in ‘WU’, since this is the smallest of the five sub-catchments and does not receive water from any further upstream catchments. The axis for this plot is thus scaled differently than for the remaining plots. The flow increases through the system as more water aggregates. Most of the outflow from the most downstream catchment (WD) is generated in the most upstream catchment, EU, which also has by far the most consumers. The residuals between Q_obs and Q_sim (Equation (5)) are shown in the upper left corner of Figure 6 for all five sub-catchments.

Figure 6 shows that the flows in WU match well in October and November. There are negative discrepancies between Q_obs and the smart meter-based flows (Q_SM and Q_sim) in EU in October and in WU in February, and many time periods and sub-catchments with positive residuals. In the following, we use Table 2 (besides unaccounted for consumers and uncertainty related to the smart meters, data transmission and data handling, which were previously deemed valid) and Figure 6 to deduct what could be plausible causes for the observed discrepancies between the flow results from various data sources.

The negative diurnal residuals likely do not stem from exfiltration from the sewer system since Q_res,EU exhibits a clear diurnal pattern in October while exfiltration is expected to vary according to the season. Furthermore, a net exfiltration is not expected to occur in February (where the groundwater level generally is higher) without also occurring in October and November. Neither do the negative residuals stem from consumed water not discharged to the sewer system since residential areas like ‘EU’ and ‘WU’ should discharge at least 86% of the used water to the sewer system, whereas the observed differences on average are 60 and 40% in EU (October) and WU (February), respectively. Furthermore, the smart meters in Elsinore are located inside the houses, which means that it is fair to assume that leakage on the consumers' side of the smart meters is minimal and can be disregarded. It is not likely that the positive diurnal residuals stem from pumping of groundwater to the sewer system because this will not lead to diurnal residuals. Furthermore, the Capital Region of Denmark, who stores records of such pumping sites, confirmed that no such pumping of groundwater to the sewer is taking place in the area of Elsinore where the in-sewer sensors are placed (Vormbak, F. 2019, Personal Communication, the Capital Region of Denmark). Even though it rained 1, 5 and 2 days before the three analysed time periods, respectively, the positive diurnal residuals do not stem from rainwater harvesting because this would have led to a systematic increase in the in-sewer flows within each sub-catchment across the three time periods. The positive diurnal residuals also do not stem from snow melting, since there was no snow in Elsinore in any of the three considered time periods. Neither rain-induced infiltration nor infiltration from pumped groundwater exhibit diurnal patterns. The deviation between the waterwork outlet and consumed water is more or less constant over the three time periods (Figure 2), but the gap between, for example, Q_obs,ED and Q_sim,ED varies heavily and is on average 14, 7 and 1.5 L/s in October, November and February, respectively, which indicates that the positive diurnal residuals are not likely to stem mainly from infiltration from a leaking water distribution system. This is supported by the fact that leakages in the water distribution system were actually recorded by the utility in EU in October and March (after the investigated periods), but Figure 6 shows signs of outflowing, not inflowing, water in October.

The constant positive residuals for WD are not likely to stem from a constant infiltration from groundwater since infiltration is expected to be larger in February than in October, whereas the observed offset is around 20 L/s in October and November and 10 L/s in February. No repair work was carried out for the sewer pipes in WD between November and February to explain this drop in infiltration. The pipe in which the WD flow sensor was located is heavily influenced by sedimentation, which could result in the constant offset from zero. The sediment was, however, not removed between November and February to explain the drop in the offset level, and it is therefore not expected that the offset is due to sedimentation. However, there is a chance that the amount of sediment may have been affected by the storm water flow in the sewer system.

Neither the positive nor negative residuals likely stem from a wrong conceptualisation of the sewer system, since this would lead to similar residuals within each sub-catchment in all three time periods.

Erroneous in-sewer observations thus remain the only likely explanation for the deviations between Q_obs and Q_sim. The observed flow changes greatly from time period to time period; for example, Q_obs,CD is much larger than Q_sim,CD in October and February, but fits well in the intervening period in November. Looking at October, there is only one day where Q_obs,ED and Q_sim,ED match; however, this is according to Contractor 4 due to faulty velocity observations. Contrarily, the water consumption, as expected, more or less follows the same diurnal pattern throughout October, November and February. Additionally, the assumed erroneousness of in-sewer observations is supported by the fact that Q_obs out of WD, on average, is 21 and 25 L/s larger than Q_sim in October and November, respectively. Adding this amount on top of Q_sim,WWTP would only exacerbate the difference to Q_obs,WWTP (Figure 3). We therefore expect that the in-sewer sensor measurements are the main contribution to the deviation between simulated and observed sewage flows. These deviations are larger than the declared uncertainty of 12% (as supported by Figure 4) and often also larger than what, as stated earlier, is generally considered ‘normal’ deviations of up to 20%.

Figure 7 shows the relationship between the deviation between Q_obs and Q_sim and the time since the last calibration of the in-sewer sensors for the three time periods. No clear correlation exists, either within each time period or when comparing each location across the three time periods. This indicates that the differences are not due to slowly drifting sensors.

Since the sensors were installed as duplicate sensors, it might at first seem unlikely that these observations are as uncertain as indicated by this study. It is, however, worth noting that the duplicate sensors are not truly independent since they are installed at the same location and use the same sensing technology. This means that they are likely to produce the same systemic errors when exposed to the same conditions and thereby confirm each other's misconception of the flow. This could, for example, be the case if there is sedimentation or dislocation of pipes close to where a sensor pair is installed. Truly independent data sources are thus key to anomaly detection and data validation. To fully understand the anomalies detected in this study, it would be necessary to perform further independent measurements, for example:

• -

  Independent and reliable flow observations at the five sensor locations to validate that the in-sewer flow observations indeed caused the deviation to the smart meter-simulated wastewater flow.

• -

  CCTV (Closed Circuit Television) footage and groundwater-level observations to estimate the reason for the constant positive residuals for WD and whether these could stem, at least partly, from groundwater infiltration, sedimentation or dislocated pipes.

These issues could not be investigated in the present study as the measurement campaign in the urban drainage system was terminated in June 2019 but should be considered in future studies focusing directly on closing the urban water balance.

A hydrodynamic 1D model poses an easy way of coupling smart meter data and in-sewer observations, since the model automatically ensures that only wastewater from the consumers upstream from an in-sewer sensor contributes to the flow at the sensor location. Furthermore, it is possible to include infiltration/exfiltration dynamics and thus make a better comparison between the simulated and observed data, which will not be possible when simply summing the smart meter data.

Figure 8 shows Q_obs,CD, Q_SM,CD and Q_sim,CD for 3 days in each of the considered time periods. The timing of the observed flow is considered reliable, despite the magnitude of the wastewater flow being off as a result of the likely erroneous in-sewer observations. It is clear that the routing of the smart meter data through the MIKE URBAN model catches the timing of the observed peaks and low flows much better than simply aggregating the smart meter data. The model can thus be used to capture the time dynamics of the system in dry weather in the five sensor locations without further calibration (however, the model may still need to be calibrated for rain situations). The dynamics shown in Figure 8 are representative of the entire dataset for all five flow observation locations. The difference in time between the summed smart meter data and the simulated flow will naturally be more pronounced the longer the water has travelled in the system and thus depends on the size of the catchment upstream of the flow observations.

Sewer system analysis will likely never be the sole reason for installation of smart meters, which are typically installed for better operation and management of the water distribution system. The urban drainage sector can, however, still take advantage of this data, which is increasingly collected in the future anyway. This study shows that water supply smart meter data can potentially be used to estimate the wastewater flow. The fact that smart meters thus become multi-purposed will likely make their operation more robust. Furthermore, the robustness of the approach also stems from the sheer amount of smart meters, where a failure of a single meter will have much smaller consequences regarding the information level than the failure of a sewage flow sensor.

The use and comparison of data across the water supply, urban drainage and wastewater sectors and the subsequent anomaly analysis performed in this study is a tedious process due to the many aspects affecting both the water distribution and urban drainage networks. Furthermore, it is laborious to systematically gain access to and compare all relevant data sources. This was further complicated by the fact that not one single person could access all the models and data as they were managed and stored in different silos both within the utility and by different contractors. To get the most value out of data, we need to break down these silos in the future and aim for more open standards for data exchange. Many utilities are currently taking the first steps towards using data in new and more integrated ways. This study shows that digitalisation is not easy, and sometimes the data quality remains unknown until the data are used and compared with other data sources. More integrated data analysis and systematic uncertainty assessments are clearly needed to bring this field further ahead. The process of actually using data will provide important learnings regarding good practices within sensing and data accessibility, and enable the utility and their contractors to further refine their work processes.

The coupling between smart meter data and urban drainage models can be done offline for post-analysis of data and system performance (as done in this study). If the model is trustworthy, this coupling could also be done, and the sewer model run in real time, to get an up-to-date picture of the system state. This would open up enhanced real-time data validation as well as entirely new ways of operating sewage infrastructure during dry weather, for example, for improved heat recovery from sewage systems or better sewer system control.

The current study aimed at using smart meter water consumption data to simulate the wastewater flow and to combine this information with in-sewer observations to detect system and data anomalies, such as infiltration, exfiltration and sensor errors.

Smart meter data were validated with data from other independent sources, including data from the waterworks' outflow, WWTP inflows and households' annual water consumption audits for a period prior to the installation of the smart meters. Subsequently, we illustrated the feasibility of using the smart meter data as input to a 1D hydrodynamic sewer model to simulate the wastewater component of the DWF. An estimate of the wastewater flow was thus obtained with a high spatial and temporal resolution. Even though there is still uncertainty related to this estimation, we believe that having this data-based approach to estimate the wastewater flow directly from its source (the water consumption) is an important step towards closing the water balance in urban drainage systems.

The main difference in the results from simply summing the smart meter data and using an urban drainage model is the effect of the routing time in the sewer system. If time dynamics are insignificant, a simple summation may be sufficient. Time dynamics are, however, important when comparing data on a fine time resolution as done in this study. Our comparisons and analysis of observed in-sewer DWF with both smart meter-simulated wastewater flow and WWTP inflow data suggest that the in-sewer flow observations used here may be subject to unrecognised uncertainties at times exceeding the expected 10–20%. These specific in-sewer observations are thus unsuitable as a realistic estimate of the DWF and thus also for decomposition of the DWF into wastewater flow and other input and output sources (such as infiltration and exfiltration). The in-sewer flow sensors were here installed in a very thorough way with two identical flow sensors at each location for validation, but it cannot be excluded that both sensors were influenced by the same error sources under the same conditions. This underlines the necessity of using truly independent data sources from different measuring techniques for future data quality assessments.

The main take-away messages from this study are:

• 1.

  Using smart meter data as input to hydrodynamic urban drainage models is feasible and promising as a means of estimating the wastewater flow directly from water consumption data. This is an important step towards closing the water balance in urban drainage systems.

• 2.

  Installing multiple in-sewer flow sensors of the same type at the same location is not enough to assess the data quality. Including independent data sources using different sensor types in future investigations is paramount.

• 3.

  It is a tedious task to compare and analyse data from multiple sources. However, the only way to start breaking down data silos and realise the real quality of collected data is to start using it in integrated studies as exemplified in the case study.

We would like to thank Andreas Mikkelsen, Lotte Krogh, Brian Laursen and their colleagues at Utility Elsinore for contributing with data, models and expertise. Furthermore, thank you to Lina Jensen (LNH water), Kristoffer Andersen (Kamstrup), Oliver Jodehl and Tommy Petersen (BlueNorth), and Rasmus Halvgaard (DHI) for professional discussions related to the data and models used.

We declare no conflict of interest.

Data cannot be made publicly available; readers should contact the corresponding author for details.