Acoustic feature-based leakage event detection in near real-time for large-scale water distribution networks

Reducing water losses in water distribution networks (WDNs) is of great significance to control carbon footprints and improve infrastructure sustainability (Wu et al. 2011). While water losses comprise apparent and physical components, it is the latter, due to hidden pipe leakages and/or other anomaly events (Mounce et al. 2010; Guo et al. 2021), which pose the biggest challenge for water utilities. Hidden leakages in water pipelines contribute significantly to the total non-revenue water (NRW) (Cassidy et al. 2020) in WDNs and they can be classified into reported, hidden, and background leaks. Among them, hidden leaks are the most difficult to manage as the unaccounted water does not appear readily on the open streets and can remain insidious before becoming disruptive events to the public.

Over the years, different techniques have been developed to assist field engineers to detect and localize hidden leakage events in WDNs. For example, the use of physics-based hydraulic models to analyse flow and pressure data (Wu et al. 2010, 2021; Pérez et al. 2011; Goulet et al. 2013; Gerard et al. 2016; Gianfredi et al. 2017; Chew et al. 2022), and device-based methods which collect leak acoustic signals in water pipelines (Khulief et al. 2012; Muntakim et al. 2017; Butterfield et al. 2018a, 2018b; Cody et al. 2020; Guo et al. 2021). Other leak detection methodologies include Bayesian updating, coupled with optimal distribution methods (Alawadhi & Tartakovsky 2020), transient test-based techniques (TTBTs) (Meniconi et al. 2021), and trend-change analysis (Xue et al. 2022).

Over the last decade, acoustics sensors have been increasingly deployed by utility companies as part of their 24/7 permanent monitoring or temporary leakage program(s). Typically, an acoustic signal, as caused by pipe leakages, is due to the complex interaction between the flowing water and interiors of the underground pipe wall(s), hence generating random wave signals with both short-term nonstationary and long-term stationary frequency components (Almeida et al. 2014). In the practical field context, the emitted acoustic signals are, however, mixed with a range of environmental noises which can lead to a low signal-to-noise ratio (SNR) (Wen et al. 2004; Guo et al. 2021). Thus, it is imperative to first pre-process raw acoustic signals, followed by leveraging the processed data to detect possible hidden leakages underground. Examples of acoustic data pre-processing techniques involve decomposing waveform profiles into intrinsic mode functions (IMFs) via empirical mode decomposition (EMD) (Norden et al. 1998), or its improved version in ensemble empirical mode decomposition (EEMD) (Wu & Huang 2009; Luukko et al. 2016).

On the whole, leakage detection in WDNs via acoustic signal data analysis can be broadly grouped into two main categories, namely: (a) traditional signal processing-based methods and (b) data-driven methods using machine learning. Table 1 summarizes the key methods and findings reported from previous notable studies in the current literature.

While numerous experimental works have been performed to develop signal processing-based methods for detecting hidden leaks in underground pipes, they mainly deployed a high density of acoustic sensors to effectively capture the leak acoustic signals within a near spatial distance range. For example, Mark et al. (2020) reported that accelerometers could detect leaks within 20–50 m for their case study analysis in Adelaide. Similar findings have also been shared by Zhang et al. (2020). While both field studies/investigations may represent the actual operations of real-world WDNs, as compared with lab-scale or experimental studies, the high sensor density requirement is not likely to represent long-term acoustic monitoring in most large-scale WDNs, where sensors are typically installed permanently and sparsely, as exemplified in our subsequent case study analyses.

For data-driven modelling using machine learning methods, despite its well-known effectiveness, an important pre-condition is to have sufficiently large and balanced acoustic datasets in terms of the total number of leakage and non-leakage data records for effective model training. However, such data prerequisites may not be possible in all well-managed WDNs, where the number of historical leakage records are likely to be less than that of the non-leakage category. In addition, it is worth noting that most of the well-documented acoustic data-driven studies for leak detection in WDNs (Kang et al. 2018; Cody et al. 2020; Guo et al. 2021) were mainly carried out under small-scale experimental or controlled field tests with sufficient deployment of acoustic sensors (see Table 1). To handle the data constraint issue for leak detection using acoustic data, Ravichandran et al. (2021) recently developed a novel multi-strategy ensemble learning (MEL) approach that exploited 48 acoustics-based features which were extracted from historical power spectral density (PSD) and time-series profiles. While the authors demonstrated the effectiveness of MEL to reduce the total number of detected false positives by an order of magnitude, their research analysis deployed a large number of acoustic sensors which thus increased the likelihood/probability of detecting the pipe leaks, as compared with the context of sparse deployment of acoustic sensors for the same practical purpose.

To mitigate the requirement of large acoustic datasets for effective model training, the need to identify useful acoustic data features for leak detection has since garnered some research interest. For example, Kothandaraman et al. (2020) coupled conventional cross-correlation analysis with EMD to detect and localize pipe leakages underground, and Ning et al. (2021) recently developed a useful framework that couples EEMD and random forest algorithm to classify the different types of leaks. While these studies have quantitatively demonstrated the usefulness of EMD or EEMD pre-processing methods to remove ambient noises embedded in the acoustic signals, we again highlight that they were mainly collected under controlled lab-scale experiments where the frequency of the environmental noises (e.g., blowing fans, pump noise) can be identified easily and removed/filtered to extract the most useful leakage acoustic frequency. It is therefore still a major engineering challenge to process complex real-world acoustic signals to best identify useful leakage acoustic characteristics for detecting leaks in WDNs.

To address the above-discussed challenges, and shortcomings from previously reported research studies, for leak detections in large-scale WDNs having permanently and sparsely installed acoustic sensors, this technical paper aims to develop an effective acoustic feature-based methodology by encompassing: (1) data quality assessment, (2) data decomposition, (3) features identifications, (4) model training for outlier detection and classification, and (5) near real-time leakage detection. In collaboration with PUB, Singapore's National Water Agency, our proposed approach has since been verified with historical acoustic signal datasets collated over 13 months between 2019 and 2020 for large-scale operational WDNs with a total pipe length of 1,000 km in Singapore.

A spectrogram (see Figure 1(b)) comprises two dimensions where the x-axis represents time (seconds), while the y-axis represents the frequency (Hz) of the acoustic signal. An additional third dimension, as represented by normalized colour intensity values in decibels (dB), quantifies the signal strength at a specific frequency value where brighter colours represent a higher power range, and vice versa for darker colours. PSD profile (see Figure 1(c)) analyses the power density distribution of the same signal over its frequency range, where its x-axis represents the signal's frequency (Hz) and the y-axis represents the corresponding power density (db/Hz) values across the frequency values.

During the operations of WDNs in the practical field context, a mixture of unknown environmental noises, caused by connected pumps, valves, pipe bends, road traffic, and weather conditions, is expected to be embedded in the collected acoustic signals. It is also common that permanently installed sensors do not function correctly in the field at all times, hence resulting in numerical errors being introduced into the recorded acoustic readings over time.

Physically, it is expected that the acoustic power caused by the leakage condition will be greater than that of the baseline non-leakage condition. To reasonably approximate this leakage acoustic power, it is imperative to first identify the key leakage acoustic data features which can be derived from the decomposed data signals, namely, (1) specific IMF component(s) and (2) dominant frequency range, which can best represent the leakage acoustic characteristics. We thus propose a power feature that models the signal acoustic power computed using a specific IMF component (and its subsequent PSD profile) for a defined frequency range after performing the EEMD step. The general steps to compute for any given IMF component are as follows: (1) convert IMF waveform profile into its corresponding PSD and (2) sum the power density values in the derived PSD profile for a defined frequency range using a simple trapezoidal rule.

To develop a general methodology, a solution workflow process is thus proposed to programmatically identify the specific IMF component(s) which can best capture the dominant leakage characteristics/frequency for any historical leakage event reported in the modelled WDN. The details are as follows:

• i.

  For each modelled leakage event, we first identify the acoustic sensor located nearest to the reported event location within the modeller-defined pipeline distance range, as typically measured in metres (m). If this condition is met, proceed with the subsequent steps.

• ii.

  Leak PSD (⁠⁠): Derive the time-averaged PSD profile for selected timestamps (e.g., 2 am–4 am) of the individual IMF component(s) pertaining to the historical leakage date(s).

• iii.

  Non-Leak PSD (⁠⁠): Using the same IMF component(s), derive their time-averaged PSD profile for the identical timestamps pertaining to the baseline non-leakage date(s).

• iv.
  For every selected IMF from the previous steps (ii–iii), compute the level of similarity between their associated and profiles using: (a) cross-correlation ⁠, as defined in Equation (1); and (b) coefficient of determination ⁠, as defined in Equation (2). If no frequency lags are considered, in (1) essentially represents the conventional correlation coefficient value in statistical analysis.

  

  (1)

  

  (2)

  where N is the total number of data points within the PSD profile, represents the profile, and represents the profile.

• Leakage and non-leakage pipe conditions are thus differentiated via the highest dissimilarity between and that can be quantified by lower and values for a selected IMF component. To avoid a large number of enumerations, we consider IMFs 2–4 as the arbitrary starting selection. If IMF-4 can quantitatively be shown to have limited observable leakage signals, then the remaining higher-order IMFs are most unlikely to be useful to capture the leakage condition.

Upon identifying a dominant IMF data feature that may best represent the leakage condition, we then proceed to identify a common frequency range associated with the identified data feature via the following solution steps:

• i.
  For each historical leakage event being modelled, identify a sequential frequency range having the largest area under the curve (AUC) difference between and profiles.

  

  (3)

  where represents the difference between and for a certain frequency range ⁠.
• ii.
  For the frequency range identified, estimate the rate of change in the power density values, with respect to frequency (Hz) by using the following formulae:

  

  (4)
• iii.

  Locate a critical frequency value (⁠⁠) within that has the largest associated positive value.

The IMF(s) and frequency range identified for the dominant leakage acoustic characteristics are further leveraged to generate the values, followed by using them for subsequent model training and NRT analysis (see Figure 2) for leak detection. The resulting model performances obtained from using will then be compared against those of two other common acoustic power features in and ⁠. Note that is computed by directly converting the waveform (without applying EEMD beforehand) into its corresponding PSD profile, followed by summing the power density values across its entire frequency range. is simply estimated by locating the minimum power density value of the PSD profile produced from the original waveform without applying EEMD during the data processing step.

Outlier detection is first performed at the local station level by using audio files from the MNF hours between 2 am and 4 am daily where environmental noises are expected to at a minimum. The proposed outlier detection method depends on a defined reference time-window which considers the historical leakage event date, modeller-defined lead-time and lagged-time from the reported event date, and a selected total time-window size ⁠. All proposed time windows are measured in days (e.g., 24 h, 72 h) for the purpose of leakage detections in WDNs modelling and analysis. The selected and values for any distribution network are based on the modeller's level of understanding of the network's operational complexity and communications with the utility company. In our subsequent case study analyses, and are set as 3 days and 1 day, respectively, based on the agreed recommendations with our collaborator, PUB, Singapore's National Water Agency.

After the outliers are detected for each individual sensor station (i), they are then clustered and classified into systemwide leakage events by using a set of user-defined temporal proximity criteria as follows.

• Basic Criterion 1 (BC-1): Total number of stations having detected outliers on any random day must be greater than the minimum number of stations having outliers (⁠⁠).

• Basic Criterion 2 (BC-2): Total number of detected outliers between 2 am and 4 am for each sensor station on the same random day must be greater than the minimum number of outliers at each station (⁠⁠).

• Advanced Criterion (AC): Total number of detected outliers from all stations must be greater than the minimum number of outliers (⁠⁠).

• Time Gap Criterion (TGC): Maximum time difference between two adjacent detected outliers within each event cluster must be less than the maximum allowable time gap (⁠⁠).

To initiate near real-time leakage event detection, upon training a systemwide detection model for a given set of acoustic sensor stations, several key model parameters, as listed below, must be tuned to train and optimize the model's detection performance at the systemwide level.

• Pre-select different combinations of ⁠, ⁠, ⁠, ⁠, and m to optimize the detection model's F1-score during model training.

• For each combination of the model parameters, perform checks for BC-1 and BC-2 daily. Note that at the starting date of the training phase, the system's current cluster size is set as 0.

• On a daily basis during the model training period, both BC-1 and BC-2 must be fulfilled, followed by adding the total number of detected outliers from all possible stations into ⁠. If either BC-1 or BC-2 is not fulfilled on any day, AC and TGC checks are checked if their respective criterion is fulfilled for ⁠. Once this condition is met, is then reset to 0 on the following day for continuing the detection analysis.

• At the end of the model training phase, classify the identified event cluster(s) into TP or FP event(s) accordingly by using the defined for each of the historical leakage event(s) being modelled during the training phase. The model's resulting performance scores are then computed using Equations (9)–(11) at the systemwide level.

A total of 236 historical leakage events, ranging between 1 August 2019 and 31 August 2020, were reported for both modelled WDNs. Due to their size scale, there was generally a low probability for each historical leakage event to be situated close to a permanently installed hydrophone, especially since the events do not occur daily and the number of installed hydrophones is less than the total number of historical events. Tables 3 and 4 summarize the list of hydrophones deployed in both zones having historical leakage event(s) reported to be within an arbitrary 450 m, or less, pipeline distance from the nearest station, hence at least 90% of the total reported events in both zones were located more than 450 m away from the nearest hydrophone station(s). Note that the threshold pipeline distance is again a user-defined parameter. Finally, Table 5 summarizes the details of the audio files collected across all hydrophones in both zones for the above listed period.

The reported leakage events, as summarized in Tables 3 and 4, are then analysed to identify the key leakage acoustic features, namely: (1) dominant IMF component and (2) leakage frequency range.

For each of the listed cases in Tables 3 and 4, we adopt of 5 days that encompass the reported leakage date while assuming and of 3 days and 1 day, respectively. Hence, a total of 5 profiles are generated to represent the pipe leakage condition, where each profile is obtained by considering the MNF hours of 2 am to 4 am for each assumed leak day within the defined period. The singular profile is then derived by averaging across the remaining days ranging between 1 August 2019 and 31 August 2020 (excluding the leak window dates) outside of the period, as subject to the audio files availability and data quality assessment.

Systemwide leakage event detection and classification analyses are then performed for the historical leakage events reported between 1 August 2019 and 31 August 2020 by using 100–750 Hz range, coupled with either the IMF-2 or IMF-3 component. In both modelled WDNs, the 13-month period is first divided into 75% for model training (T) and the remaining 25% to emulate the near real-time (NRT) context for the detection analysis. Readers are referred to Table 6 which summarizes the details for the model's T and NRT analyses in the respective zones.

Using IMF-2 and IMF-3 with the common frequency range of 100–750 Hz, we further iterate the model parameters of ⁠, m, ⁠, and ⁠, while maintaining ⁠. On the daily basis, the model collates the detected outliers between 2 am and 4 am across all acoustic sensor stations in the respective zones. The resulting system cluster size is then represented by the respective sum of all detected outliers over a period until any of BC-1 or BC-2 is not met for the model's T phase, followed by checking against AC and TGC criteria (see Figure 11).

The model performances are susbsequently compared for the respective zones using the different data features proposed. Tables 9 and 10 summarize the optimized model configurations which can attain the highest possible balance for the computed values from the combined T and NRT phases, while attaining a reasonable size in the model's T phase. Note that for all model runs is again kept at 24 h. In each of the zones, we compare the detection results obtained from using different power features, where the following observations can be derived:

• with IMF-2 component can detect leakage events with more than 70.0% score, especially for Zone-1. In Table 9 for Zone-1, the resulting values from the NRT analysis using and ⁠, even with their optimal model configuration(s), are lower than that of ⁠.

• For Zone-2, all power features result in optimal values for the model's T and NRT phases, as shown in Table 9. However, for the same reasons cited previously, the present results are unlikely to provide useful insights for well-managed WDNs.

• The average lead-times for detecting the reported leakage events across all model configurations in Zone-1 and Zone-2 are 0.9–1.0 days and 0.6 days, respectively, prior to the actual leakage events reported by PUB's current technique in practice. At the same time, the average lead-times also appear to be closer to one another when using for both zones, as compared with that of and ⁠.

Table 9

Comparison of model performance using (IMF-2), ⁠, and for detection analysis in Zone-1

Power feature .	Phase .	Optimal⁠, ⁠, ⁠, .	No. TP events .	No. FP events .	Maximum cluster period (days) .	.	.	.	Avg. lead-time (days) .
	Ta	10, 2.3, 3, 1	9	3	8	75.0%	73.1%	74.0%	0.9
	NRTb	10, 2.3, 3, 1	20	2	5	90.9%	60.0%	72.3%	0.7
	T	10, 1.0, 13, 1	12	6	6	66.7%	76.9%	71.4%	1.1
	NRT	10, 1.0, 13, 1	0	0	3, 5, 7	NA	0.0%	NA	NA
	T	7, 1.4, 3, 1	11	20	7	35.5%	65.4%	46.0%	1.2
	NRT	7, 1.4, 3, 1	33	20	7	62.3%	70.0%	65.9%	0.8

Power feature .	Phase .	Optimal⁠, ⁠, ⁠, .	No. TP events .	No. FP events .	Maximum cluster period (days) .	.	.	.	Avg. lead-time (days) .
	Ta	10, 2.3, 3, 1	9	3	8	75.0%	73.1%	74.0%	0.9
	NRTb	10, 2.3, 3, 1	20	2	5	90.9%	60.0%	72.3%	0.7
	T	10, 1.0, 13, 1	12	6	6	66.7%	76.9%	71.4%	1.1
	NRT	10, 1.0, 13, 1	0	0	3, 5, 7	NA	0.0%	NA	NA
	T	7, 1.4, 3, 1	11	20	7	35.5%	65.4%	46.0%	1.2
	NRT	7, 1.4, 3, 1	33	20	7	62.3%	70.0%	65.9%	0.8

^aT – Model training step (28 historical leakage events).

^bNRT – near real-time analysis (10 historical leakage events).

View Large

Table 10

Comparison of model performance using (IMF-2), ⁠, and for detection analysis in Zone-2

Power feature .	Phase .	Optimal⁠, ⁠, ⁠, .	No. TP events .	No. FP events .	Maximum cluster period (days) .	.	.	.	Avg. lead-time (days) .
	Ta	3, 1.2, 7, 1	35	2	9	94.6%	81.0%	87.3%	0.8
	NRTb	3, 1.2, 7, 1	39	0	5	100%	83.6%	91.1%	0.8
	T	3, 1.6, 7, 1	39	4	9	90.7%	91.2%	91.0%	1.2
	NRT	3, 1.6, 7, 1	16	0	3	100%	49.2%	65.9%	0.4
	T	3, 3.8, 2, 1	38	2	9	95.0%	83.9%	89.1%	0.8
	NRT	3, 3.8, 2, 1	56	1	3	98.2%	93.4%	95.8%	0.1

Power feature .	Phase .	Optimal⁠, ⁠, ⁠, .	No. TP events .	No. FP events .	Maximum cluster period (days) .	.	.	.	Avg. lead-time (days) .
	Ta	3, 1.2, 7, 1	35	2	9	94.6%	81.0%	87.3%	0.8
	NRTb	3, 1.2, 7, 1	39	0	5	100%	83.6%	91.1%	0.8
	T	3, 1.6, 7, 1	39	4	9	90.7%	91.2%	91.0%	1.2
	NRT	3, 1.6, 7, 1	16	0	3	100%	49.2%	65.9%	0.4
	T	3, 3.8, 2, 1	38	2	9	95.0%	83.9%	89.1%	0.8
	NRT	3, 3.8, 2, 1	56	1	3	98.2%	93.4%	95.8%	0.1

^aT – Model training step (148 historical leakage events).

^bNRT – near real-time analysis (50 historical leakage events).

View Large

The summarized detection results in Tables 9 and 10 highlight the level of effectiveness in our proposed acoustic feature-based approach to detect hidden leak events in both Zone-1 (industrial activities) and Zone-2 (mixture of commercial and domestic activities) with an average 70% F1-score and 1-day lead-time. To ensure that the approach can generally be extended to other WDNs having different operational configurations, we bring forth two major discussions as follows:

• Overall applicability of the proposed model approach: In the context of near real-time analysis, the proposed acoustic feature-based approach can be applied by operators to detect and classify hidden leaks in the modelled WDN, even under the constraints of sparse acoustic sensors deployment. For example, on a daily basis by using the proposed protocol in Figure 11, operators will first initiate the outlier detection analysis during MNF hours (2 am–4 am) by using acoustics data at the individual station level and accumulate the total detected outliers across all detected acoustic sensors to form a single systemwide event cluster that grows over time. If the same event cluster lasts for a minimum number of consecutive days (e.g., 3 days), as demonstrated in Figures 15 and 16 for the respective zones, then the field maintenance team may proceed to investigate the likely location(s) of the detected hidden leaks in the system by searching the pipes near to the sensor station(s) which detect the outliers over the same number of consecutive days. In addition, we note that the adopted assumptions for the average reference acoustic power (e.g., minimum 30% normalized power) and acoustic frequency range (e.g., 100–750 Hz) in our current study are generally tunable by the operators, as based on their level of quantitative understanding of the modelled system's operational complexity.

• Overall generality of the proposed model approach: The proposed feature-based approach should also work in all contexts having deployed hydrophone stations in the underground pipes. Due to the direct submersion of the hydrophones into the flowing waters within the underground pipes, the generated acoustic signals are expected to be independent of the pipe material that traditionally can affect the speed of sound travelling through the material. Hence, the focus is to iterate on the optimal frequency range, independent of the type of pipe material, that can best benefit the leakage detection analysis for any modelled WDN. As for pipe flow, that can be associated with the pipe diameter, we have quantitatively demonstrated, from our case study analyses, the general usefulness of using recorded acoustic signals from MNF hours (2 am–4 am) to optimize, to the best possible extent, the leak detection results. Importantly, we again note that our proposed method is generic and flexible which enables modellers to fine tune the different model parameters (e.g., reference acoustic power, leakage frequency range) for any given acoustic dataset from the modelled system to ensure adequately accurate leak detection results during the model training phase.

This paper presents a generalized acoustic feature-based approach to detect and classify leakage events in large-scale WDNs having permanently installed acoustic sensors in sparse locations of the large networks. The proposed approach consists of a series of systematic analytical components, namely: (1) data quality assessment; (2) data decomposition; (3) features identification; (4) model training for leakage event detection and classification; and (5) near real-time leakage detection.

In collaboration with PUB, Singapore's National Water Agency, the proposed approach is verified with large-scale WDNs in Singapore having around 1,000 km of underground water pipelines installed with 74 permanently installed hydrophones in sparse locations of the large networks. Multiple historical leakage events were found to be situated close to independent hydrophones within 450 m pipeline distance, where a common leakage frequency range of 100–750 Hz embedded within the IMF-2 data feature is found to best represent the dominant leakage acoustic characteristics in the modelled WDNs. At the subsequent systemwide leakage event classification step, our proposed temporal-based clustering analysis can detect and classify reported leakage events with an overall F1-score of more than 70%, while emulating the near real-time (NRT) analysis with an average leakage detection lead-time of 0.5 days.

This research is supported by the Singapore National Research Foundation under its Competitive Research Programme (CRP) (Water) and administered by PUB (PUB-1804-0087), Singapore’s National Water Agency.

Data cannot be made publicly available; readers should contact the corresponding author for details.

The authors declare there is no conflict.