Data-driven predictive analysis and visualisation of air–water dynamics in an air vessel

Water and sanitation systems are crucial for public health and sustainable development, particularly SDG6. Global challenges in this sector require greater ambition and innovation, especially through advancements in science and technology. These challenges span a wide range of issues, including enhancing service levels, addressing the role of sanitation in antimicrobial resistance, and building climate resilience (Howard 2021). This study focuses on water systems, which face a multitude of interconnected challenges such as ageing infrastructure, high maintenance costs, and significant energy consumption. These challenges often manifest in various forms, such as water loss and inefficient operations. Smart Water Systems (SWS) offer a promising solution by leveraging advanced technologies to address these issues and improve operations, management, and decision-making (Xiang et al. 2021). SWS can be defined in various ways, from a data analysis tool to a comprehensive transformation of decision-making (Xiang et al. 2021). Ingildsen & Olsson (2016) defines SWS as a real-time data-driven decision-making system spanning the entire water cycle, optimising water quality and quantity while minimising resource consumption.

The current study adopts a data-centric definition of SWS as an intelligent framework across the water cycle, improving sustainability through rapid analysis and decision-making, emphasising real-time data analysis as a fundamental aspect. Therefore, this research focuses on machine learning and predictive analysis for complex water system data.

Water loss, particularly leakage, serves as a pertinent example of the complex challenges facing water systems. While significant, it is one facet of a broader set of issues. Water scarcity is increasing globally, with urban residents in water-scarce regions projected to rise dramatically by 2050 (He et al. 2021). Current trends suggest that a significant portion of the global population could face water scarcity by 2025 (Eliasson 2015; Abd Rahman et al. 2018). Non-revenue water (NRW), the difference between supplied and billed water (Frauendorfer & Liemberger 2010), often exceeds 40% in cities worldwide, and even higher in developing countries (Abd Rahman et al. 2018). While factors such as metering issues and water theft contribute to NRW, pipe leakage is a significant component (Ramos et al. 2023). The scale of water loss is substantial, with millions of cubic metres lost daily in the developing world (Bell 2016). Addressing water loss is important, but it must be viewed within the context of broader system optimisation.

Mitigating the various challenges facing water systems, including water loss, is complex. Water networks are vast and often underground, making observation and analysis difficult (Kim et al. 2016). Various techniques exist, including pressure management, active leakage control, leak detection, smart metering, and asset management (Ramos et al. 2023). Machine learning can play a crucial role in addressing these challenges, including those related to NRW (Conejos Fuertes et al. 2020). These challenges can stem from factors such as ageing pipes, geotechnical issues, faulty installation, high pressures, and transient events such as water hammer (Covas et al. 2005). Amongst them, transient events are particularly challenging, potentially causing pump failures, pipe ruptures, and other malfunctions (Boulos et al. 2005; Besharat et al. 2018). Factors such as entrapped air can exacerbate these events (Izquierdo et al. 1999; Boulos et al. 2005). Transient flows are unavoidable due to operational events like valve closures and pump activity (Chaudhry 2014).

Modelling transient flows, which is computationally intensive and depends on numerous variables and dynamic system characteristics (Chaudhry 2014), presents a significant challenge. Real-time monitoring and prediction of these variables are particularly difficult, especially within large and complex networks (Boulos et al. 2005; Chaudhry 2014). Furthermore, existing transient management methods often concentrate on mitigating the consequences of transients rather than preventing their occurrence (Chaudhry 2014). The dynamic nature of flow parameters, coupled with the inherent complexity of water networks, necessitates the development of responsive, real-time modelling and prediction capabilities. These challenges are further exacerbated by the presence of ageing infrastructure (Sanders et al. 2022), as water companies strive for significant improvements in overall water management (Sanders et al. 2022). However, technological advancements are now enabling more precise monitoring and analysis of water processes through the utilisation of real-time sensor data (Sanders et al. 2022), leading to a better understanding and modelling of previously uncertain phenomena. Smart networks, coupled with advanced sensors and analytics, are therefore key to achieving increased efficiency (Sanders et al. 2022). This approach is strongly supported by Water UK’s routemap (Sanders et al. 2022), which advocates for the integration of these technologies to reduce costs and improve system efficiency. This strategic direction aligns with the World Bank’s Utility of the Future program (Lombana Cordoba et al. 2022), which emphasises a new strategic management approach for water utilities worldwide.

Machine learning, particularly deep learning, has shown significant potential in tasks such as image classification and predictive analysis (Goodfellow et al. 2014). Deep learning models, such as convolutional neural networks (CNNs), have been applied to fluid dynamics, helping predict fluid flow characteristics and improve the management of water systems (Morimoto et al. 2021). Recent advancements in machine learning can also support the development of digital twins, providing real-time, high-fidelity representations of water systems, which are invaluable for monitoring and predicting failures (Wu et al. 2023). Furthermore, machine learning techniques are being applied to optimise various aspects of water systems. For example, Jafari-Asl et al. (2024) introduced a novel optimisation algorithm for pumping stations to reduce water loss and energy consumption. In hydrological forecasting, Khosravi et al. (2024) developed an ensemble machine learning model for enhanced daily river flow prediction. Machine learning is also proving valuable in broader environmental modelling, as demonstrated by Donnelly et al. (2024) who combined autoencoders and Gaussian Processes for improved spatiotemporal regression of global temperature and pressure. Khosravi et al. (2023) reviewed the applications of deep learning in hydrology, including rainfall–runoff simulation and soil moisture prediction, highlighting its potential for improved land management.

Existing research on SWS includes large-scale hydraulic models, but these often rely on simplifying assumptions (Tomić et al. 2022). While sophisticated unsteady flow models exist (He et al. 2022), their high computational demands limit applicability to large infrastructure. Simplified models, meanwhile, can be inadequate for real-time operations like pressure management (Vicente et al. 2015). This study addresses these limitations by developing machine learning and image processing models for transient flow prediction, trained on limited experimental data. Specifically, experimental data from transient flow tests – including water hammer induced by rapid valve closure – are used to predict pressure variation and air–water mixture patterns within an air vessel. The key innovation lies in achieving accurate predictions with minimal data including visual and numerical, significantly reducing computational intensity. This potentially enables practical real-time SWS implementation. The trained models and methodologies are detailed extensively withing the paper following a standard structure where the Methodology section outlines the experimental data and analysis techniques, the Results and Discussion sections present findings and implications, and the Conclusion summarises key achievements and future research directions.

This section introduces the data acquisition system, explains the structure of the data used in the research, and discusses the methods employed. The study focuses on the water hammer event, a computationally intensive phenomenon for simulation, which can trigger multiple issues in pipeline systems. Water hammer in pipelines refers to a sudden increase in pressure caused by an abrupt stop or change in the flow of water. This phenomenon generates hydraulic shock waves, leading to pressure surges that can potentially damage the pipeline system (Chaudhry 2014; Feng et al. 2024). A common solution for controlling the excessive pressure spikes caused by water hammer events is the use of air vessels, which help absorb and mitigate these surges (Besharat et al. 2016). The pressure surge is managed by the compression of gas within the air vessel, which helps dampen the shock. In addition to the design and sizing of air vessels, which directly benefit from accurate calculations of the pressure inside the vessel, the overall operation of the pipeline systems also benefits from predicting pressure variations within the air vessel. This ensures the system operates safely and without significant risks. For large-scale systems, conventional pressure calculations based on theoretical approaches may not be feasible due to the high computational load. Therefore, a predictive analysis approach proves to be quite beneficial. Accordingly, this section will delve into deep learning models, providing detailed information about the specific techniques utilised. The discussion will then transition to image-based models, which serve as another tool employed in the pursuit of predictive analysis within this study.

Tests encompass various air pocket sizes and flow velocities, including seven air pocket lengths (2, 3, 4, 5, 10, 20, and 40 cm) and seven flow velocities with corresponding Reynolds numbers (36,000, 56,000, 75,000, 93,000, 115,000, 132,000, and 155,000). These ranges resulted in a set of 11 tests, each containing 7 sub-tests, totalling 77 sub-tests, as demonstrated in Table 1.

Table 1

Sub-test parameters: Re number and air pocket size

Test Number .	1 .	2 .	3 .	4 .	5 .	6 .	7 .	8 .	9 .	10 .	11 .
Air Pocket size (cm) .	0 .	2 .	3 .	4 .	5 .	7 .	10 .	15 .	20 .	30 .	40 .
Re 36,000	1	8	15	22	29	36	43	50	57	64	71
Re 56,000	2	9	16	23	30	37	44	51	58	65	72
Re 75,000	3	10	17	24	31	38	45	52	59	66	73
Re 93,000	4	11	18	25	32	39	46	53	60	67	74
Re 115,000	5	12	19	26	33	40	47	54	61	68	75
Re 132,000	6	13	20	27	34	41	48	55	62	69	76
Re 155,000	7	14	21	28	35	42	49	56	63	70	77

Test Number .	1 .	2 .	3 .	4 .	5 .	6 .	7 .	8 .	9 .	10 .	11 .
Air Pocket size (cm) .	0 .	2 .	3 .	4 .	5 .	7 .	10 .	15 .	20 .	30 .	40 .
Re 36,000	1	8	15	22	29	36	43	50	57	64	71
Re 56,000	2	9	16	23	30	37	44	51	58	65	72
Re 75,000	3	10	17	24	31	38	45	52	59	66	73
Re 93,000	4	11	18	25	32	39	46	53	60	67	74
Re 115,000	5	12	19	26	33	40	47	54	61	68	75
Re 132,000	6	13	20	27	34	41	48	55	62	69	76
Re 155,000	7	14	21	28	35	42	49	56	63	70	77

View Large

Tests are conducted by rapidly closing and opening a fast-closure electro-pneumatic Ball Valve – labeled as the Main Operating Valve in Figure 1 – to induce a water hammer event in the system. The valve’s closing-opening action is controlled by an electrical trigger, with each maneuver lasting 0.20 seconds. This actuation generates a pressure surge that travels upstream. A check valve is installed in line with the pipe upstream of the air vessel to prevent water from flowing back while directing it toward the air vessel. Another check valve at the entrance of the air vessel opens toward the air vessel and closes toward the pipeline. This configuration allows water to enter the air vessel but prevents any flow from returning to the pipeline. The setup is designed to accumulate pressure within the air pocket to study its behaviour under fully confined high-pressure conditions. Consistent initial pressure conditions are maintained across tests, with the initial air pocket pressure set to atmospheric pressure.

The quantitative data have been meticulously divided into training and test sets for machine learning purposes to ensure a broad selection of Reynolds numbers for a comprehensive representation. The test set constitutes of the available data, with the remaining forming the training set.

The image segmentation model known as U-Net (Ronneberger et al. 2015) consists of an Encoder and a Decoder. The encoder extracts features from the input image through a series of convolutional layers. Each convolutional layer acts as a filter to detect different features of the image. By passing the image through multiple convolutional layers, the encoder transforms the image into lower-dimensional feature representations. The decoder then takes these encoded feature representations and reconstructs the original input image through a series of upsampling and convolutional layers. Each step in the decoder uses the output of the corresponding encoder step to guide the reconstruction. This helps the decoder to produce an accurate mapping from the encoded features back to the original input image. Within the current model, the encoder has three blocks. In the first block, two convolutional layers with 64 filters are applied to the input image of size ⁠. This is followed by max pooling to downsample the feature maps to ⁠. The second block takes these feature maps as input, applies two convolutional layers with 128 filters, and downsamples to ⁠. The third block applies two convolutional layers with 256 filters and downsamples to ⁠. After two more convolutional layers with 512 filters, the original image is encoded into 512 feature maps of size ⁠. This compressed representation captures the essential features needed to reconstruct the input image.

The features were selected on the basis of model requirements and normalised using Min-Max scaling to ensure uniformity and facilitate model convergence. Furthermore, to optimise model performance and training stability, a custom Huber loss function was used. This loss function is less sensitive to outliers in the data and seeks a balance between the mean squared error and the mean absolute error. By minimising this hybrid loss, the model was effectively trained to make accurate predictions while mitigating the influence of anomalous data points, which is common when using realistic measured flow and pressure data.

In Model #6 (Figure 7(b)), an additional input feature was introduced, i.e., air percentage in the vessel, dynamically calculated using high-speed camera recordings and as the output of the discussed segmentation network. Specifically, the segmentation model outputs pixel-wise segmentation masks with three channels representing air, mixture, and water regions, which are post-processed to compute the proportional volumes of each phase. These computed proportions are normalised and used as input features alongside conventional parameters such as upstream and downstream pressures, fluid velocity, and time. This integration bridges the gap between discrete sensor readings and continuous spatial visual data, enabling the model to leverage both numerical measurements and visual information for pressure prediction. The model retained the architecture of Model #5 but incorporated this additional processed feature.

Model #7 (Figure 7(c)) extended upon Model #6 by including another additional input feature, namely the water percentage in the vessel, also calculated and normalised based on the segmentation of camera recordings. Other details in the model are similar to the Model #5. The trained models were evaluated on separate test datasets, including interpolation and extrapolation subsets, to assess their generalisation ability. Evaluation metrics such as loss and R-squared were calculated to quantify model performance. The predictions were compared with actual pressure values to visualise model accuracy.

The analysis follows a systematic evaluation of both the image segmentation and pressure prediction components of the deep learning framework. This two-stage approach first validates the accuracy of visual feature extraction before assessing how these features contribute to pressure predictions, which allows for verification of each component’s contribution to the overall methodology.

The training curves show that as the epochs increase, the accuracy of the model steadily improves while the loss decreases. Although this might be an overfitting case, considering the difficulty of the manual segmentation, it was not feasible to create additional validation samples, so the decision was made to use early stopping at epoch 20 to minimise overfitting issues. Visual inspection of the model revealed that further training brings negligible improvement. Rapid convergence within around 20 epochs highlights insufficient data, as the limited 675-image dataset cannot support extensive training. For more robust predictions, an expansion of the dataset is necessary.

With the training completed, the model was applied to new images. Using the predicted component, the proportional volumes of air, mixture, and water at varying water level heights and morphology were calculated. These fractions were stored in a matrix of shape (738, 3, 1,500), where the first dimension represented the height of the water pipe, the second dimension represented the categories air, mixture, and water, and the third dimension represented the total number of frames for each experiment. Thus, the fraction of each phase was visualised versus the height of the vessel, as shown in Figure 8.

As inferred from Figure 10(b), it can be stated that model #6 is more capable of predicting pressure in the air vessel, especially in the extrapolation sub-tests. However, for sub-test #77, a disturbance in the predicted pressure is observed approximately 1 seconds after closing the valve. This deviation is postulated to originate from image-based feature of air percentage, which becomes disturbed due to the highly turbulent and transient behavior of the fluid in the vessel, thereby introducing inaccuracy into the phase segmentation (Figure 10(k)).

Comparing the results with existing studies in the field of transient flow analysis, several key advancements emerge. While traditional numerical models achieve satisfying predictions of pressure variations in air vessels, they typically require significant computational resources and processing time, particularly when modelling complex phenomena like entrapped air. Model #6, achieving an value of 0.987 and successfully handling both interpolation and extrapolation cases, demonstrates acceptable accuracy but with substantially reduced computational overhead. The integration of image-based features with conventional sensor data represents a novel contribution to the field, as previous studies have typically relied solely on numerical measurements from pressure transducers and flow meters for transient analysis. This visual-numerical hybrid approach enables direct quantification of air–water interactions during pressure surges, providing insights that are typically approximated or simplified in conventional numerical models. Furthermore, the model’s ability to extrapolate beyond trained flow rates (as evidenced by the extrapolation loss of 2.25E04 in Model #6), addresses a common limitation in hydraulic modelling, where performance often depends heavily on calibration within specific operating ranges. The combination of reasonable computational cost and robust prediction accuracy, even under varying flow conditions, makes this approach particularly suitable for real-time monitoring and control applications in water distribution systems. However, availability of visual data in real field applications remains a limitation, which could be addressed via modern high resolution sensors.

A valuable application of predictive analysis lies in its ability to visualise complex phenomena within pipeline systems, providing insights that extend beyond traditional methods. This visualisation capability plays a crucial role in various stages including design, education and operational management. By leveraging predictive analysis models, which are trained on historical data from real systems, engineers and operators can simulate and visualise the outcomes of adjusting flow parameters, thus enhancing decision-making processes. Moreover, these visualisation tools can be integrated into digital twin frameworks, where real-time data feeds into the predictive models to simulate the current state of the system and forecast future conditions.

It is important to note that the segmented images presented are predictions from the encoder-decoder machine learning model used in this study. Currently, the visualisation tool is not using live field readings but works retrospectively. It is proposed that, in the future, machine learning models could enable the use of sensor readings as input, allowing the creation of multiple frames depicting the interior of the air vessel. Such an approach would facilitate practical visualisation of the air vessel enhancing the understanding and control of these systems.

While this study presents promising results in applying machine learning techniques to predict and visualise transient flow phenomena in air vessels, several limitations should be acknowledged.

The models’ performance under extreme conditions or rare events not represented in the training data remains uncertain. This underscores the need for continuous model refinement and validation as more diverse data becomes available.

Finally, while the integration of visual and numerical data demonstrates potential, the approach may be constrained in real-world applications where visual data from inside pipes or vessels is not readily accessible. Future research could explore reverse modeling using generative AI techniques to create visualisations based solely on sensor readings. This method would enable internal observation of opaque pipes and vessels, providing valuable insights without requirung transparent experimental setups. However, a major challenge in implementing this method would be the scarcity of training data that pairs sensor readings with corresponding visual representations. To address this, transfer learning techniques could be explored. By leveraging pretrained models from related domains or simulated data, it may be possible to adapt and fine-tune the generative models with limited real-world data.

This study successfully demonstrated the potential of machine learning, including deep learning, for predicting and visualising complex transient flow phenomena in air vessels, specifically focusing on the challenging water hammer effect. A novel hybrid approach was developed, integrating conventional sensor data with visual information extracted from high-speed camera images using a U-Net segmentation model. This allowed for a more comprehensive understanding of the air–water dynamics within the vessel. Three neural network models were trained, progressively incorporating visual features (air and water percentages) derived from the image segmentation. The results showed that incorporating visual data, particularly air percentage, improved the accuracy of pressure predictions, achieving a high R-squared value and demonstrating the model’s ability to generalise to both interpolation and extrapolation scenarios. Feature importance analysis confirmed the significant role of initial air pocket height in pressure prediction, aligning with theoretical understanding. While limitations remain, including the need for larger image datasets for segmentation and the challenge of acquiring visual data in real-world applications, this work provides a valuable foundation for future research exploring generative AI for visualisation from sensor data and the development of robust, real-time monitoring and control systems for water distribution networks.

By bridging the gap between visual observations and pressure predictions in complex hydraulic systems, the research highlights the potential of machine learning to enhance water system management. Managing pressure surges is crucial for preventing significant water loss, and air vessels are highly effective in mitigating such risks. The models studied, particularly Model #6, demonstrated the ability to interpolate and extrapolate pressure beyond the trained flow rate, achieving R-squared values up to 0.987. This high accuracy, especially in extrapolation scenarios, underscores the potential of machine learning in real-world applications where operational conditions vary.

One key aspect explored was the relationship between training epochs and model loss. While accuracy improved as the number of epochs increased, diminishing returns were observed beyond 20 epochs due to dataset limitations. This highlights the need for expanding the dataset to enhance future model training. Additionally, the study identified several practical challenges. The experimental setup, while controlled, represents a simplified version of real-world systems, and the reliance on transparent vessels for image acquisition limits immediate industrial applications. However, future advances in high-resolution acoustic sensors could provide an alternative to transparent vessels. Moreover, leveraging correlations between image-based features and air pressure presents new possibilities for inverse modelling, enabling indirect predictions of air vessel behaviour from pressure and flow data.

For practical implementation, integrating these models into existing SCADA systems could enable real-time pressure prediction and system monitoring. Water utilities could leverage such approaches for predictive maintenance and operational optimisation, particularly in systems using air vessels. The visualisation capabilities demonstrated in this study also offer potential for operator training and system diagnostics, although adaptations would be required for opaque industrial systems.

Future research should focus on expanding training datasets to encompass a broader range of operational conditions and system configurations. The development of synthetic training data could help overcome experimental limitations, while transfer learning approaches could facilitate adaptation from laboratory-scale models to full-scale industrial applications. Additionally, generative AI techniques may enable the visualisation of internal flow conditions without requiring direct optical access.

The coupling of advanced visualisation techniques with the digital twin concept represents a forward-looking approach that could transform water system management. By continuously updating digital models with live data, operators can gain real-time insights into system performance, leading to more efficient and resilient management practices. In this context, machine learning and artificial intelligence offer a means to reduce computational intensity compared to traditional numerical methods, making large-scale system analysis feasible in live or near-live modes.

This interdisciplinary approach, combining hydraulic engineering, computer science, and data science, provides a novel framework for tackling current challenges in water systems. As water utilities worldwide strive for greater sustainability and efficiency, the integration of such technologies will be crucial in addressing issues related to ageing infrastructure, water loss, and the demand for more responsive and intelligent systems. Ultimately, this research paves the way for more intelligent and adaptive smart water systems (SWS), contributing to the ongoing evolution of smart infrastructure in the face of global water challenges.

The experiments were conducted in the Hydraulic Laboratory of Instituto Superior Técnico, University of Lisbon, Portugal, with support from the Civil Engineering Research and Innovation for Sustainability (CERIS) research centre.

This research was funded by School of Civil Engineering, University of Leeds, UK.

All relevant data are available from an online repositoryor repositories (please ensure the DOI/URL has been provided as a submssion item).

The authors declare there is no conflict.