ACWA: an AI-driven cyber-physical testbed for intelligent water systems

Water supply systems (WSS) are primary sources that provide and ensure water quality and availability, a prerequisite for flourishing economies, protecting public health, and promoting national prosperity (Batarseh & Kulkarni 2023). However, the lack of water availability has become a growing concern that requires increasing attention. The United Nations (UN) World Water Development Report of 2021 (UN 2021) states that the global population is experiencing severe water scarcity and will rise from 32 million people in 1900 to 3.1 billion people by 2050.

Additionally, cyber threats on WSS have been a critical threat to water resources in the United States and around the world. In WSS, it is common to use Supervisory Control and Data Acquisition (SCADA), a standard ‘legacy’ software infrastructure, making systems susceptible to cyber threats (CISA 2021). Recently, Hassanzadeh et al. (2020) reported 15 cyber incidents on WSS and mentioned that more sophisticated attacks, such as data poisoning, minimum perturbations, and botnets, require algorithms that can detect, classify, and counter adversarial actions. According to Batarseh & Kulkarni (2023), AI is the leading approach to such defences due to its ability to identify unwarranted pattern shifts in networks and datasets. Despite AI being the primary approach, there is an absence of historical data (Sobien et al. 2023) concerning cybersecurity measures in water resources management. The dissemination of water data is rare due to their lack of integration and interoperability across various data archives (Bustamante et al. 2021). Further, difficulties in data accessibility arise from challenges in locating, obtaining, and comparing data across different regions, given that various parties and agencies store and manage data in different formats and have different governance limitations. Additionally, there are inconveniences in data accessibility caused by political, economic, and cultural barriers, along with varying legal requirements. For instance, most WSS have restrictions that stop them from sharing their data, also especially with cyber-related data, there is a challenging scarcity of data points that could be used for identifying patterns via AI and other methods. Considering these challenges and their severity, the need to create an innovative testbed that can create data and validate scenarios in a dynamic manner is evident. ACWA therefore, is an innovative cyber-physical system designed to tackle these challenges by generating valuable scenarios and datasets through experimental procedures. The underlying concept of the ACWA laboratory involves creating a water testbed with varying topologies (i.e., dynamic structures) to simulate diverse water distribution scenarios. Moreover, a soil topology is also established to explore the impacts of water on irrigation and agricultural practices. This comprehensive setup will incorporate different sensors (water and soil) and chemical agents to enable seamless data collection, feeding directly into a database. The accumulated data within the database are visualised via real-time dashboards, offering proactive monitoring and evaluation capabilities.

A testbed can be defined as a ‘composite abstraction of systems used to study system components and interactions to gain further insight into the essence of the real system’ (Fortier & Michel 2003). This section reviews seven existing testbeds developed in the water domain for the simulation of monitoring and control processes. The reviewed testbeds are shown in Table 1, and the testbeds are also compared and contrasted with ACWA.

Karray et al. (2016) presented EARNPIPE to provide a low-power wireless sensor network (WSN) solution that detects leaks (and provides localisation). The meaning of localisation in this context is the sectional leaks in an otherwise leakage-free pipe. This testbed is constructed in a rectangular shape with polyethylene pipes, which support pressure up to 12 bar (1 bar = 100,000 pa). Further, two valves are installed as an inlet and outlet to vary water demand by varying the pressure. EARNPIPE also includes a 1,000 m³ reservoir for water storage and an electric pump with one Horsepower (hp) for pumping water from the reservoir to pipes. To induce leaks, two garden taps are used in their setup. The authors also have proposed and discussed the implementation of leak detection (Leak Detection Predictive Kalman Filter) and localisation algorithms (modified time difference of arrival) (Karray et al. 2016).

Laso et al. (2017) presented a dataset generated from the physical water testbed to enable the detection of anomalies and malicious acts in cyber-physical systems. This testbed utilises two tanks (one with a 7-L capacity and the other with a 9-L) for storing water or fuel, one ultrasound depth sensor, four discrete sensors, and two pumps. The physical components are controlled and monitored using a computer connected to a programmable logic controller (PLC). This water testbed simulates 15 unique situations affecting ultrasound sensors, discrete sensors, the underlying network, or the whole subsystem.

Ahmed et al. (2017) presented an architecture of a Water Distribution testbed (WADI), a scaled-down version of a large water distribution network of a city, in 2023, the same team released an additional SWaP testbed as indicated in Table 1. WADI consists of three stages (P1, P2, and P3), when in operation, it can supply 10 gallons/min of filtered water. The P1 is a primary grid that contains two water tanks (2,500 L each), a water level sensor, and a chemical system that maintains the water quality. Additionally, water quality sensors are also installed in P1. The second stage is P2, which indicates the secondary grid. This stage consists of two elevated reservoir tanks and six consumer tanks. The raw tanks from P1 supply water to these elevated tanks based on pre-set demand patterns set by the authors. Lastly, P3 deals with the return water grid, which is also equipped with a tank. When the demands of the consumer tanks are met in P2, the water drains to the return water grid. Detailed information on communication infrastructures and how they can help analyse an attack's impact is provided by Ahmed et al. (2017).

Oberascher et al. (2022a, 2022b) described the Smart water campus testbed for monitoring water networks that can be useful for fault detection in real-time along with involving cross-system improvements like rainwater harvesting. The authors noted that the Smart water campus is a network-based urban water infrastructure that integrates a water distribution network, an urban drainage network, and nature-based solutions. The testbed leverages perception, communication, middleware, and processing layers. A detailed description of the four layers used in the Smart water campus testbed and a demonstration of smart applications can be found in Oberascher et al. (2022a, 2022b).

The scope of most of the existing testbeds summarised in this section is either within one or two experimental spectrums. For example, WaterBox (Kartakis et al. 2015) focuses on the simulation of monitoring and control processes of water networks and EARNPIPE (Karray et al. 2016) is designed to detect leaks and localisation, ACWA (the new testbed presented in this paper) aims to provide a dynamic environment that allows for multiple scenarios, including related to irrigation and soil, a notion missing from existing testbeds. SWaT (Goh et al. 2017), WADI (Ahmed et al. 2017), and FACIES (Bernieri et al. 2017) are explicitly developed for studying the effects of attacks (cyber and physical) on water plants while Laso et al. (2017) presented a testbed for anomaly/malicious act detection. These testbeds involve components such as pipes, pumps, and valves along with flow and water level sensors, but these testbeds do not capture water chemicals and related parameters (which is commonly a manner of attack on water systems), the lack of such experimentation renders these testbeds limited. Smart water campus (Oberascher et al. 2022a, 2022b) is the only testbed focusing on water quality parameters and on capturing soil and climate-related variables, although it is spread across eight hectares and does not provide the ability to run distributed AI algorithms. Considering these aspects, the ACWA testbed differentiates from the reviewed testbeds via the following main characteristics:

• 1.

  Modularity: The components of the testbed (multiple water networks) are designed modularly. This unique flexibility allows for the creation of new topologies and to add more nodes (such as pumps, valves, pipes, tanks, sensors, soil, and reservoirs) in any topology based on the experiment's design and goals.

• 2.

  Sensors: The ACWA testbed has 14 water sensors that capture data on traditional water parameters, such as water level, water pressure, and water flow; and quality variables, such as pH, temperature, dissolved oxygen (DO), nitrate, and electrical conductivity (EC) – among other data points listed in this manuscript. Additionally, the soil topology is equipped with a soil moisture sensor, water turbidity, and two soil probes to capture EC and temperature data.

• 3.

  Data: The big data created at ACWA are captured at 1-, 5-, and 30-s intervals and are stored in a local MongoDB database. This unique feature allows researchers to capture granular-level data and analyse minuscule changes in sensor values, including the deployment of AI and security algorithms. A data collection frequency not found anywhere else.

• 4.

  Water and soil: The ACWA testbed is carefully designed and planned to accommodate a soil topology with four soil beds along with Line, Star, and Bus water network topologies.

Furthermore, there is a lack of existing software water simulators, the most commonplace water simulator is EPANET, a prolonged hydraulic and water quality dynamics simulator built within a pressurised piping systems environment; based on Rossman (2000), EPANET is also used in commercial software development – in our open access GitHub repository (https://github.com/AI-VTRC/ACWA-Data), we also share the EPANET model for ACWA (in Appendix E), accessible to other researchers. A main addition that we present via ACWA is also a digital twin, namely, the ACWA simulator, also presented in this paper. The next section presents the main parts of ACWA, the main contribution of this paper.

To dynamically simulate and collect real-time data on different scenarios of WSS, the laboratory's sensors are built into four topologies to capture water and soil quality attributes, such as pH, temperature, DO, turbidity, nitrate levels, EC, soil moisture, water level, pressure, and flow rate. These data are stored in a MongoDB database for analysis, model development, and AI assurance, which can assist in tackling rising challenges in the water and agricultural domains as found in recent literature (Dobermann et al. 2004; Li et al. 2020; Saad et al. 2020; Batarseh & Kulkarni 2023).

A topology is a physical and logical arrangement of nodes and connections in a network. Literature (Peterson & Davie 2007; Liu & Liu 2014) indicates seven commonly used network topologies: Line, Bus, Star, Ring, Mesh, Tree, and Hybrid. These topologies are foundational in computer networks. However, literature on WSS indicates four main structures: grid-iron, ring, radial, and dead-end (Adeosun 2014). These four structures can be designed using the fundamental computer network topologies. In dead-end WSS (National Research Council 2007), the main line is at the centre, and sub-main lines are divided into branches. This structure can be developed using a Bus topology at ACWA in which the nodes are connected to a central line using a branch-like structure. Similarly, Ring WSS is a circular system, and a line topology can be turned into a Ring WSS if we connect the first and last nodes of a network. Also, in Radial WSS (National Research Council 2007), a reservoir is present at the centre, and water is distributed directly from it, which can be represented using ACWA's Star topology. Considering these aspects and motivation from computer networks, we have built Line, Star, and Bus topologies. In these topologies, nodes are represented by water tanks and they are connected to pumps and reservoirs using pipes and tubes. These topologies are placed independently on three tables of sizes 5 inches × 2.6 inches × 2.5 inches (length × width × height), which can also be connected into a hybrid topology using Poly Vinyl Chloride (PVC) or Chlorinated Poly Vinyl Chloride (CPVC) pipes. In our setup, we used two big water tanks (35 gallons) as reservoirs to provide water to the testbed. The detailed descriptions of these three topologies are as follows.

In this topology, four rectangular tanks and two diaphragm water pumps are connected via ¾ inch C-PVC pipes, as Figure 4(b) indicates. In Bus topology, the C-PVC pipe is installed at the centre, alternatively distributing water to two tanks on each side. The water flow also can be changed using manual valves, which are installed for every tank and line. The four tanks in this topology are 16.25 inches × 8.375 inches × 10.5 inches (length × width × height) in size and have a 5.5-gallon water storage capacity. For data collection in real time, the water level and EC sensors are mounted while the pressure sensor is installed on a main line ¾ inch C-PVC pipe.

The Star topology is developed by connecting one medium-size and four small-size water tanks via ½ inch C-PVC pipes. A schematic diagram for Star topology is shown in Figure 4(c). This topology is designed using four diaphragm water pumps and five cubical tanks. In this topology, one diaphragm water pump transfers water from the main water reservoir to the central tank (node), and then two additional diaphragm water pumps distribute this water to four other small tanks. The manual valves for each tank are provided to change or control the water flow. Lastly, at the end of the experiment, the fourth pump pulls water from all four tanks back to the reservoir. This process is performed using a four-way water splitter and tubes. The four tubes are immersed in four small tanks to pull the water and transfer it back to the reservoir. The water flow can also be controlled or turned on/off using the individual valves on the splitter. The central tank used in this topology is 15.25 inches × 15.25 inches × 15.25 inches (length × width × height) in size and has a 14-gallon water storage capacity. The four small tanks are 9.25 inches × 9.25 inches × 9.25 inches (length × width × height) in size and each tank has a 3-gallon water storage capacity. For data collection in real-time, water level, pH, temperature, and EC sensors are installed. Also, one water level sensor is mounted to capture data from the reservoir that provides water for this topology.

The mentioned topologies are connected to computational nodes and sensors, and via multiple communication protocols; the details of which are presented in the following sections. Appendix A shows other photos from the testbed.

This section includes details on the sensors built into the ACWA topologies, including their communication protocols, as well as the chemicals and solutions applied for AI and Cyber experimentation. The list of sensors is presented in Table 2. Table 3 lists the main experimental chemicals and chemical solutions at ACWA.

The sensors are connected to computers and GPU devices via three communication protocols, Modbus, Long Range, and Zigbee, as follows: (1) Modbus: A widely used communication protocol (Thomas 2008) in industrial automation and control systems. It allows different electronic devices, such as sensors, actuators, and PLCs, to exchange data and control commands over a network. Modbus is characterised by its simplicity and versatility, making it suitable for simple and complex applications in various industries. It operates over different physical layers, including serial connections like RS-232, RS-485, and Ethernet, making it adaptable to different communication needs. (2) Long Range: LoRa, or Long Range, is a wireless communication technology for long-distance data transmission with low-power consumption (Devalal & Karthikeyan 2018). It enables devices to communicate wirelessly over extended ranges. This makes LoRa useful for applications such as Internet of Things (IoT) devices, smart city solutions, and rural connectivity. LoRa uses spread spectrum modulation techniques to achieve reliable communication even in challenging environments, and its low-power requirements make it suitable for battery-operated devices, extending its operational lifespan. (3) Zigbee: A wireless communication protocol (Ergen 2004) developed for creating low-power, short-range networks among various devices. It is designed for home automation and industrial control. The Zigbee protocol enables efficient and reliable data exchange. It operates on the IEEE 802.15.4 standard and uses a mesh network topology. This allows devices to communicate directly or indirectly through other devices, enhancing coverage and reliability. Zigbee's focus on low-power consumption makes it well-suited for battery-operated devices in smart homes and IoT applications.

To manage data streams for AI development and Cyberbiosecurity measures, the following softwares are used (but not limited to): MongoDB, NodeRed, R Dashboards, Python, TensorFlow, Keras, and other data management technologies, more information on that is listed in the public access GitHub page: https://github.com/AI-VTRC/ACWA-Data. The next section presents the ACWA simulator and its structural details.

The ACWA simulator generates a CSV file incorporating time series data of all the parameters for every simulation. The novel part of the simulator is that it provides a time series for both hydraulic parameters, such as water level and pressure, along with water nutrient parameters, such as pH, Nitrate, BOD, and NaOH, which change every second. These generated data can be used for calibration, validation, AI modelling, data poisoning experiments, and comparison with the physical system. Inputs to the simulator environment include pH, BOD, DO, Nitrate, NaOH, Water Level, Water Temperature, Water Density, Kinematic Viscosity, Discharge, Pipe Material, Pipe Length, Pipe Diameter, Tank Shape, Tank Size – the data outcomes of the simulator as presented in Table 4.

The software simulator has built-in physical assumptions that can be updated and tuned using the code; the following nine assumptions are set:

• 1.

  Water is assumed to be an incompressible Newtonian fluid (Mott & Untener 2015).

• 2.

  Water is in a steady state, i.e., the type of flow throughout the process will not change.

• 3.

  Flow should either be laminar or turbulent. This is implemented in the constraints section.

• 4.

  The reservoir and tanks are at atmospheric pressure.

• 5.

  Only head loss due to friction, entry, and exit loss is considered.

• 6.

  pH interacts with NaOH; BOD interacts with DO. No other mutual interactions are considered. The change in constituents is determined only by wall reaction, bulk reaction, and decay function.

• 7.

  Temperature can change only due to differences in air temperature and convection.

• 8.

  The model assumes a steady-state scenario, i.e., the conditions are not changing with time, only with position along the pipe. The fluid properties (like density, specific heat, and the heat transfer coefficient) are constant. This might not be the case, especially if the temperature changes significantly.

• 9.

  The surface temperature of the pipe is assumed to be constant along the length of the pipe.

Additionally, multiple physical characteristics are built into the system, related to the input variables mentioned previously, those are presented in Table 5 (Simulator inputs are presented in Appendix C).

The data created from all the mentioned parts in this section are the main outcome of ACWA; the data variables are presented in Table 6 (data samples are available in the mentioned public GitHub repository; other datasets are available upon demand from the authors).

Additionally, data collected via the simulators includes the following variables: Time (seconds), Tank 1 Water Level (m), Tank 2 Water Level (m), Tank 1 Pressure (Pa), Tank 2 Pressure (Pa), Nitrate Concentration (mg/L), BOD Concentration (mg/L), DO Concentration (mg/L), pH, Temperature (°C). All data are visualised in multiple data dashboards (presented in Appendix D).

The scope of the ACWA laboratory is extensive, covering a diverse range of applications relevant to its design and operation. It includes various concepts, such as data creation and collection, benchmark scenarios for Cyberbiosecurity research (Sobien et al. 2023), AI assurance techniques to address adversarial attacks, fine-tuning process of soft sensors, innovative ideas for water and agricultural-related public policies, and more. Considering these aspects, the potential use cases that can be developed in the ACWA laboratory include (but are not limited to):

• 1.

  Digital twin: A digital twin is a replication of a physical sensor that allows monitoring, visualisation and prediction of its physical counterpart (Eckhart & Ekelhart 2019). The data collected from water and soil sensors could be used for the development of digital twins (besides the presented ACWA simulator). This can help in different applications, such as soil quality, systems' simulation, along with other potential agriculture and water-related uses (Eckhart & Ekelhart 2018).

• 2.

  Data poisoning: The EPA (Environmental Protection Agency) has established water regulation limits, and water distribution plants use sensors to control the level of chemicals (such as Phosphorus) in the water. The ACWA laboratory allows the simulation of data poisoning via advanced AI algorithms that can help simulate cases where the data used for decision-making are compromised, similar to the many cyber incidents presented by Hassanzadeh et al. (2020).

• 3.

  Water quality: WSSs use filtration and chlorination processes to remove turbidity from water (Stevenson & Bravo 2019). The use of high chlorination forms trihalomethanes (THMs) and Haloacetic acids (HAAs), which can create human health hazards (Lowe et al. 2022). In these scenarios, AI can be used to develop an early warning system for predicting reclaimed water (U.S. Environmental Protection Agency 2023) turbidity. This may help WSS in planning the use of the chlorination process efficiently.

• 4.

  Pump operations: These operations, such as determining capacity, number of pumps, and when to start them, can be determined optimally. This can assist pump operators by providing actionable recommendations on pump operations and reducing pump usage and energy consumption.

• 5.

  Simulation: The ACWA simulator generates different scenarios to showcase water flow from a reservoir to a tank. The simulator provides complete control of inputs such as selecting material for the pipe, tank dimensions, and chemicals. This simulator can be used to study the effects of different conditions (chemicals, pipe materials, etc.) on the water.

• 6.

  Decision-support system: A decision-support system represents the role of technology, such as AI, to provide the stakeholders with intuition, insight, and understanding (Keen 1980; Brill et al. 1990). These systems can be developed by combining advanced AI techniques with AI assurance for different water and agricultural stakeholders.

• 7.

  Process monitoring: ACWA data visualisations can represent the information and act as a diagnostic tool to bring situational awareness (Few 2007). This can help monitor different agricultural and/or water plant processes in real-time to check for subtle accuracies ingested into sensor networks.

• 8.

  Data generation: The need for more datasets for securing cyber-physical systems (CPS) (Goh et al. 2017) can be tackled by generating data from the ACWA laboratory topologies. ACWA laboratory provides a facility to generate realistic datasets for network, physical properties, and soil and water nutrient data with sufficient complexities of WSS, a notion not found in any other testbed.

• 9.

  Fertigation: Fertigation is injecting fertilisers and other soil products into the soil. ACWA laboratory allows experimentation of different fertiliser concentrations along with other parameters such as pH, and moisture content. This can be useful for instance for developing a recommendation system using AI models for farmers.

• 10.

  Economics and policy-making: The use of AI, specifically Deep and Reinforcement Learning (DL and RL) – subfields of AI – has been increasingly assisting farm, forest, and ranch managers in decision-making (Environmental Protection Agency, n.d.). In ACWA, advanced AI methods with causal aspects will be researched focusing on applications such as water quality, agricultural market structures, agricultural production, resource allocation, and international trade.

• 11.

  Anomaly detection: It is common in smart farms to have physical devices such as sensors, actuators, network cables, and routers outdoors. Also, these devices do not have tamper-resistant boxes, and the changes of intentional or unintentional physical modifications of these devices can occur (Rettore de Araujo Zanella et al. 2020). These anomalies can be identified using AI, and data generated from these devices can be detected in real time, a cyber scenario that can be tested at ACWA.

• 12.

  Network attack detection: A report published by the US Council of Economic Advisors (2018) reported >11 large-scale cyber incidents in the food and agriculture sector in 2016 for instance. This creates a need to provide robust security solutions, mainly when digital systems and IoT-based technologies are heavily used in the agriculture and water domains. The advanced sensors used in the ACWA laboratory can help create various anomalous datasets with a combination of different network attacks such as denial of service (DOS), man-in-the-middle, and address resolution protocol (ARP) Spoofing that may help in the development of assured AI models for network attacks' mitigation and detection.

• 13.

  Soft sensors: In various industrial and scientific applications, accurate and reliable sensor measurements are crucial for making informed decisions and maintaining high-quality processes (Wang et al. 2022). However, hardware sensors are subject to environmental conditions, calibration drift uncertainties, expensive prices, hard set up, and other unreliable aspects (Geng et al. 2015). One solution to this problem is the development of soft sensors, which are mathematical models that output specific target parameters using hardware sensor data (Ching et al. 2021). This solution can be explored in ACWA considering the availability of advanced high-quality water and soil sensors.

Other novel use cases will be researched considering the need and advancement of water technologies. Additionally, the ACWA laboratory actively collaborates with external organisations, industry partners, and academic institutions, fostering a collaborative community dedicated to addressing other potential complex challenges in water and agriculture.

This manuscript presents the new ACWA testbed, a novel cyber-physical system designed and developed to test, simulate, and experiment with cutting-edge technologies such as AI and cyber solutions for intelligent water systems. As established in the literature review section, ACWA is the first of its kind in the domain, and it fills a needed gap for evaluating existing challenges and validating some of the proposed solutions in academia and industry. It is rather difficult to represent all potential AI applications in the water sector, for that reason, the four ACWA topologies (line, bus, star, and soil) are developed in a modular manner that allows researchers to manipulate the structure, expand on it, and collect different forms of datasets based on the desired outcomes.

All relevant data are available from an online repository or repositories. ACWA data samples are available in our open access GitHub repository (https://github.com/AI-VTRC/ACWA-Data). More detailed datasets including requests for specific simulations and complete timeseries are available upon request from the authors.

The authors declare there is no conflict.