The treatments to be applied for water purification must be dynamically adaptable to the raw water conditions. Currently, treatments are applied based on standards that are not optimized for the circumstances of each drinking water treatment plant (DWTP), neither for critical events. This paper presents a methodology for the creation of an Artificial Intelligence (AI) decision support system (DSS), encompassing the principal steps of the drinking water treatment processes (coagulation, sedimentation, filtration and disinfection), based on reinforcement learning techniques, that provides suggestions about the most efficient treatments (coagulant and chlorine dosages) for various raw water conditions, including critical events such as heavy rain and saline intrusions. Together with the model, a retraining strategy is included so the DSS adapts itself to the specific circumstances of each different DWTP. The model has been developed and validated in a DWTP replica. Furthermore, the model has been provided to a real DWTP to obtain feedback from experienced staff. The results and evaluation of the model are promising as a first approach on a DSS for drinking water treatments suggestion, although future versions might require more water quality parameters to characterize the raw water.

  • AI decision support system (DSS) for the suggestion of the most efficient dosages (coagulant and chlorine) to be used in drinking water treatment plants (DWTP).

  • Multi-armed bandits applied on the whole process of most common DWTPs (coagulation/filtration/disinfection).

  • Combination of simulated data, data from a scaled-down replica DWTP, and a real DWTP.

  • Auto-optimization routine.

  • Outcomes validated by a real DWTP.

The water used for any domestic purposes must be wholesome. The requirements for treatment may vary according to the weather, the catchment activity, an accidental spillage, or other reasons. Although monitoring may indicate that the water is bacteriologically safe, sometimes, it is highly probable that there will be a bacteriological challenge (WHO 2022). The absence of microbiological contamination is generally the most important factor for human health as pathogens can lead to infectious diseases. Also, the substances in the water which only affect water's appearance, odour, or taste could make water objectionable to consumers and lead to rejection on aesthetic grounds.

Water treatment consists of sequential units to eliminate pollutants and pathogens. It usually includes pretreatment, coagulation, flocculation, sedimentation, filtration, and disinfection (Farhaoui & Derraz 2016; Chiavola et al. 2023). Most treatment systems are designed to remove microbiological contamination and physical constituents, such as suspended solids (turbidity), which affect aesthetic acceptability and prevent effective disinfection. Since microorganisms can be associated with particles and turbidity in water (Schiperski 2018; WHO 2022), physical contamination may represent a health risk and it makes disinfection more difficult.

Treatment models can help in understanding and predicting the treatment's effectiveness, especially in extreme events when abstraction water changes abnormally, like during storms or droughts (Aliashrafi et al. 2021). If these models have good accuracy, they could be used for treatment control to ensure the quality of the treated water, thereby mitigating any risks and protecting the drinking water supply.

There are mathematical models that describe and simulate the most common water treatment processes such as sedimentation, coagulation/flocculation, filtration, aeration, chemical oxidation, or granular activated carbon adsorption (Fair & Gemmell 1964; Hidayah & Hendriyanto 2014; Sun et al. 2019). These mathematical models are robust methods for the simulation of drinking water treatment plants' processes (DWTP's processes), allowing the DWTP staff to check the results of using different treatments. However, this implies the staff search for the best treatment using a try-and-error method (based on their expertise). The DWTP staff usually utilizes the treatment standards supplied by the treatment providers and by dedicated applications, which recalculate the treatments based on the current water flow (Rowe 2020). Nevertheless, none of their suggestions consider the peculiarities of each individual DWTP (small differences in water composition produced by local source water conditions and infrastructure materials). Data-driven approaches are recommended for these cases (Yin et al. 2021). Artificial intelligence (AI), and in particular machine learning methods such as artificial neural networks (ANN), and reinforcement learning (RL), are robust technologies that can handle the complex and dynamic nature of water treatment processes considering such individual particularities.

The goal of this research is the development of a decision support system (DSS) based on AI treatment models able to react to the events related to the quality of water in a dynamic data-driven approach, which allows for increased performance and safety of the current systems. The AI models are meant to provide efficient and robust treatment recommendations even when water quality changes occur due to extreme events such as heavy rains and droughts. Events that are not frequent, and that are expected to happen more often due to climate change, so the DWTP's staff are not prepared to react to them. Therefore, the outcome water treatment AI models are expected to help the DWTP's operators and managers monitor and control the processes. The purpose of the proposed DSS is to provide an augmented decision-making capacity for the most efficient treatments, especially during abnormal events which may be not part of the staff experience.

Currently, most of the research related to drinking water treatment plants using AI solutions focuses on forecasting some of the water quality parameters through the process (Abba et al. 2019) or focuses on part of the process, e.g., coagulant dosing prediction, or the prediction of the outcomes of the disinfection phase (O'Reilly et al. 2018; Li et al. 2021).

Our work presents a DSS that considers the whole treatment process to propose the dosages of coagulant and disinfectant, not only focusing on normal operation but also considering extreme events. An RL approach was used to create a model capable of estimating the most efficient dosages of coagulant and disinfectant that guarantee drinking water at the outlet of the treatment process for any water quality at the inlet. The DSS also includes a retraining routine to automatically adapt to the particularities of the different DWTPs. Data acquisition and generation was the main bottleneck of this work. The data from the real DWTP were scarce due to confidentiality. Therefore, these data were reserved for model optimization. The volume of data used for training was generated in a scaled-down replica of a real DWTP and a water treatment processes simulator. The model was tested in the replica and in the real DWTP. The data generated in the replica depended on the raw water used. This raw water was collected from a nearby water stream which was affected by unexpected climate events that slightly affected our results (see Section 3). Nevertheless, both replica and real DWTP results are promising, requiring improvements, such as including more quality parameters to increase robustness to biological changes. Furthermore, since DWTPs are critical infrastructures where it is not possible to do tests; it is necessary to further validate the results to be able to convince the DWTPs' staff of the reliability of the suggestions.

The content of this paper is structured as follows: Section 2 presents the methodology employed to develop the DSS, focusing on the data requirements and the RL algorithm used. Section 3 shows the results obtained during validation. The validation was done both under a controlled environment in a laboratory (replica) and in a real DWTP. Finally, the conclusions are presented in Section 4.

This section is divided into three main topics: the DWTP configuration, the data production, and the RL algorithm. Figure 1 shows a diagram of the methodology followed. The DWTP configuration section starts by describing the processes and the operation conditions considered for this work. The data production section describes the two tools used to generate data: a simulator of water treatment processes; and a scaled-down replica of a real DWTP which was created to generate data for optimization and to validate the DSS. Finally, the RL section explains the RL algorithm used and how it was applied in this work to produce the transition dictionary model (TD model). This last section finishes explaining the retraining routine required for the DSS to adapt to different DWTPs and data drifts. The retraining routine is an extra feature that has not been extensively validated, thus it is not included in this diagram, although it is part of the DSS.
Figure 1

Methodology diagram.

Figure 1

Methodology diagram.

Close modal

The validation in the replica and in the real DWTP is presented in Section 3.

DWTP configuration and operation modes

For this study, they considered various DWTP's operation conditions to generate a resilient AI treatment suggestion model:

  1. Normal operation is characterized by low input turbidity and conductivity.

  2. Heavy rain which has high input turbidity.

  3. Saline intrusion which has high conductivity.

Machine learning techniques are data-driven so they are highly dependent on data quality and quantity. Therefore, for each of those conditions, it was necessary to obtain enough data points. The data for this study were obtained from two sources: a DWTP's processes simulator and a scaled replica (see Section 2.2). For the purpose of simplifying the models in this study, the inlet water was characterized by pH, conductivity, and turbidity (see Table 1). Turbidity is important to correlate the solids and organic matter content of inlet water with the efficiency of the treatment (Shao et al. 2019). A change in pH could be a signal of high values in other parameters, such as organic matter or nitrogen forms (Adams et al. 2022). Conductivity is important for the detection of salinity intrusion and other sources of pollution (Adjovu et al. 2023). Their ranges were selected based on the real DWTP historical samples and related research (Khan et al. 2015; Bertone et al. 2016).

Table 1

Inlet water characterization for the three operation conditions considered in this study

ParameterNormal conditionHeavy rainSaline intrusion
pH 7–7.5 7–8.5 7–7.5 
Turbidity (NTU) 0–10 0–70 0–20 
Conductivity (μS/cm) 60–120 100–450 100–3,500 
ParameterNormal conditionHeavy rainSaline intrusion
pH 7–7.5 7–8.5 7–7.5 
Turbidity (NTU) 0–10 0–70 0–20 
Conductivity (μS/cm) 60–120 100–450 100–3,500 

The outlet water was characterized by free chlorine and turbidity. It was considered that conductivity can be linked with chloride (Cox et al. 1967; Peinado-Guevara et al. 2012; Kothari et al. 2021), and turbidity can be an indirect measurement of total dissolved solids (TDS) (Serajuddin et al. 2019) and organic matter (Bright & Mager 2018). Based on WHO recommendations (WHO 2022), if the outlet water contains between 0.5 and 1 mg/L of free chlorine and the turbidity is below 5 NTU, it means there is no biological pollution present. However, depending on the country, these thresholds can be more relaxed.

The treatment processes considered in this study are three of the most common drinking water treatment processes: coagulation (with sedimentation), filtration and disinfection (chlorination).

  • Coagulation is used to remove colour, turbidity, algae, and other microorganisms from surface waters. The addition of a chemical coagulant to the water causes the formation of a precipitate, or floc, which entraps these impurities. Iron and aluminium can also be removed under suitable conditions. The floc is separated from the treated water by sedimentation and/or filtration, although flotation processes may be used in place of sedimentation. The coagulant used in the experiments is an aluminium-based coagulant.

  • Filtration. Turbidity and algae are removed from raw waters by screens, gravel filters, slow sand, rapid gravity filters, or cartridge filters. The difference between slow and rapid sand filtration is not a simple matter of the speed of filtration, but it is in the underlying concept of the treatment process. Slow sand filtration is essentially a biological process whereas rapid sand filtration is a physical treatment process.

  • Disinfection. Contamination by sewage or animal faeces is the greatest danger associated with drinking water. This is because sewage from human or animal sources may contain the causative organisms of many communicable diseases. The use of disinfectants to kill or inactivate pathogenic microorganisms is necessary if the raw water contains such organisms. Several disinfection methods are used in water treatment. Disinfection with chlorine is the most widely used method for large water supplies but its application is less common in small supplies.

The next sections describe the methodology followed during this study based on the aforementioned DWTP's configuration. Section 2.2 provides more details about the two main sources of data, which were also used for optimization and validation. Section 2.3 describes the AI modelling carried out to create the treatment suggestion model.

Testbed for data generation, optimization, and validation

Drinking water treatment plant (DWTP) scaled replica (see Figure 1): The objective of the creation of a DWTP replica was the generation of data from a near-to-real environment for any event affecting the raw water quality. Furthermore, its second objective was the validation of the models' suggestions. Therefore, a DWTP replica was created with the most common processes for drinking water production (see Figure 2). The replica is a scaled-down version of a real DWTP designed following downscale methodologies (Inoue et al. 2017; Silva et al. 2022); hence, its components and their dimensions. It consists of a reservoir tank with sensors for real-time characterization of the inlet water (pH, turbidity, and conductivity); followed by a tank where water is mixed with coagulants and flocculants, which then passes to a precipitation chamber. The next stage is a filtration unit filled with sand, finalizing with a chlorination section which applies the chlorine and measures the free chlorine at the outlet water with a chlorine sensor. The outlet water is stored in a tank where a sensor monitors the turbidity. The DWTP replica was designed for a maximum flow rate of 50 L/day, although it was operated at a flow rate of 20 L/day.
  • Reservoir tank. It has a total capacity of 200 L; and pH, conductivity, and turbidity sensors were installed on it.

  • Precipitation. It consists of a mixed chamber of 2 L of total capacity where the inlet water and the coagulant/flocculant are added in pulses. It is continuously mixed with a magnetic stirrer. The mixing time is set to 30 min. Afterward, the water passes to a 5 L sedimentation chamber where the settleable solids will fall regularly to the bottom. A period of 60 min of total stillness was configured. The solids will be then removed periodically by hand.

  • Filtration. It consists of a 50 cm sand bed with a residential time of 45 min.

  • Chlorination. It consists of a mixed chamber (residence time is 45 min), where the chlorine solution is pulse-dosed. After the mixed chamber, a sensor monitors the chlorine concentration in the storage tank.

  • Storage tank. A 200 L tank to store treated water. A turbidity sensor is installed in it.

Figure 2

DWTP replica on the left. Design of the replica on the right.

Figure 2

DWTP replica on the left. Design of the replica on the right.

Close modal

The full setup adds up to a total residence time of 11 h. The selected sensors are based on commercial availability, at a medium-low price, to be affordable for any size DWTP. The list of sensors is:

  • Turbidity (inlet & outlet) – Turbimax CUS51D.

  • pH (inlet) – Memosens CPS31D.

  • Conductivity (inlet) – Condumax CLS21D.

  • Chlorine (outlet) – Memosens CCS51D.

The raw water (without treatment) was collected from a stream near AIMEN's facilities. This stream passes through forests and a rural zone, so it is lowly polluted, and its quality highly depends on the weather conditions. Fresh water was transported daily to AIMEN's facilities. Then, synthetic water was produced to obtain the desired water conditions. The synthetic water was prepared by mixing the raw water with sediments from the catchment, marine salt, and wastewater treatment plant's sludge. A minimum of 25 L per batch was prepared. The pH, conductivity and turbidity sensors were used to check if the desired experiment's initial conditions for the inlet water were set up correctly.

Dilutions of NaOH and HNO3 were used to control the pH. A coagulation/flocculation commercial product for drinking water was used in the precipitation process. Most commercial products are based on ferric chloride or aluminium salts. The selected product is based on aluminium sulphate salts (25% in weight of Al2(SO4)3). The recommended dosage by the manufacturer (3 mg/L treat water) was used in normal operation conditions and it was adapted for extreme events conditions. This methodology provided information about the correlation between the product dosage and the inlet water characteristics (mainly turbidity and conductivity). To change the coagulant dosage, a solution with distilled water was made. As a baseline, 1 mL of raw commercial product per litre was considered, but it was modified to varying operation conditions.

A chlorine solution was used to obtain a Cl concentration in the outlet water up to 0.5 mg/L, to ensure drinking water microbiological quality. The maximum Cl concentration should be less than 1 mg/L in the DWTP replica's outlet to avoid ClO2 formation. A commercial liquid solution (37% in weight of sodium hypochlorite) was selected because of its ease of manipulation. As well as in the coagulant case, a solution in distilled water was prepared (approximately 1 mL of raw commercial product per L).

The solutions were dosed using peristaltic pumps with a timer controlled by a PC. The replica is controlled by a single-board PC (Nvidia Jetson Nano). It temporizes the pumps, to operate them sequentially in the selected retention time (11 h). This retention time is based on data from a real DWTP.

The replica was built in 2019 and finished by mid-2020. The data from the replica were generated from 2020 to 2022. It was used for validation during the second half of 2022.

Stimela: Even though we generated data with the replica, the gross of the data produced to train the AI models was simulated. The replica's residence time of 11 h limited the amount of data that could be generated; therefore, a simulator was necessary. Stimela is a mathematical model of drinking water treatment processes embedded in MATLAB/Simulink (Van der Helm & Rietveld 2002; Dudley et al. 2008). It was developed by Delft University of Technology, Kiwa and Royal Haskoning DHV. Stimela offers many models that can be interconnected to represent drinking water plants such as filtration modules, flocculation modules, aeration modules, etc. Stimela simulates the changes experimented in the water quality parameters when the water is processed through the modules.

Despite the fact that Stimela simulator has not been maintained since 2012 (its last version), it has been selected due to its immediate accessibility, ease of use and free license. There are other mathematical simulators for drinking water, although not as many as for water treatment. Table 2 shows a short comparison between some of the reviewed tools for drinking water process simulation (Dudley et al. 2008; Swan 2015).

Table 2

Drinking water simulators comparison

SimulatorsStimelaWEAPWatProOTTERSimEau
Free Yes Under certain conditions No No Yes 
Open Source Yes No No No No 
Availability Under Matlab Simulink Desktop application Desktop application Desktop application Desktop application 
SimulatorsStimelaWEAPWatProOTTERSimEau
Free Yes Under certain conditions No No Yes 
Open Source Yes No No No No 
Availability Under Matlab Simulink Desktop application Desktop application Desktop application Desktop application 

A Stimela module was developed considering the basic drinking water treatment processes defined previously. Stimela lets us concatenate modules of different water treatment processes (see Figure 3), the water travels through the model following the arrows. Our model consists of a raw water module in which the water's quality parameters are initialized (it can be thought of as the water that enters the DWTP).
Figure 3

Layout of Stimela with the schema of the simulated DWTP.

Figure 3

Layout of Stimela with the schema of the simulated DWTP.

Close modal

Then, the water is mixed with the coagulant. To simulate this process, three modules are needed, the ‘doscon’ (in which the quantity and type of coagulant are chosen), ‘coagul’ and ‘vlokvo’ (both modules are needed to simulate the mixing of water with coagulant). The following process is the filtering done in the ‘Single media filter’, the module ‘bacwa 1’ simulates the backwashing needed to clean the filter periodically. The water leaves the filter to enter the chlorination process, ‘chloro_m’ module. The ‘chloro_m’ module is also connected to ‘chldos’ which is a module used to specify the quantity of chlorine used in the chlorination process.

Through the process, there are two other types of modules, the ‘Graphical outputs’ which are used to visualize the water parameters through different stages of the process, and the blocks which have a name with the extension ‘.mat’. These are used to export the water quality parameters through various stages of the process. All the modules have a set of parameters associated with them. The parameters were chosen based on a real DWTP associated with the NAIADES project to make the simulations as accurate as possible.

The variables that were determined important to this study were the turbidity, pH, and conductivity of the water entering the DWTP, the coagulant dosage and chlorine dosage for the treatment, and the turbidity and free chlorine of the water leaving the DWTP. For the variables referring to the raw water some ranges were determined, those ranges were calculated considering that a change of range implied a significant change in the treatment dosage. The treatment parameters were chosen based on the fact that a change from one option to another of treatment would imply a significant difference in the quality of the water leaving the DWTP. Finally, the parameters of the exiting water were also classified in ranges, but these were based on health, safety, and legal limits.

From the raw water parameters, the one that was determined to influence the most in the choice of the treatment process was the turbidity. If the turbidity of the raw water increases the coagulant dosage must increase to maintain the quality of the treated water; if the coagulant dosage increases the amount of chlorine must also be higher to maintain its effectiveness. So, the ranges defined for the turbidity were sampled n-times, meanwhile the pH and conductivity were set to their median values for each range. For each sampled combination of (turbidity, pH, and conductivity) all the possible combinations of treatment values were simulated. The total number of simulations reached:
formula
(1)
where is the number of samples taken in each turbidity range, is the total number of turbidity ranges, is the total number of pH ranges, is the total number of Conductivity ranges, is the total number of coagulant dosage's options, and is the total number of chlorine dosage options.

Ideally, the same sampling process that was done in the turbidity would be done in the pH and the conductivity as well, but the computing time scales linearly with the number of samples taken and our simulations already took several days, and since it was decided that the replica will not experiment with those values due mainly to time restrictions, then it was deemed unnecessary. This decision would reduce the accuracy of the suggestions for those cases, making those suggestions less efficient but still valid, though future versions of the DSS should be retrained including all the possible data combinations. Stimela was configured by the end of 2020, it was tested during 2021, and it was used to produce data during the end of 2021 and the beginning of 2022.

RL-based treatments' suggestion

RL is a branch of the AI approaches which consist of selecting the best actions for an agent in a defined environment to reach the states with the maximum reward. In each event the agent receives some representation of the environment's state; based on that, it selects an action. As a consequence of the action, the agent receives a reward and finds itself in a new state. It was decided to use RL due to its auto-learning capabilities (Sutton & Barto 2018).

RL is very versatile, and it is used in a lot of applications in different industries: algorithms that play games (Mnih et al. 2013; Perolat et al. 2022), self-driving cars (Shalev-Shwartz et al. 2016; Liang et al. 2022), text generation (Li et al. 2016; Lu et al. 2022), etc. In fact, RL algorithms have already been applied to water treatment plant processes and water management with promising results (Hernández-del-Olmo et al. 2018; Chen et al. 2021; Makropoulos & Bouziotas 2023). The work by Chen et al. (2021) presents a multiagent RL approach to optimize two agents, the amount of chemical dosages and dissolved oxygen, in a waste water treatment plant (WWTP). The RL was trained for different objectives, such as the optimization of the environmental impact, based on a life cycle analysis, and cost reduction. Each of those objectives had a specific reward policy. Their results showed beneficial results for the WWTP, like the reductions of costs and energy by reducing the aeration and chemical dosages. Hernández-del-Olmo et al. (2018) proposed a solution to make the RL systems to be trustable by the WWTP managers so the RL could be applied directly to control the plant. Their RL approach-based on one agent tries to minimize the operational costs (costs derived from energy consumption and disposal of waste sludge). The costs are used as a reward to teach the agent based on the environment state (oxygen and N-ammonia values) reached after a set of actions (dissolved oxygen concentration). Their RL approach showed improvement in the N-ammonia removal process and reduced costs in comparison with the operators' decisions.

Among the RL methods, the Q-learning algorithm (Hernández-del-Olmo et al. 2018) was studied. Q-learning is an algorithm that looks for the optimal action to be applied to a specific state. When Q-learning is performed, a Q-table is created following the shape of [state, action], with the values initialized to zero. Those values are updated after an episode (simulation of all the possible states). The Q-table becomes a reference table for selecting the best action based on the Q-value. Some approaches were considered in this case, but the main problem was determining the states. They must contain the Inlet Water Quality parameters, but they also have to contain the Outlet Water Quality parameters so one can update the Q-table. A state that has information on both the Inlet and Outlet Water Quality parameters does not have a physical meaning and it was decided to drop this convoluted method.

Then, a multi-armed bandit model was considered. This specific type of RL has already been studied and suggested for some processes of WWTPs (Van Moffaert et al. 2014; Martin & Johnson 2020). The paper in Van Moffaert et al. (2014) presents a new approach to multi-armed bandits, the multi-objective X armed bandits. They mention that multi-armed bandits are recommended for problems like water purification in WWTP, this is a problem where multiple problems are trying to be tackled at the same time, with water purification minimizing the operational costs and the amount of sludge. Martin & Johnson (2020) studied different multi-armed bandits strategies based on the assumptions about the monitored data, in this case, using wastewater treatments as an example. The problem they aim to tackle is to find the best treatment (they consider three different treatment technologies) to remove the N effluent. Although this paper was focused on the different multi-armed bandits' strategies, their findings show that the multi-armed approach could have suggested the treatments that could have reduced the N effluent by more than 25%. In the RL multi-armed bandits modeling there is a fixed limited set of resources. The agent must pick between a set of choices which have a probability distribution that is partially known at the time of picking. There is a limited number of picks, the agent must maximize the total reward. In our case, the inlet water quality parameters define the environment, the treatment combinations are the actions (see examples in Supplementary material, Appendix B), and the outlet water quality parameters define the States (see examples in Supplementary material, Appendix C). Different Environments were considered (see Supplementary material, Appendix A).

Following this method, the transitions that happen for each combination of Environments and Actions were represented in a TD. The TD is a dictionary that has a key for each Environment and the element associated with each key is the Transition Matrix (TM) calculated for the Environment. A TM is a matrix in which each element represents the probability of ending in State i by taking Action j (the superscript refers to the Environment k). To calculate those elements, the following equation was used:
formula
(2)

The difficulty in the creation of the TD is the correct definition of the environment, actions and state. For example, for the TD related to the higher turbidity values, the turbidity range was taken following an exponential distribution (see Supplementary material, Table A1) meanwhile the turbidity range for the sets with lower input turbidity follows a lineal distribution (see Supplementary material, Table A2); for each case (low and high turbidity) 36 States and 35 Actions were defined, reaching 72 States and 70 actions in total. This is due to the relationship between the turbidity and the coagulant dosage. For higher input turbidity, it is needed a higher dosage of coagulant to maintain the output turbidity constant. But the amount of coagulant increases in an exponential way in relation to the input turbidity, so the turbidity range must follow an exponential distribution. For the lower turbidity cases, that exponential behaviour is not so significant, and a linear distribution creates the desired States. This is also the reason behind the fact that the range taken for the coagulant dosage for the high turbidity's case also follows an exponential distribution (see Supplementary material, Tables B1 and B2) meanwhile the coagulant dosage for the lower turbidity's values follows a linear distribution (see Supplementary material, Tables B3 and B4). The chlorine dosage always follows a linear distribution because the free chlorine has a linear dependency with it. Finally, the state ranges are defined based on the legal values required (see Supplementary material Appendix C).

The reward function designed for the multi-armed bandit is a combination of three different components (see Equation (3)). The first one, α, accounts for the effectiveness of the action taken; this component is also a prohibitive one, if the action does not provide drinkable water, the agent will not be able to choose it. The second component, β, refers to the amount of product used. The third one, γ, is calculated based on the water's turbidity before entering the sand filter. Replacing the filter is an expensive procedure so minimizing the water's turbidity before the filter elongates the time in which it can be operational. All the different components are multiplied by a factor, , that normalizes the maximal input for each component so that every factor is equally accounted for. These factors can be tweaked if the user values one factor more important than another (i.e., using less product before the longevity of the sand filter, then increases and decreases):
formula
(3)

Something to consider, the simulations were made with aluminium sulphate, Al2(SO4)3, as the coagulation agent. Therefore, the coagulation treatments recommended by the treatments suggestion model will be of g/L of Al2(SO4)3. If the DWTP does not use this type of coagulant, the user must make the normalization. The chlorine treatment is given in grams per liter.

Optimization: The first treatment suggestion model was generated with data from the simulator. During the data generation process, it was detected that the simulator was not capable of generating data with very high values in turbidity in the inlet water (values above 45–50 NTU). Therefore, a set of dedicated experiments was defined and executed to optimize the model. Furthermore, they were also used to check if the treatments suggested by the TD are correct, and to optimize them when necessary.

Regarding the check of the TM's suggested treatments: we have selected a total of 36 In-States and 35 Actions per In-State to be optimized based on some of the most usual values of the real DWTP. Ideally, all the 1,260 transitions should be characterized but because of time limitations (taking into account the DWTP replica's operation time is 11 h), it was decided to characterize the Actions that would be chosen with higher probability (for example studying the case of low coagulant for an extremely high turbidity would be a waste of time because that Action would never be chosen). Furthermore, since samples of historical inlet data from the real DWTP were also available, the experiments related to the most frequent In-States from the real DWTP were prioritized. If the Action suggested for the In-State returned an undesired Out-State, the treatment would be changed until the desired output was achieved. This information would later be introduced in the TD to make it more accurate.

Once the probable Actions were characterized, experiments with a remarkably high turbidity were also performed, not only because Stimela has a problem simulating those cases but also because our objective is to be able to predict the treatment for those extreme cases. These cases also would be later introduced in the TD to enhance accuracy.

The data obtained in experiments were also stored locally as hdf5 and excel files. As it was explained before, from the inlet parameters (pH, conductivity, and turbidity) it was considered an average value from the 4 h corresponding to the feed period of the sequential treatment. For outlet parameters (residual Cl and turbidity), the corresponding four last hours of the cycle when the water gets out (see Table 3).

Table 3

Experimental conditions considered for data extraction for artificial intelligent models: inputs (turbidity_in, conductivity, and pH), operation conditions (coagulant and chloride dose) and outputs (turbidity_ out and residual chlorine

PointTurb_IN (NTU)Cond. (μS/cm)pHCoagul. dose (g/L)Chloride dose (g/L)Turb_OUT (NTU)Res_Cl (mg/L)
5.2 552.7 7.7 0.000184 0.035 1.4 2.40 
5.1 552.2 7.7 0.000184 0.035 1.6 3.60 
5.2 552.7 7.7 0.000184 0.035 1.4 2.40 
5.1 552.2 7.7 0.000184 0.035 1.6 3.60 
11.5 775.8 7.6 0.000184 0.035 2.9 3.82 
16.7 472.0 7.4 0.000368 0.035 2.6 2.83 
21.8 473.0 7.4 0.000368 0.035 2.6 2.67 
24.0 359.6 7.3 0.000368 0.031 2.6 2.90 
26.1 361.0 7.5 0.000368 0.031 2.6 2.40 
62.4 470.0 7.5 0.023598 0.085 2.5 11.3 
76.9 420.0 7.7 0.023598 0.091 2.3 4.00 
10 102.4 430.9 7.6 0.023598 0.057 1.8 7.10 
11 54.3 394.4 7.5 0.023598 0.057 1.5 6.80 
12 43.8 397.3 7.5 0.023598 0.057 1.4 4.14 
13 43.6 413.2 7.5 0.023598 0.037 1.3 4.80 
14 41.7 414.8 7.6 0.023598 0.009 1.3 0.59 
15 28.7 440.0 7.5 0.000368 0.009 1.3 1.03 
16 7.3 401.0 7.6 0.000368 0.006 1.2 0.44 
17 4.3 438.1 7.6 0.000138 0.002 1.2 0.16 
18 6.3 411.4 7.6 0.000138 0.004 1.2 0.42 
19 6.0 442.7 7.6 0.000138 0.006 1.2 0.64 
20 2.7 447.2 7.7 0.000138 0.006 1.1 0.54 
21 12.9 440.3 7.5 0.000552 0.006 1.1 0.80 
22 10.5 517.7 7.6 0.000552 0.004 1.6 0.37 
23 3.6 531.2 7.6 0.000598 0.046 2.5 0.30 
24 7.5 426.3 7.4 0.000598 0.050 1.6 0.39 
25 4.3 428.3 7.7 0.000598 0.050 1.5 0.35 
26 3.8 433.1 7.8 0.000598 0.050 1.5 0.53 
27 3.2 419.8 7.7 0.000598 0.050 1.6 1.20 
28 2.7 468.2 7.7 0.000598 0.050 1.5 1.20 
29 8.4 418.1 7.5 0.000552 0.006 1.5 0.30 
30 13.7 466.2 7.5 0.000276 0.004 1.6 0.16 
31 9.6 465.0 7.5 0.000138 0.007 1.6 0.05 
32 5.1 466.9 7.7 0.000138 0.007 1.6 0.05 
33 4.2 350.0 7.6 0.000046 0.015 1.7 9.80 
34 26.9 408.5 7.7 0.000092 0.006 1.7 0.04 
35 14.4 411.7 7.7 0.000092 0.006 1.7 0.02 
36 16.1 414.4 7.7 0.000276 0.006 1.7 0.01 
37 35.5 341.2 7.8 0.000322 0.007 1.8 0.10 
PointTurb_IN (NTU)Cond. (μS/cm)pHCoagul. dose (g/L)Chloride dose (g/L)Turb_OUT (NTU)Res_Cl (mg/L)
5.2 552.7 7.7 0.000184 0.035 1.4 2.40 
5.1 552.2 7.7 0.000184 0.035 1.6 3.60 
5.2 552.7 7.7 0.000184 0.035 1.4 2.40 
5.1 552.2 7.7 0.000184 0.035 1.6 3.60 
11.5 775.8 7.6 0.000184 0.035 2.9 3.82 
16.7 472.0 7.4 0.000368 0.035 2.6 2.83 
21.8 473.0 7.4 0.000368 0.035 2.6 2.67 
24.0 359.6 7.3 0.000368 0.031 2.6 2.90 
26.1 361.0 7.5 0.000368 0.031 2.6 2.40 
62.4 470.0 7.5 0.023598 0.085 2.5 11.3 
76.9 420.0 7.7 0.023598 0.091 2.3 4.00 
10 102.4 430.9 7.6 0.023598 0.057 1.8 7.10 
11 54.3 394.4 7.5 0.023598 0.057 1.5 6.80 
12 43.8 397.3 7.5 0.023598 0.057 1.4 4.14 
13 43.6 413.2 7.5 0.023598 0.037 1.3 4.80 
14 41.7 414.8 7.6 0.023598 0.009 1.3 0.59 
15 28.7 440.0 7.5 0.000368 0.009 1.3 1.03 
16 7.3 401.0 7.6 0.000368 0.006 1.2 0.44 
17 4.3 438.1 7.6 0.000138 0.002 1.2 0.16 
18 6.3 411.4 7.6 0.000138 0.004 1.2 0.42 
19 6.0 442.7 7.6 0.000138 0.006 1.2 0.64 
20 2.7 447.2 7.7 0.000138 0.006 1.1 0.54 
21 12.9 440.3 7.5 0.000552 0.006 1.1 0.80 
22 10.5 517.7 7.6 0.000552 0.004 1.6 0.37 
23 3.6 531.2 7.6 0.000598 0.046 2.5 0.30 
24 7.5 426.3 7.4 0.000598 0.050 1.6 0.39 
25 4.3 428.3 7.7 0.000598 0.050 1.5 0.35 
26 3.8 433.1 7.8 0.000598 0.050 1.5 0.53 
27 3.2 419.8 7.7 0.000598 0.050 1.6 1.20 
28 2.7 468.2 7.7 0.000598 0.050 1.5 1.20 
29 8.4 418.1 7.5 0.000552 0.006 1.5 0.30 
30 13.7 466.2 7.5 0.000276 0.004 1.6 0.16 
31 9.6 465.0 7.5 0.000138 0.007 1.6 0.05 
32 5.1 466.9 7.7 0.000138 0.007 1.6 0.05 
33 4.2 350.0 7.6 0.000046 0.015 1.7 9.80 
34 26.9 408.5 7.7 0.000092 0.006 1.7 0.04 
35 14.4 411.7 7.7 0.000092 0.006 1.7 0.02 
36 16.1 414.4 7.7 0.000276 0.006 1.7 0.01 
37 35.5 341.2 7.8 0.000322 0.007 1.8 0.10 

Around 30,000 simulations were run to generate training data and 136 experiments were carried out in the replica to produce optimization data; some of which were discarded due to setup problems (water stirrer and sensor issues).

Model retraining: The TD at this point is created from simulated and replica data, but Stimela, as a numerical modeling program, and the replica, as a scaled version of a real DWTP, have some limitations and by themselves, they might be an insufficient way to represent the behavioural complexity of a real DWTP. This can be solved by introducing data from the real processes to the TD. For that purpose, an ‘Update Transition Dictionary’ (UTD) algorithm was developed. This algorithm was already used to take the data supplied by the replica to create a TD from the data, using the already predefined ranges. Then it performs a weighted sum between the TD created with the data from the simulations and the replica (pretrained and optimized model), and the TD generated using real data from a DWTP. The sum of both weights must be 1 so a new TD that has information on the real process, the replica, and the simulations is generated. The weight could be modified if it is preferred to give more importance to the simulated data or the introduced experimental data. For example, during the optimization, the TD was updated with experimental data provided by the replica with a weight of 50%. This means that the TD that was generated by simulated data has been optimized with the TD generated by the replica, both having the same importance. If the TD is updated regularly, the TD will end up being formed mainly by the experimental data. To apply the retraining, all the variables in their respective units (see Section 2) must be provided by the users.

The TD, as of right now, cannot be used to infer treatment to a different random DWTP (with different treatment processes). However, by using the UTD, the TD can be changed to represent the desired treatment from the new DWTP. For example, if the TD is updated given a weight of α (0 < α < 1 to the simulation data, considering that the TD is updated n-times, if n tends to be a high number the simulated data presence becomes null):
formula
(4)

Evaluation of the DWTP replica

A set of experiments was formulated with the purpose of validating the dosage results in the three operation conditions presented in Section 2. In Figure 4, the three settings are depicted as the coloured regions. The pH parameter was set to be between 7 and 7.5, although wider ranges are considered in drinking water sources. The water source of the replica usually took pH values between 7 and 7.5.
Figure 4

Conductivity vs inputted turbidity.

Figure 4

Conductivity vs inputted turbidity.

Close modal

Once the experiments were defined, the trained model was used to calculate the coagulant and chlorine dosages. In total, 17 valid experiments were done, four for normal conditions, seven for heavy rain conditions, and seven for saline contamination conditions.

Initial conditions (turbidity, pH, and conductivity): In Figure 4, the values of input turbidity and conductivity recorded in the laboratory are represented as the ‘x’ marks. The colour in each mark is related to the setting that the experiment was trying to reciprocate. The blue ‘x’ markers are not inside the blue region; the reason is they have a higher conductivity than the planned experiments accounted for. The other experiments are inside their respective regions.

On the other hand, the pH values do not follow the guidelines. In Figure 5, a histogram of the pH values is done, and the values are above the maximum limit imposed. This may be caused by a change in the water source. These experiments took place between the 17th of August and the 4th of October of 2022. During these months, a drought-affected Galicia (Spain), where the laboratory is located, dramatically reduced the flow of the river where the water was collected. That could be the main reason for the inlet pH rise. Nevertheless, the values are in the range of most drinking water catchments, so the validation was continued.
Figure 5

Histogram of the pH values recorded for each experiment. Note how the lowest pH recorded is higher than 7.5.

Figure 5

Histogram of the pH values recorded for each experiment. Note how the lowest pH recorded is higher than 7.5.

Close modal
Recalculating dosages: Because the pH levels differ so much from the initially proposed (between 7 and 7.5), the dosages calculated must be different from the ones the DSS would have suggested if the actual parameters were introduced. In Figure 6, the differences in the dosages are represented. The dots are the values that were calculated in the planning stage and the ‘x’ markers represent the dosages that the DSS would have returned if the real water parameters were inputted.
Figure 6

Comparison between the dosage values used for the experiments (‘o’) and the values that the DSS would have recommended considering the water turbidity, pH, and conductivity (‘x’).

Figure 6

Comparison between the dosage values used for the experiments (‘o’) and the values that the DSS would have recommended considering the water turbidity, pH, and conductivity (‘x’).

Close modal

As we can see in Figure 6, the actual coagulant dosages (‘x’ marks) are quite like the ones that were used in the experiments (dots). On the other hand, the chlorine dosage is very different from the one used. It must be noted that the actual chlorine dosage is usually higher than the one used.

There could be multiple reasons for this behaviour, but the most probable ones are chloride presence or other pollutants in the raw water, a change in the characteristics of the chloride product used, or even an undetected failure in the replica.

Results: Two parameters were measured at the outlet: turbidity and free chlorine. The turbidity is related to the coagulant dosage. The more coagulant dosage, the less turbidity in the water leaving the DWTP, and vice versa. The same can be said about the free chlorine and chlorine dosage relationship.

Output turbidity: The coagulant dosage used in the experiments is quite like the model's suggestions, so the results of the experiments can be used to prove the validity of the coagulant dosage provided by the model suggestion. In Figure 7, the turbidity of the water that leaves the replica is represented. The water is consistently under 1 NTU; reaching the higher values only with high levels of input turbidity, and never surpassing 2 NTUs. These results are satisfactory, even if the input water of the DWTP is of low quality, the output water is very clear. For example, in experiment 6, the input water is around 40 NTUs and by the time the water leaves the replica, it has a turbidity of 1 or 2 NTUs.
Figure 7

Turbidity and residual chlorine of the outlet for each experiment with their uncertainty. The water produced in experiments number 9, 10, and 16 is not acceptable for human consumption.

Figure 7

Turbidity and residual chlorine of the outlet for each experiment with their uncertainty. The water produced in experiments number 9, 10, and 16 is not acceptable for human consumption.

Close modal

Residual chlorine: In the case of the residual chlorine, the recommended dosages tend to be higher than the ones used during the experiments. As we can see in Figure 7, the residual chlorine tends to be higher than the desired maximum limit of 1 mg/L. Still, the vast part of the data is in the acceptable range for drinking water (up to 4 mg/L (Dery et al. 2019)).

As it was explained before, there could be multiple reasons for this behaviour, but the most probable ones are chloride presence or other pollutants in the raw water, a change in the characteristics of the chloride product used, or even an undetected failure in the replica.

Evaluation of a real DWTP

As part of the European H2020 project NAIADES, the models were also optimized for a real DWTP. The resultant model was provided to the DWTP's staff to evaluate the treatments suggested by the optimized model. For 2 weeks, the real DWTP collected the real values of pH, turbidity, and conductivity at the inlet of their plant to feed the model and obtain the suggestions.

The summary of the suggested vs the values used by the real plant is presented in Figure 8. The dosages calculated by the model are lower than the ones used by the real DWTP. The suggested coagulant dosage represents 73.26% of Braila's coagulant dosage, on average. The suggested chlorine dosage represents 70.68% of Braila's coagulant dosage, on average. So, the new treatments represent 26.74 and 29.32% average drop in chemical usage. The model was created to make a more efficient use of chemicals, so a drop was expected. Nevertheless, the DWTP's staff informed us this drop is too big.
Figure 8

Comparison between the DSS and Braila's dosage for coagulant and chlorine.

Figure 8

Comparison between the DSS and Braila's dosage for coagulant and chlorine.

Close modal

According to the DWTP's staff, the suggestions were within the correct order of magnitude, a fact which was already validated by the laboratory replica of the treatment plant. However, the amount is around 30% lower than the real-world dosage used. At that level, it was deemed that the treatment suggestion may not be sufficient.

The staff considers that one cause for the discrepancy may be the microbiological factor since the biological component of the water being treated is not considered. Although, this is only an educated guess and cannot be verified.

During the evaluation done by the staff, the following feedback was collected:

  • • The system is workable, with no issues when interacting with the solution.

  • • The treatment suggestions are within the same order of magnitude as the real-world values, which is a positive point. It means that the replica DWTP is close enough to its real-world equivalent.

  • • The exact values for the substances are smaller than the ones used at the real DWTP. One of the reasons is that the real DWTPs tend to overprovision treatments to account for unforeseen consequences.

  • • The values for the coagulant suggested by the system tended to be around 2.8–2.9, as opposed to the real-world values which were around 3.6–3.8.

  • • The values for the suggested chlorination dosage were around 1.2–1.3, whereas real-world values ranged from 1.6 to 1.8.

  • • The values are not considered as being high enough to provide sufficient water treatment on real DWTP′s.

  • • Other possible reasons related to the differences could be:

    • o the biological components, which are only considered through the turbidity, are underestimated.

    • o the industrial scale of the operation, 49,000 m3 per day, its sheer size, the equipment used and the maintenance levels, imply inefficiencies and quirks that are not still properly accounted for when optimizing the model from the smaller scale replica.

In this study, a methodology was presented to create an AI treatment suggestion model for basic DWTP's configuration, this is, with coagulation, sedimentation, filtration, and disinfection processes. The AI approach followed is similar to others which were applied to wastewater treatment plant processes. An AI model based on RL multi-armed bandits was created using data generated from a treatment processes simulator and it was optimized with data from a replica DWTP. The AI model was validated in this replica. Furthermore, the model was also optimized for a real DWTP and validated by its staff. As a result, it was concluded that the AI treatment suggestion model is capable of suggesting efficient treatment that guarantees drinking water at a replica scale for various scenarios (normal conditions and critical events) as it has been successfully demonstrated in the replica. Furthermore, the treatment suggestions for real DWTP are in the correct order of magnitude although the DWTP's staff consider them too low since they usually overprovision treatment dosages. Nevertheless, the staff will continue evaluating them since they consider the tool results as promising. The inlet water was characterized by three water quality parameters (pH, turbidity, and conductivity) to simplify the modelling. The future approach for the treatment's suggestion model should consider biological factors since turbidity may not be representative enough of all biological factors. Furthermore, the model should be optimized with more data from the real DWTP to better adapt to their needs, since the data the real plant provided for optimization were scarce. The solution provided with the model has an attribute that allows the decision matrix to be updated with historical data. Even if the actual treatments are not high enough right now, as time passes the system will learn to give higher dosages (closer to the ones expected by the real DWTP). Finally, further demonstration of the suggestions' reliability must be provided to the real DWTPs, so they start relying on the optimal suggested treatments and reducing their dependence on overprovisioned treatments.

Therefore, this work presents a promising DSS that recommends coagulant and chlorine dosages for basic-configuration DWTPs. The DSS does not focus only on one treatment process and on normal operation conditions, as much of the research done for DWTPs, but considers the whole process and critical events, such as heavy rains, that are prone to occur due to climate change. Furthermore, it proposes a retraining methodology to allow the DSS to adapt to water quality particularities and data shifts. In addition, the DSS has already been evaluated by a real DWTP. Nevertheless, there is still a lot of room for improvement.

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 820985.

All relevant data are available from an online repository or repositories: https://zenodo.org/records/7870529.

The authors declare there is no conflict.

Abba
S. I.
,
Nourani
V.
&
Elkiran
G.
2019
Multi-parametric modeling of water treatment plant using AI-based non-linear ensemble
.
Journal of Water Supply: Research and Technology – AQUA
68
(
7
),
547
561
.
Adams
H.
,
Burlingame
G.
,
Ikehata
K.
,
Furatian
L.
&
Suffet
I. H.
2022
The effect of pH on taste and odor production and control of drinking water
.
AQUA – Water Infrastructure, Ecosystems and Society
71
(
11
),
1278
1290
.
Aliashrafi
A.
,
Zhang
Y.
,
Groenewegen
H.
&
Peleato
N. M.
2021
A review of data driven modelling in drinking water treatment
.
Reviews in Environmental Science and Bio/Technology
20
(
4
),
985
1009
.
Bertone
E.
,
Sahin
O.
,
Richards
R.
&
Roiko
A.
2016
Extreme events, water quality and health: A participatory Bayesian risk assessment tool for managers of reservoirs
.
Journal of Cleaner Production
135
,
657
667
.
Bright
C.
&
Mager
S.
2018
Particulate organic matter, suspended sediment and turbidity in Otago, New Zealand tussock grasslands
. In
EGU General Assembly, 2018
.
Chen
K.
,
Wang
H.
,
Valverde-Pérez
B.
,
Zhai
S.
,
Vezzaro
L.
&
Wang
A.
2021
Optimal control towards sustainable wastewater treatment plants based on multiagent reinforcement learning
.
Chemosphere
279
, 130498.
Chiavola
A.
,
Di Marcantonio
C.
,
D'Agostini
M.
,
Leoni
S.
&
Lazzazzara
M.
2023
A combined experimental-modeling approach for turbidity removal optimization in a coagulation–flocculation unit of a drinking water treatment plant
.
Journal of Process Control
130
,
103068
.
Cox
R. A.
,
Culkin
F.
&
Riley
J. P.
1967
The electrical conductivity/chlorinity relationship in natural sea water
. Deep Sea Research 14 (2), 203–220.
Dery
J. L.
,
Brassill
N.
&
Rock
C. M.
2019
Minimizing Risks: Use of Surface Water in Pre-Harvest Agricultural Irrigation
.
The University of Arizona College of Agriculture and Life Sciences Tucson, Arizona, USA
.
Dudley
J.
,
Dillon
G.
&
Rietveld
L. C.
2008
Water treatment simulators
.
Journal of Water Supply: Research and Technology – AQUA
57
(
1
),
13
21
.
Fair
G. M.
&
Gemmell
R. S.
1964
A mathematical model of coagulation
.
Journal of Colloid Science
19
(
4
),
360
372
.
Farhaoui
M.
&
Derraz
M.
2016
Review on optimization of drinking water treatment process
.
Journal of Water Resource and Protection
8
(
8
),
777
786
.
Hernández-del-Olmo
F.
,
Gaudioso
E.
,
Dormido
R.
&
Duro
N.
2018
Tackling the startup of a reinforcement learning agent for the control of wastewater treatment plants
.
Knowledge-Based Systems
144
,
9
15
.
Hidayah
E. N.
&
Hendriyanto
O.
2014
Hydrodynamic model of sedimentation and disinfection to predict water quality in water treatment plant
.
International Journal of Science, Technology and Society
2
(
4
),
73
77
.
Inoue
J.
,
Yamagata
Y.
,
Chen
Y.
,
Poskitt
C. M.
&
Sun
J.
2017
Anomaly detection for a water treatment system using unsupervised machine learning
. In:
2017 IEEE International Conference on Data Mining Workshops (ICDMW)
.
IEEE
, New Orleans, pp.
1058
1065
.
Khan
S. J.
,
Deere
D.
,
Leusch
F. D.
,
Humpage
A.
,
Jenkins
M.
&
Cunliffe
D.
2015
Extreme weather events: Should drinking water quality management systems adapt to changing risk profiles?
Water Research
85
,
124
136
.
Kothari
V.
,
Vij
S.
,
Sharma
S.
&
Gupta
N.
2021
Correlation of various water quality parameters and water quality index of districts of Uttarakhand
.
Environmental and Sustainability Indicators
9
,
100093
.
Li
J.
,
Monroe
W.
,
Ritter
A.
,
Galley
M.
,
Gao
J.
&
Jurafsky
D.
2016
Deep reinforcement learning for dialogue generation. arXiv preprint arXiv:1606.01541
.
Liang
X.
,
Liu
Y.
,
Chen
T.
,
Liu
M.
&
Yang
Q.
2022
Federated transfer reinforcement learning for autonomous driving
. In:
Federated and Transfer Learning
(Razavi-Far, R., Wang, B., Taylor, M. E. & Yang, Q., eds).
Springer International Publishing
,
Cham
, pp.
357
371
.
Lu
X.
,
Welleck
S.
,
Hessel
J.
,
Jiang
L.
,
Qin
L.
,
West
P.
, Ammanabrolu, P. &
Choi
Y.
2022
Quark: Controllable text generation with reinforced unlearning
.
Advances in Neural Information Processing Systems
35
,
27591
27609
.
Martin
D. M.
&
Johnson
F. A.
2020
A multiarmed bandit approach to adaptive water quality management
.
Integrated Environmentalassessment and Management
16
(
6
),
841
852
.
Mnih
V.
,
Kavukcuoglu
K.
,
Silver
D.
,
Graves
A.
,
Antonoglou
I.
,
Wierstra
D.
&
Riedmiller
M.
2013
Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602
.
O'Reilly
G.
,
Bezuidenhout
C. C.
&
Bezuidenhout
J. J.
2018
Artificial neural networks: Applications in the drinking water sector
.
Water Science and Technology: Water Supply
18
(
6
),
1869
1887
.
Peinado-Guevara
H.
,
Green-Ruíz
C.
,
Herrera-Barrientos
J.
,
Escolero-Fuentes
O.
,
Delgado-Rodríguez
O.
,
Belmonte-Jiménez
S.
&
Ladrón de Guevara
M.
2012
Relationship between chloride concentration and electrical conductivity in groundwater and its estimation from vertical electrical soundings (VESs) in Guasave, Sinaloa, Mexico
.
Ciencia e investigación agraria
39
(
1
),
229
239
.
Perolat
J.
,
De Vylder
B.
,
Hennes
D.
,
Tarassov
E.
,
Strub
F.
,
de Boer
V.
, Muller, P., Connor, J. T., Burch, N., Anthony, T.,McAleer, S., Elie, R., Cen, S., Wang, Z., Gruslys, A., Malysheva, A., Khan, M., Ozair, S., Timbers, F. &
Tuyls
K.
2022
Mastering the game of Stratego with model-free multiagent reinforcement learning
.
Science
378
(
6623
),
990
996
.
Rowe
J.
2020
1.1: Chemical Feed Calculations in Water Treatment Processes – Workforce LibreTexts
.
Schiperski
F.
2018
Turbidity as an indicator of contamination in karst springs: A short review
. In:
Karst Groundwater Contamination and Public Health
. Advances in Karst Science (White, W., Herman, J., Herman, E. & Rutigliano, M., eds.). Cham, Springer, pp.
127
139
.
Serajuddin
M.
,
Chowdhury
A. I.
,
Haque
M.
&
Haque
E.
2019
Using turbidity to determine total suspended solids in an urban stream: A case study
. In:
Proceedings of the 2nd International Conference on Water and Environmental Engineering
,
Dhaka
, pp.
19
22
.
Shalev-Shwartz
S.
,
Shammah
S.
&
Shashua
A.
2016
Safe, multiagent, reinforcement learning for autonomous driving. arXiv preprint arXiv:1610.03295
.
Silva
T. C.
,
Eppink
M.
&
Ottens
M.
2022
Automation and miniaturization: Enabling tools for fast, high-throughput process development in integrated continuous biomanufacturing
.
Journal of Chemical Technology & Biotechnology
97
(
9
),
2365
2375
.
Sun
Y.
,
Vaidya
R.
,
Khunjar
W. O.
,
Rosenfeldt
E. J.
,
Selbes
M.
,
Wilson
C.
, Bott, C. B., Titcomb, M. &
Wang
Z. W.
2019
Mathematical modeling of biologically active filtration (BAF) for potable water production applications
.
Water Research
167
,
115128
.
Sutton
R. S.
&
Barto
A. G.
2018
Reinforcement Learning: An Introduction
.
MIT Press, Cambridge, MA, USA
.
Swan
R. W.
2015
Optimisation of Water Treatment Works Using Monte-Carlo Methods and Genetic Algorithms
.
Doctoral Dissertation
,
University of Birmingham
.
Van der Helm
A. W. C.
&
Rietveld
L. C.
2002
Modelling of drinking water treatment processes within the Stimela environment
.
Water Science and Technology: Water Supply
2
(
1
),
87
93
.
Van Moffaert
K.
,
Van Vaerenbergh
K.
,
Vrancx
P.
&
Nowé
A.
2014
Multiobjective X-armed bandits
. In
2014 International Joint Conference on Neural Networks (IJCNN)
.
IEEE
, pp.
2331
2338
,
2014, July
.
World Health Organization
2022
Guidelines for Drinking-Water Quality: Incorporating the First and Second Addenda
.
World Health Organization, Geneva, Switzerland
.
Yin
Y.
,
Le Guen
V.
,
Dona
J.
,
de Bézenac
E.
,
Ayed
I.
,
Thome
N.
&
Gallinari
P.
2021
Augmenting physical models with deep networks for complex dynamics forecasting
.
Journal of Statistical Mechanics: Theory and Experiment
2021
(
12
),
124012
.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying, adaptation and redistribution, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/).

Supplementary data