Reinforcement learning-based DSS for coagulant and disinfectant dosage selection on drinking water treatment plants

The water used for any domestic purposes must be wholesome. The requirements for treatment may vary according to the weather, the catchment activity, an accidental spillage, or other reasons. Although monitoring may indicate that the water is bacteriologically safe, sometimes, it is highly probable that there will be a bacteriological challenge (WHO 2022). The absence of microbiological contamination is generally the most important factor for human health as pathogens can lead to infectious diseases. Also, the substances in the water which only affect water's appearance, odour, or taste could make water objectionable to consumers and lead to rejection on aesthetic grounds.

Water treatment consists of sequential units to eliminate pollutants and pathogens. It usually includes pretreatment, coagulation, flocculation, sedimentation, filtration, and disinfection (Farhaoui & Derraz 2016; Chiavola et al. 2023). Most treatment systems are designed to remove microbiological contamination and physical constituents, such as suspended solids (turbidity), which affect aesthetic acceptability and prevent effective disinfection. Since microorganisms can be associated with particles and turbidity in water (Schiperski 2018; WHO 2022), physical contamination may represent a health risk and it makes disinfection more difficult.

Treatment models can help in understanding and predicting the treatment's effectiveness, especially in extreme events when abstraction water changes abnormally, like during storms or droughts (Aliashrafi et al. 2021). If these models have good accuracy, they could be used for treatment control to ensure the quality of the treated water, thereby mitigating any risks and protecting the drinking water supply.

There are mathematical models that describe and simulate the most common water treatment processes such as sedimentation, coagulation/flocculation, filtration, aeration, chemical oxidation, or granular activated carbon adsorption (Fair & Gemmell 1964; Hidayah & Hendriyanto 2014; Sun et al. 2019). These mathematical models are robust methods for the simulation of drinking water treatment plants' processes (DWTP's processes), allowing the DWTP staff to check the results of using different treatments. However, this implies the staff search for the best treatment using a try-and-error method (based on their expertise). The DWTP staff usually utilizes the treatment standards supplied by the treatment providers and by dedicated applications, which recalculate the treatments based on the current water flow (Rowe 2020). Nevertheless, none of their suggestions consider the peculiarities of each individual DWTP (small differences in water composition produced by local source water conditions and infrastructure materials). Data-driven approaches are recommended for these cases (Yin et al. 2021). Artificial intelligence (AI), and in particular machine learning methods such as artificial neural networks (ANN), and reinforcement learning (RL), are robust technologies that can handle the complex and dynamic nature of water treatment processes considering such individual particularities.

The goal of this research is the development of a decision support system (DSS) based on AI treatment models able to react to the events related to the quality of water in a dynamic data-driven approach, which allows for increased performance and safety of the current systems. The AI models are meant to provide efficient and robust treatment recommendations even when water quality changes occur due to extreme events such as heavy rains and droughts. Events that are not frequent, and that are expected to happen more often due to climate change, so the DWTP's staff are not prepared to react to them. Therefore, the outcome water treatment AI models are expected to help the DWTP's operators and managers monitor and control the processes. The purpose of the proposed DSS is to provide an augmented decision-making capacity for the most efficient treatments, especially during abnormal events which may be not part of the staff experience.

Currently, most of the research related to drinking water treatment plants using AI solutions focuses on forecasting some of the water quality parameters through the process (Abba et al. 2019) or focuses on part of the process, e.g., coagulant dosing prediction, or the prediction of the outcomes of the disinfection phase (O'Reilly et al. 2018; Li et al. 2021).

Our work presents a DSS that considers the whole treatment process to propose the dosages of coagulant and disinfectant, not only focusing on normal operation but also considering extreme events. An RL approach was used to create a model capable of estimating the most efficient dosages of coagulant and disinfectant that guarantee drinking water at the outlet of the treatment process for any water quality at the inlet. The DSS also includes a retraining routine to automatically adapt to the particularities of the different DWTPs. Data acquisition and generation was the main bottleneck of this work. The data from the real DWTP were scarce due to confidentiality. Therefore, these data were reserved for model optimization. The volume of data used for training was generated in a scaled-down replica of a real DWTP and a water treatment processes simulator. The model was tested in the replica and in the real DWTP. The data generated in the replica depended on the raw water used. This raw water was collected from a nearby water stream which was affected by unexpected climate events that slightly affected our results (see Section 3). Nevertheless, both replica and real DWTP results are promising, requiring improvements, such as including more quality parameters to increase robustness to biological changes. Furthermore, since DWTPs are critical infrastructures where it is not possible to do tests; it is necessary to further validate the results to be able to convince the DWTPs' staff of the reliability of the suggestions.

The content of this paper is structured as follows: Section 2 presents the methodology employed to develop the DSS, focusing on the data requirements and the RL algorithm used. Section 3 shows the results obtained during validation. The validation was done both under a controlled environment in a laboratory (replica) and in a real DWTP. Finally, the conclusions are presented in Section 4.

The validation in the replica and in the real DWTP is presented in Section 3.

For this study, they considered various DWTP's operation conditions to generate a resilient AI treatment suggestion model:

1. Normal operation is characterized by low input turbidity and conductivity.

2. Heavy rain which has high input turbidity.

3. Saline intrusion which has high conductivity.

Machine learning techniques are data-driven so they are highly dependent on data quality and quantity. Therefore, for each of those conditions, it was necessary to obtain enough data points. The data for this study were obtained from two sources: a DWTP's processes simulator and a scaled replica (see Section 2.2). For the purpose of simplifying the models in this study, the inlet water was characterized by pH, conductivity, and turbidity (see Table 1). Turbidity is important to correlate the solids and organic matter content of inlet water with the efficiency of the treatment (Shao et al. 2019). A change in pH could be a signal of high values in other parameters, such as organic matter or nitrogen forms (Adams et al. 2022). Conductivity is important for the detection of salinity intrusion and other sources of pollution (Adjovu et al. 2023). Their ranges were selected based on the real DWTP historical samples and related research (Khan et al. 2015; Bertone et al. 2016).

The outlet water was characterized by free chlorine and turbidity. It was considered that conductivity can be linked with chloride (Cox et al. 1967; Peinado-Guevara et al. 2012; Kothari et al. 2021), and turbidity can be an indirect measurement of total dissolved solids (TDS) (Serajuddin et al. 2019) and organic matter (Bright & Mager 2018). Based on WHO recommendations (WHO 2022), if the outlet water contains between 0.5 and 1 mg/L of free chlorine and the turbidity is below 5 NTU, it means there is no biological pollution present. However, depending on the country, these thresholds can be more relaxed.

The treatment processes considered in this study are three of the most common drinking water treatment processes: coagulation (with sedimentation), filtration and disinfection (chlorination).

• Coagulation is used to remove colour, turbidity, algae, and other microorganisms from surface waters. The addition of a chemical coagulant to the water causes the formation of a precipitate, or floc, which entraps these impurities. Iron and aluminium can also be removed under suitable conditions. The floc is separated from the treated water by sedimentation and/or filtration, although flotation processes may be used in place of sedimentation. The coagulant used in the experiments is an aluminium-based coagulant.

• Filtration. Turbidity and algae are removed from raw waters by screens, gravel filters, slow sand, rapid gravity filters, or cartridge filters. The difference between slow and rapid sand filtration is not a simple matter of the speed of filtration, but it is in the underlying concept of the treatment process. Slow sand filtration is essentially a biological process whereas rapid sand filtration is a physical treatment process.

• Disinfection. Contamination by sewage or animal faeces is the greatest danger associated with drinking water. This is because sewage from human or animal sources may contain the causative organisms of many communicable diseases. The use of disinfectants to kill or inactivate pathogenic microorganisms is necessary if the raw water contains such organisms. Several disinfection methods are used in water treatment. Disinfection with chlorine is the most widely used method for large water supplies but its application is less common in small supplies.

The next sections describe the methodology followed during this study based on the aforementioned DWTP's configuration. Section 2.2 provides more details about the two main sources of data, which were also used for optimization and validation. Section 2.3 describes the AI modelling carried out to create the treatment suggestion model.

The full setup adds up to a total residence time of 11 h. The selected sensors are based on commercial availability, at a medium-low price, to be affordable for any size DWTP. The list of sensors is:

• Turbidity (inlet & outlet) – Turbimax CUS51D.

• pH (inlet) – Memosens CPS31D.

• Conductivity (inlet) – Condumax CLS21D.

• Chlorine (outlet) – Memosens CCS51D.

The raw water (without treatment) was collected from a stream near AIMEN's facilities. This stream passes through forests and a rural zone, so it is lowly polluted, and its quality highly depends on the weather conditions. Fresh water was transported daily to AIMEN's facilities. Then, synthetic water was produced to obtain the desired water conditions. The synthetic water was prepared by mixing the raw water with sediments from the catchment, marine salt, and wastewater treatment plant's sludge. A minimum of 25 L per batch was prepared. The pH, conductivity and turbidity sensors were used to check if the desired experiment's initial conditions for the inlet water were set up correctly.

Dilutions of NaOH and HNO₃ were used to control the pH. A coagulation/flocculation commercial product for drinking water was used in the precipitation process. Most commercial products are based on ferric chloride or aluminium salts. The selected product is based on aluminium sulphate salts (25% in weight of Al₂(SO₄)₃). The recommended dosage by the manufacturer (3 mg/L treat water) was used in normal operation conditions and it was adapted for extreme events conditions. This methodology provided information about the correlation between the product dosage and the inlet water characteristics (mainly turbidity and conductivity). To change the coagulant dosage, a solution with distilled water was made. As a baseline, 1 mL of raw commercial product per litre was considered, but it was modified to varying operation conditions.

A chlorine solution was used to obtain a Cl concentration in the outlet water up to 0.5 mg/L, to ensure drinking water microbiological quality. The maximum Cl concentration should be less than 1 mg/L in the DWTP replica's outlet to avoid ClO₂ formation. A commercial liquid solution (37% in weight of sodium hypochlorite) was selected because of its ease of manipulation. As well as in the coagulant case, a solution in distilled water was prepared (approximately 1 mL of raw commercial product per L).

The solutions were dosed using peristaltic pumps with a timer controlled by a PC. The replica is controlled by a single-board PC (Nvidia Jetson Nano). It temporizes the pumps, to operate them sequentially in the selected retention time (11 h). This retention time is based on data from a real DWTP.

The replica was built in 2019 and finished by mid-2020. The data from the replica were generated from 2020 to 2022. It was used for validation during the second half of 2022.

Stimela: Even though we generated data with the replica, the gross of the data produced to train the AI models was simulated. The replica's residence time of 11 h limited the amount of data that could be generated; therefore, a simulator was necessary. Stimela is a mathematical model of drinking water treatment processes embedded in MATLAB/Simulink (Van der Helm & Rietveld 2002; Dudley et al. 2008). It was developed by Delft University of Technology, Kiwa and Royal Haskoning DHV. Stimela offers many models that can be interconnected to represent drinking water plants such as filtration modules, flocculation modules, aeration modules, etc. Stimela simulates the changes experimented in the water quality parameters when the water is processed through the modules.

Despite the fact that Stimela simulator has not been maintained since 2012 (its last version), it has been selected due to its immediate accessibility, ease of use and free license. There are other mathematical simulators for drinking water, although not as many as for water treatment. Table 2 shows a short comparison between some of the reviewed tools for drinking water process simulation (Dudley et al. 2008; Swan 2015).

Then, the water is mixed with the coagulant. To simulate this process, three modules are needed, the ‘doscon’ (in which the quantity and type of coagulant are chosen), ‘coagul’ and ‘vlokvo’ (both modules are needed to simulate the mixing of water with coagulant). The following process is the filtering done in the ‘Single media filter’, the module ‘bacwa 1’ simulates the backwashing needed to clean the filter periodically. The water leaves the filter to enter the chlorination process, ‘chloro_m’ module. The ‘chloro_m’ module is also connected to ‘chldos’ which is a module used to specify the quantity of chlorine used in the chlorination process.

Through the process, there are two other types of modules, the ‘Graphical outputs’ which are used to visualize the water parameters through different stages of the process, and the blocks which have a name with the extension ‘.mat’. These are used to export the water quality parameters through various stages of the process. All the modules have a set of parameters associated with them. The parameters were chosen based on a real DWTP associated with the NAIADES project to make the simulations as accurate as possible.

The variables that were determined important to this study were the turbidity, pH, and conductivity of the water entering the DWTP, the coagulant dosage and chlorine dosage for the treatment, and the turbidity and free chlorine of the water leaving the DWTP. For the variables referring to the raw water some ranges were determined, those ranges were calculated considering that a change of range implied a significant change in the treatment dosage. The treatment parameters were chosen based on the fact that a change from one option to another of treatment would imply a significant difference in the quality of the water leaving the DWTP. Finally, the parameters of the exiting water were also classified in ranges, but these were based on health, safety, and legal limits.

Ideally, the same sampling process that was done in the turbidity would be done in the pH and the conductivity as well, but the computing time scales linearly with the number of samples taken and our simulations already took several days, and since it was decided that the replica will not experiment with those values due mainly to time restrictions, then it was deemed unnecessary. This decision would reduce the accuracy of the suggestions for those cases, making those suggestions less efficient but still valid, though future versions of the DSS should be retrained including all the possible data combinations. Stimela was configured by the end of 2020, it was tested during 2021, and it was used to produce data during the end of 2021 and the beginning of 2022.

RL is a branch of the AI approaches which consist of selecting the best actions for an agent in a defined environment to reach the states with the maximum reward. In each event the agent receives some representation of the environment's state; based on that, it selects an action. As a consequence of the action, the agent receives a reward and finds itself in a new state. It was decided to use RL due to its auto-learning capabilities (Sutton & Barto 2018).

RL is very versatile, and it is used in a lot of applications in different industries: algorithms that play games (Mnih et al. 2013; Perolat et al. 2022), self-driving cars (Shalev-Shwartz et al. 2016; Liang et al. 2022), text generation (Li et al. 2016; Lu et al. 2022), etc. In fact, RL algorithms have already been applied to water treatment plant processes and water management with promising results (Hernández-del-Olmo et al. 2018; Chen et al. 2021; Makropoulos & Bouziotas 2023). The work by Chen et al. (2021) presents a multiagent RL approach to optimize two agents, the amount of chemical dosages and dissolved oxygen, in a waste water treatment plant (WWTP). The RL was trained for different objectives, such as the optimization of the environmental impact, based on a life cycle analysis, and cost reduction. Each of those objectives had a specific reward policy. Their results showed beneficial results for the WWTP, like the reductions of costs and energy by reducing the aeration and chemical dosages. Hernández-del-Olmo et al. (2018) proposed a solution to make the RL systems to be trustable by the WWTP managers so the RL could be applied directly to control the plant. Their RL approach-based on one agent tries to minimize the operational costs (costs derived from energy consumption and disposal of waste sludge). The costs are used as a reward to teach the agent based on the environment state (oxygen and N-ammonia values) reached after a set of actions (dissolved oxygen concentration). Their RL approach showed improvement in the N-ammonia removal process and reduced costs in comparison with the operators' decisions.

Among the RL methods, the Q-learning algorithm (Hernández-del-Olmo et al. 2018) was studied. Q-learning is an algorithm that looks for the optimal action to be applied to a specific state. When Q-learning is performed, a Q-table is created following the shape of [state, action], with the values initialized to zero. Those values are updated after an episode (simulation of all the possible states). The Q-table becomes a reference table for selecting the best action based on the Q-value. Some approaches were considered in this case, but the main problem was determining the states. They must contain the Inlet Water Quality parameters, but they also have to contain the Outlet Water Quality parameters so one can update the Q-table. A state that has information on both the Inlet and Outlet Water Quality parameters does not have a physical meaning and it was decided to drop this convoluted method.

Then, a multi-armed bandit model was considered. This specific type of RL has already been studied and suggested for some processes of WWTPs (Van Moffaert et al. 2014; Martin & Johnson 2020). The paper in Van Moffaert et al. (2014) presents a new approach to multi-armed bandits, the multi-objective X armed bandits. They mention that multi-armed bandits are recommended for problems like water purification in WWTP, this is a problem where multiple problems are trying to be tackled at the same time, with water purification minimizing the operational costs and the amount of sludge. Martin & Johnson (2020) studied different multi-armed bandits strategies based on the assumptions about the monitored data, in this case, using wastewater treatments as an example. The problem they aim to tackle is to find the best treatment (they consider three different treatment technologies) to remove the N effluent. Although this paper was focused on the different multi-armed bandits' strategies, their findings show that the multi-armed approach could have suggested the treatments that could have reduced the N effluent by more than 25%. In the RL multi-armed bandits modeling there is a fixed limited set of resources. The agent must pick between a set of choices which have a probability distribution that is partially known at the time of picking. There is a limited number of picks, the agent must maximize the total reward. In our case, the inlet water quality parameters define the environment, the treatment combinations are the actions (see examples in Supplementary material, Appendix B), and the outlet water quality parameters define the States (see examples in Supplementary material, Appendix C). Different Environments were considered (see Supplementary material, Appendix A).

The difficulty in the creation of the TD is the correct definition of the environment, actions and state. For example, for the TD related to the higher turbidity values, the turbidity range was taken following an exponential distribution (see Supplementary material, Table A1) meanwhile the turbidity range for the sets with lower input turbidity follows a lineal distribution (see Supplementary material, Table A2); for each case (low and high turbidity) 36 States and 35 Actions were defined, reaching 72 States and 70 actions in total. This is due to the relationship between the turbidity and the coagulant dosage. For higher input turbidity, it is needed a higher dosage of coagulant to maintain the output turbidity constant. But the amount of coagulant increases in an exponential way in relation to the input turbidity, so the turbidity range must follow an exponential distribution. For the lower turbidity cases, that exponential behaviour is not so significant, and a linear distribution creates the desired States. This is also the reason behind the fact that the range taken for the coagulant dosage for the high turbidity's case also follows an exponential distribution (see Supplementary material, Tables B1 and B2) meanwhile the coagulant dosage for the lower turbidity's values follows a linear distribution (see Supplementary material, Tables B3 and B4). The chlorine dosage always follows a linear distribution because the free chlorine has a linear dependency with it. Finally, the state ranges are defined based on the legal values required (see Supplementary material Appendix C).

Something to consider, the simulations were made with aluminium sulphate, Al₂(SO₄)₃, as the coagulation agent. Therefore, the coagulation treatments recommended by the treatments suggestion model will be of g/L of Al₂(SO₄)₃. If the DWTP does not use this type of coagulant, the user must make the normalization. The chlorine treatment is given in grams per liter.

Optimization: The first treatment suggestion model was generated with data from the simulator. During the data generation process, it was detected that the simulator was not capable of generating data with very high values in turbidity in the inlet water (values above 45–50 NTU). Therefore, a set of dedicated experiments was defined and executed to optimize the model. Furthermore, they were also used to check if the treatments suggested by the TD are correct, and to optimize them when necessary.

Regarding the check of the TM's suggested treatments: we have selected a total of 36 In-States and 35 Actions per In-State to be optimized based on some of the most usual values of the real DWTP. Ideally, all the 1,260 transitions should be characterized but because of time limitations (taking into account the DWTP replica's operation time is 11 h), it was decided to characterize the Actions that would be chosen with higher probability (for example studying the case of low coagulant for an extremely high turbidity would be a waste of time because that Action would never be chosen). Furthermore, since samples of historical inlet data from the real DWTP were also available, the experiments related to the most frequent In-States from the real DWTP were prioritized. If the Action suggested for the In-State returned an undesired Out-State, the treatment would be changed until the desired output was achieved. This information would later be introduced in the TD to make it more accurate.

Once the probable Actions were characterized, experiments with a remarkably high turbidity were also performed, not only because Stimela has a problem simulating those cases but also because our objective is to be able to predict the treatment for those extreme cases. These cases also would be later introduced in the TD to enhance accuracy.

The data obtained in experiments were also stored locally as hdf5 and excel files. As it was explained before, from the inlet parameters (pH, conductivity, and turbidity) it was considered an average value from the 4 h corresponding to the feed period of the sequential treatment. For outlet parameters (residual Cl and turbidity), the corresponding four last hours of the cycle when the water gets out (see Table 3).

Around 30,000 simulations were run to generate training data and 136 experiments were carried out in the replica to produce optimization data; some of which were discarded due to setup problems (water stirrer and sensor issues).

Model retraining: The TD at this point is created from simulated and replica data, but Stimela, as a numerical modeling program, and the replica, as a scaled version of a real DWTP, have some limitations and by themselves, they might be an insufficient way to represent the behavioural complexity of a real DWTP. This can be solved by introducing data from the real processes to the TD. For that purpose, an ‘Update Transition Dictionary’ (UTD) algorithm was developed. This algorithm was already used to take the data supplied by the replica to create a TD from the data, using the already predefined ranges. Then it performs a weighted sum between the TD created with the data from the simulations and the replica (pretrained and optimized model), and the TD generated using real data from a DWTP. The sum of both weights must be 1 so a new TD that has information on the real process, the replica, and the simulations is generated. The weight could be modified if it is preferred to give more importance to the simulated data or the introduced experimental data. For example, during the optimization, the TD was updated with experimental data provided by the replica with a weight of 50%. This means that the TD that was generated by simulated data has been optimized with the TD generated by the replica, both having the same importance. If the TD is updated regularly, the TD will end up being formed mainly by the experimental data. To apply the retraining, all the variables in their respective units (see Section 2) must be provided by the users.

Once the experiments were defined, the trained model was used to calculate the coagulant and chlorine dosages. In total, 17 valid experiments were done, four for normal conditions, seven for heavy rain conditions, and seven for saline contamination conditions.

Initial conditions (turbidity, pH, and conductivity): In Figure 4, the values of input turbidity and conductivity recorded in the laboratory are represented as the ‘x’ marks. The colour in each mark is related to the setting that the experiment was trying to reciprocate. The blue ‘x’ markers are not inside the blue region; the reason is they have a higher conductivity than the planned experiments accounted for. The other experiments are inside their respective regions.

As we can see in Figure 6, the actual coagulant dosages (‘x’ marks) are quite like the ones that were used in the experiments (dots). On the other hand, the chlorine dosage is very different from the one used. It must be noted that the actual chlorine dosage is usually higher than the one used.

There could be multiple reasons for this behaviour, but the most probable ones are chloride presence or other pollutants in the raw water, a change in the characteristics of the chloride product used, or even an undetected failure in the replica.

Results: Two parameters were measured at the outlet: turbidity and free chlorine. The turbidity is related to the coagulant dosage. The more coagulant dosage, the less turbidity in the water leaving the DWTP, and vice versa. The same can be said about the free chlorine and chlorine dosage relationship.

Residual chlorine: In the case of the residual chlorine, the recommended dosages tend to be higher than the ones used during the experiments. As we can see in Figure 7, the residual chlorine tends to be higher than the desired maximum limit of 1 mg/L. Still, the vast part of the data is in the acceptable range for drinking water (up to 4 mg/L (Dery et al. 2019)).

As it was explained before, there could be multiple reasons for this behaviour, but the most probable ones are chloride presence or other pollutants in the raw water, a change in the characteristics of the chloride product used, or even an undetected failure in the replica.

As part of the European H2020 project NAIADES, the models were also optimized for a real DWTP. The resultant model was provided to the DWTP's staff to evaluate the treatments suggested by the optimized model. For 2 weeks, the real DWTP collected the real values of pH, turbidity, and conductivity at the inlet of their plant to feed the model and obtain the suggestions.

According to the DWTP's staff, the suggestions were within the correct order of magnitude, a fact which was already validated by the laboratory replica of the treatment plant. However, the amount is around 30% lower than the real-world dosage used. At that level, it was deemed that the treatment suggestion may not be sufficient.

The staff considers that one cause for the discrepancy may be the microbiological factor since the biological component of the water being treated is not considered. Although, this is only an educated guess and cannot be verified.

During the evaluation done by the staff, the following feedback was collected:

• • The system is workable, with no issues when interacting with the solution.

• • The treatment suggestions are within the same order of magnitude as the real-world values, which is a positive point. It means that the replica DWTP is close enough to its real-world equivalent.

• • The exact values for the substances are smaller than the ones used at the real DWTP. One of the reasons is that the real DWTPs tend to overprovision treatments to account for unforeseen consequences.

• • The values for the coagulant suggested by the system tended to be around 2.8–2.9, as opposed to the real-world values which were around 3.6–3.8.

• • The values for the suggested chlorination dosage were around 1.2–1.3, whereas real-world values ranged from 1.6 to 1.8.

• • The values are not considered as being high enough to provide sufficient water treatment on real DWTP′s.

• • Other possible reasons related to the differences could be:

• • o the biological components, which are only considered through the turbidity, are underestimated.

  • o the industrial scale of the operation, 49,000 m³ per day, its sheer size, the equipment used and the maintenance levels, imply inefficiencies and quirks that are not still properly accounted for when optimizing the model from the smaller scale replica.

In this study, a methodology was presented to create an AI treatment suggestion model for basic DWTP's configuration, this is, with coagulation, sedimentation, filtration, and disinfection processes. The AI approach followed is similar to others which were applied to wastewater treatment plant processes. An AI model based on RL multi-armed bandits was created using data generated from a treatment processes simulator and it was optimized with data from a replica DWTP. The AI model was validated in this replica. Furthermore, the model was also optimized for a real DWTP and validated by its staff. As a result, it was concluded that the AI treatment suggestion model is capable of suggesting efficient treatment that guarantees drinking water at a replica scale for various scenarios (normal conditions and critical events) as it has been successfully demonstrated in the replica. Furthermore, the treatment suggestions for real DWTP are in the correct order of magnitude although the DWTP's staff consider them too low since they usually overprovision treatment dosages. Nevertheless, the staff will continue evaluating them since they consider the tool results as promising. The inlet water was characterized by three water quality parameters (pH, turbidity, and conductivity) to simplify the modelling. The future approach for the treatment's suggestion model should consider biological factors since turbidity may not be representative enough of all biological factors. Furthermore, the model should be optimized with more data from the real DWTP to better adapt to their needs, since the data the real plant provided for optimization were scarce. The solution provided with the model has an attribute that allows the decision matrix to be updated with historical data. Even if the actual treatments are not high enough right now, as time passes the system will learn to give higher dosages (closer to the ones expected by the real DWTP). Finally, further demonstration of the suggestions' reliability must be provided to the real DWTPs, so they start relying on the optimal suggested treatments and reducing their dependence on overprovisioned treatments.

Therefore, this work presents a promising DSS that recommends coagulant and chlorine dosages for basic-configuration DWTPs. The DSS does not focus only on one treatment process and on normal operation conditions, as much of the research done for DWTPs, but considers the whole process and critical events, such as heavy rains, that are prone to occur due to climate change. Furthermore, it proposes a retraining methodology to allow the DSS to adapt to water quality particularities and data shifts. In addition, the DSS has already been evaluated by a real DWTP. Nevertheless, there is still a lot of room for improvement.

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 820985.

All relevant data are available from an online repository or repositories: https://zenodo.org/records/7870529.

The authors declare there is no conflict.