Performance improvement of wastewater treatment processes by application of machine learning

Improving wastewater treatment processes is becoming increasingly important. This is caused by more stringent effluent quality requirements and the need to reduce both energy consumption and chemical dosing. At the same time, decreasing availability of skilled operators can demand an autopilot solution. Such challenges also apply to PUB, Singapore's National Water Agency. Growing water demands and rising operational costs require innovative technologies to guarantee water safety and security.

To assess the feasibility of deploying innovative process technologies in water reclamation plants (WRPs), an integrated validation plant (IVP) was commissioned in 2017. IVP is a demonstration plant at Ulu Pandan WRP which has a capacity of ≈12,500 m³/day. It has a membrane bioreactor (MBR) system, where the bioreactors consist of non-aerated and aerated plug flow step-feed activated sludge basins for organics and nutrient removal, and MBR membranes for sludge water separation. A schematic overview of the IVP is shown in Figure 1.

The IVP consists of two trains namely Train 1 and Train 2. Each train has an aerated plug flow step-feed reactor with five basins. Each basin has two zones with adjustable valves for the air flow and online dissolved oxygen (DO) measurements. Basins 1, 3 and 5 are equipped with online measurements for ammonium (NH₄), and additional measurements of nitrite (NO₂) and nitrate (NO₃) for Basin 5. In addition, the ammonium concentrations and flow rates of the effluent of the fine screens to the reactors are measured online. This makes it possible to determine the ammonium influent load to each bioreactor.

An example of these innovative technologies applied to the IVP is the development and deployment of an autopilot solution with advanced analytics and predictive control for the biological processes of the WRP. The aim of this autopilot solution is to achieve stable operation while meeting product water quality as feed for the downstream reverse osmosis (RO) process. Other objectives include an accurate influent load prediction for optimised nutrient removal which in turn substantially reduces aeration energy usage. Early warnings for the process operators for process and instrument anomalies are also achieved with the solution.

To achieve these goals, application of self-learning predictive control, instead of reactive control, is preferred. Conventional control solely using proportional-integral-derivative (PID) loops can cause instabilities as it can only be reactive in the shorter term and cannot be anticipatory in the longer term. In wastewater treatment, the key aberrations are primarily caused by the daily pattern of the influent load and rain events and should, therefore, be regarded for optimal operation (de Koning et al. 2013). Dry-weather flow (DWF) can be predicted, and rain-weather flow (RWF) detected. Utilising these in predictive control enables stable processes as it accurately forecasts the requested output and subtly adjusts for target variations. Additionally, predictive analytics provides the opportunity for early detection of anomalies before measured and predicted values diverge sharply.

In this study, the methods and results of the application of predictive control and advanced analytics in the IVP are explained and discussed.

The autopilot solution consists of an advanced analytics and predictive control system for the IVP. A brief description of the methods and techniques used is subsequently presented for each system.

The predictive control applies self-learning feedforward control using load prediction and machine learning fine-tuned with feedback control on effluent ammonium levels. The determination of the aeration setpoints consists of several steps which are described below.

Firstly, an influent ammonium load prediction is determined with a self-learning algorithm (Bakker et al. 2013a, 2013b). It captures the day-patterns in hourly (or quarterly) values, obtaining the regular day-patterns for each day of the week. It has a 48 (or 72) h prediction horizon. Days with large deviations are rejected automatically. This occurs if any of the array elements of the dimensionless daily curve factors exceeds the maximum allowable error, which has an adjustable default value (½). The day-patterns can also be automatically rejected due to the signal status indicating an invalid value or manually rejected by the operator. Each daily data is classified with a reason for approval or rejection. Days with deviations not exceeding the tolerance factor are approved but labelled with a warning.

This specific technique was originally developed as a fully adaptive model for the prediction of the short-term drinking water demand. Subsequently, this technique was also successfully applied for dry-weather flow (DWF) prediction for aeration control (de Koning et al. 2013) and also combined with rainfall forecast data, to carry out flow control (Icke et al. 2017). For the IVP, it is used to predict the influent flow and diurnal pattern of ammonium concentration.

Secondly, the actual influent load prediction is used in a self-learning regression model which estimates the required amount of aeration for each train. It aggregates the online measured values of aeration flow and influent load to averages and assigns weighting factors. As recent aggregates are more significant for the current process, the weighting factors start at one for the most recent ones (over a range of few days) and decreases linearly to zero (over a range of few weeks). Invalid values, due to an erroneous signal or being outside the fit constraints, are automatically rejected. The aggregates and fit are updated multiple times a day. This makes the regression model with the self-learning prediction robust: it adapts quickly, and it hardly requires maintenance.

Thirdly, the resulting estimated amount of required aeration flow is subsequently fine-tuned with feedback control. This is obtained with a proportional-integral (PI) controller on ammonium for each separated bioreactor. Additionally, for each aerator valve, the amount of aeration is adjusted with a PI controller on oxygen, fine-tuning the aeration flow per aeration zone (Yuan et al. 2019). In this way, the amount of aeration is adapted to the needs of each train and specific zone to achieve optimised control on the dynamic setpoints for ammonium and DO.

Finally, the amount of aeration per train can be dynamically suppressed with an adjustable limit for nitrate. In case nitrate concentrations exceed the ammonium concentrations, aeration is suppressed to prevent over aeration, balancing ammonium versus nitrate concentrations, thereby saving energy.

The above-mentioned predictive control philosophies were installed within an on-premise configuration for real time aeration control of each of the IVP bioreactors.

The advanced analytics applies neural network modelling for early detection of different anomalies. The detection of the anomalies consists of several steps which are described sequentially.

Firstly, pre-processing of the data is executed. All data are normalized with respect to their mean and standard deviation (i.e. z-scored). Missing values are inserted using the arithmetic mean value for the respective feature. All data are resampled to the median frequency of the target variable using mean aggregation. Additionally, measurements that fall outside an expected ratio are qualified as not a number (NaN), which are consecutively filled with the mean. Valid ranges are defined for each signal separately.

Secondly, to detect anomalies in signal time series, an artificial neural network (ANN) is applied: quantile regression in combination with multi-layer perceptron (QR-MLP) (Koenker 2005; Hastie et al. 2009). This approach created predicted intervals within which expected data values should lie. The main benefit of using a neural network is that different and tilted loss-functions can be implemented for each output unit (Cannon 2018; Rodrigues & Pereira 2018). In this way, the network can output a range of quantiles for every timepoint in a single architecture.

The neural networks all contain two hidden layers with 64 neurons each. Each hidden layer is followed by a rectified linear unit (Re LU) activation function (Hinton 2010) and then followed by a dropout layer (Srivastava et al. 2014) with a rate of 0.2. The networks are trained using a batch size of 64 and a learning rate of 0.0005. Training lasts for a maximum of 200 epochs but stops earlier if validation loss does not improve for five consecutive epochs.

Finally, post-processing of the results is executed. Resampling of data is applied, and data resolution can range from 5 min refresh rate up until 24 h. The fine resolution can highlight individual datapoint outliers, whereas the coarse resolution can highlight trend outliers, or drifts. The QR-MLP modelling including pre- and postprocessing is applied to the IVP for the detection of process and instrument anomalies. Several anomaly models are subsequently developed.

One of the developed models predicts the aeration flow needed for each train of the IVP from the influent load. It is plausible that higher influent load results in increased oxygen demand from the process and, therefore, resulting in a higher aeration flow. However, if measured aeration flow increases and cannot be explained from the measured influent load, it can be detected as a process anomaly as it exceeds the predicted interval for aeration flow. As input to the model, the ammonium influent load was summarised as the current value in addition to the value of ¼, ½, 1, 2, 4, 6, 8, 16 and 32 hours ago, as well as the mean of the past of ¼, ½, 1, 2, 4, 6, 8, 16 and 32 h. In addition, one-hot-encoded features that represented day of the week and cyclically encoded features that represented second-of-day, were setup. The latter two feature categories serve to capture daily patterns, that can differ across each weekday. In total, this resulted in 28 features.

Another model predicts the ammonium concentration in the effluent of each IVP bioreactor, based on the influent ammonium load and the ammonium concentration measured in the basins of the other bioreactor. If the measured ammonium effluent concentration deviates from the expected range, based on ammonium measurements from the influent and from the other basins, it shall be designated as an instrument anomaly. In this model, the current values of all measurements were populated. In addition, one-hot-encoded features that represented day of the week and cyclically encoded features that represented second-of-day were setup again. This resulted in 13 features.

A similar approach was applied for nitrate instruments. The nitrate concentration in the effluent of each bioreactor of the IVP is predicted based on the current value of ammonium load in the influent and the online ammonium measurements in the basins of all bioreactors. Again, one-hot-encoded features that represented day of the week and cyclically encoded features that represented second-of-day were added. In total, this resulted in 16 features.

Before implementation and integration in the online advanced analytics environment in the cloud, these models were trained on data from 15 May 2019 to 31 July 2019 and were evaluated on the period between 1 and 16 August 2019. Due to the equatorial location (climate) of IVP (Ulu Pandan Water Reclamation Plant in Singapore), process temperature remained quite constant throughout the year. Therefore, a regular (e.g. monthly) retraining schedule should cover any remaining changes after implementation.

The autopilot solution consists of an advanced analytics and predictive control system for the IVP. Results of each system are presented and discussed below.

After implementation, Train 1 became operational on 11 April 2019, followed by Train 2 on 13 May 2019. The predicted and measured influent ammonium load values are shown in Figure 2. After an initial training period (until 21 June 2019), the prediction algorithm has learned the daily patterns of the influent load. The influent load is then (from 21 June 2019) predicted with a usable forecast horizon of 48 h.

In Figure 3, an XY plot of predicted and measured values of the influent ammonium load is shown. Operational results with predictive control show that the load prediction has an accuracy (R²) of 88%. This is verified for the period 21 June–21 August 2019, and rainfall influence has not been filtered out. The prediction of the load pattern is self-learning without further manual intervention required. Using the predicted instead of the measured value of the influent load as an input for feedforward has the advantage of considering the daily pattern rather than short-term variations.

In Figure 4, an XY plot of aeration flow and influent load is showing a fit line through the aggregates. The figure shows a clear relationship between aeration flow and load which confirms that the amount of aeration needed for feedforward control can be estimated very accurately based on fit and prediction. The linear regression is refit periodically without manual tuning required.

Results show that up to 15% reduction of aeration amount can be achieved as compared to conventional control with ammonium level near setpoints. This is determined by comparison of the aeration of one train operating with predictive control (Train 1: avg. of 833 Nm³/h), and another train operating with conventional control (Train 2: avg. of 981 Nm³/h) for the period 11 April–11 May 2019. The 15% reduction of aeration amount also yields a corresponding reduction of aeration energy.

For illustration of the data science models, an event at IVP was selected. Figure 5 shows the data trend for ammonium load in the influent and concentrations in the effluent of the Trains 1 and 2 bioreactors.

It is apparent, that the ammonium concentrations in Trains 1 and 2 diverge in the period between 18 and 23 August 2019. The ammonium in Train 1 increases, which cannot be explained from the ammonium influent load, whereas the ammonium in Train 2 follows the influent load. Ammonium setpoints were identical for both trains.

This event is ideally suited to analyse the QR-MLP models and check whether they are able to detect this anomaly. Also, it enables the combination of outcomes from different models. The higher ammonium concentration could either be caused by a lower aeration amount compared to the influent load or caused by instrument drift that can lead to changes in nitrate levels.

The trained model which predicts the aeration amount based on influent load was evaluated for both trains. For Train 1, the 5-min resolution model shows various extreme outliers (i.e. values that fell outside the predicted 99.7% confidence interval (CI); see Figure 6). However, the figure also shows that the observed values are on average higher than the predicted values.

To quantify this observation, the 5-min results were resampled to daily (24 h) averages (see Figure 7). Then, outliers where daily averages fell outside the 68.3% CI were quantified. This shows that the slow drift becomes visible on 13 August and that it exceeds the CI on 19 August. Further optimization of the resampling time could offer the best result by balancing between higher resolution of single events and early detection of drifts.

The aeration controller uses the online measured ammonium concentration of the effluent of the bioreactor as input. A higher ammonium concentration results in a higher amount of aeration. The degree of the increase depends on the setting of the proportional gain of the aeration controller.

To uncover whether the increased amount of aeration was due to increase in measured ammonium, a second model was set up to predict the ammonium concentration based on the influent load and concentrations within the other basins. The same 5-min and daily-average approach was used to visualize the modelling results for Train 1 (Figures 8 and 9, respectively), which showed and corroborated the increase in ammonium within the same period.

For this model, it is apparent that the measured ammonium concentration was higher than predicted based on the ammonium influent load and concentrations of the other basins during the same event. Also, the resampling again resulted in an earlier (13 August) and more consistent anomaly detection. This model points out that it is plausible that the sensor value of the ammonium sensor installed in the effluent of Train 1 bioreactor has drifted.

However, the measured ammonium concentration of this model trends back to predicted values earlier than the previous model for aeration anomalies. After 22 August, the ammonium concentration is within the predicted range as seen in Figures 8 and 9, whereas the aeration is still outside the predicted range as seen in Figures 6 and 7. This can be explained by the mechanism of the aeration controller. After some days, the machine learning algorithm of the aeration controller picked up that for the same influent load, higher aeration is needed. It also must retrain some days, once the ammonium sensor drift is resolved. This also emphasized the importance of detecting instrument anomalies early.

The increased aeration that resulted from abnormally high measures of ammonium should also result in increased nitrate concentrations, that is relative to the influent load. It can be expected that with increased aeration, more nitrate is produced through nitrification, while the amount of denitrification gets suppressed due to less anoxic capacity. Both changes will increase the nitrate concentration in the effluent of Train 1 bioreactor.

To investigate this, a model was setup to predict nitrate concentration based on influent load and ammonium concentrations of all basins. Displaying these results in 5-min and daily-average resolution for Train 1 (Figures 10 and 11 respectively) exhibited a similar trend of increasing nitrate levels.

This model produced similar results as the aeration model that is based on influent load. Nitrate is higher than expected based on the influent load, thus resulting in a higher amount of aeration supply. After 22 August, the ammonium concentration is within its range, while nitrate concentration is still high, largely because there is a time lag inherent from the machine learning of the aeration controller based on the weight factors.

These three quantile regression models illustrate that anomaly detection during early stage is possible. Based on the outcomes of these different models, it is also possible to determine in which direction to search for the cause of anomaly so that it can be resolved early. This delivers direct value to the process operator. Therefore, these three models are currently being implemented and integrated in an online advanced analytics environment that is hosted in the cloud for real-time operational support.

Resampling resolution and quantiles are important optimisation parameters for further research. The 4-h resolution as an alternative for the 5-min and daily resolution is currently being researched upon. Also, the criteria to detect outliers, where measured values exceed CI thresholds of the model repeatedly and consecutively, are being investigated to improve the robustness of the analytics module.

For other geographical locations that display strong seasonal patterns, the currently developed models must be incorporated with additional data parameters, such as process temperature and sludge solids content. Also, several years of data needs to be collected and utilized for model training to achieve optimal performance.

Operational results with predictive control show that the load prediction has an accuracy (R²) of ≈88% and that a reduction of aeration amount up to ≈15% can be achieved at the IVP compared to conventional control. It is proven that this load prediction-based control leads to stable operation and effluent quality with autopilot functions at IVP. Additionally, it is demonstrated that it functions as an autopilot system with 48 hours forecast horizon. Preliminary results with quantile regression modelling illustrate the ability of automated early detection of process and instrument anomalies. These can be used as an advanced process control and decision support system for operations.

All relevant data are included in the paper or its Supplementary Information.