Multi-parametric modeling of water treatment plant using AI-based non-linear ensemble

Water is essential to sustain life, and an affordable and adequate supply of water must be available (World Health Organization (WHO) and United Nation Children Fund (UNICEF) 2012). Municipal or drinking water treatment plants (MWTP) are operated to remove bacteria, solids, micro-organisms, and other contaminants from untreated water through various approaches such as coagulation, filtration, etc. (Ogwueleka & Ogwueleka 2009). A satisfactory water treatment plant (WTP) is vital in order to avoid the discharge of pollutants and meet the required standards by law. The multi-parametric combination of physical, chemical, and biological characteristics are often the major factors affecting the operation and control of WTPs (Nourani et al. 2018).

Improper control for standard limits of different parameters in WTPs may lead to serious environmental and human health problems. As such, the proper control may be achieved by introducing a robust tool for modeling the WTP performance based on the importance of various key parameters. However, the WTP process is complex in nature due to the dilute mixture of numerous compositions, quality, and characteristic of the system which result in the difficulty of modeling WTP parameters (Belia et al. 2009). In addition, the biochemical and physical nature of WTPs exhibit non-linear phenomena which are too complicated to simulate by simple linear principles or mathematical models (Gaya et al. 2017). In the last decades, different linear models have been widely presented for managing the overall performance of WTPs, but most of them have limitations and are incapable of meeting the standards of modeling non-linear systems (Falah Nezhad et al. 2016). On the other hand, non-linear artificial intelligence (AI) models such as artificial neural network (ANN), adaptive neurofuzzy inference system (ANFIS), and support vector machine (SVM) have shown merit in modeling non-linear systems of WTPs (Mashaly & Alazba 2017; Rahimi et al. 2019).

For instance, Al-Baidhani & Alameedee (2017) developed an ANN model to predict the effluent pH and turbidity using various measured input parameters such as pH, temperature, and dose turbidity. The results proved the suitability of ANN in modeling the WTP parameters. Wu & Lo (2008) used the ANN and ANFIS models to compute the real-time coagulant dosage in WTPs using the measured turbidity, pH, and color. The outcomes demonstrated that the ANFIS model was capable to accurately predict the coagulant dosage with regards to the ANN model. Zhang & Stanley (1999) employed the ANN model with process control for predicting the treated turbidity. Maleki et al. (2018) predicted the influent parameters in WTP using auto regressive integrated moving average (ARIMA) and neural network auto-regression (NNAR) models, and despite an acceptable performance of ARIMA model, the results observed better prediction performance for NNAR with regard to ARIMA. Gaya et al. (2017) demonstrated the application of ANN and Hammerstein-Wiener (HW) models for forecasting the influent turbidity in WTPs using different input parameters. The simulated results indicated that ANN could outperform the HW model and may serve as an acceptable tool for modeling the turbidity of WTPs.

According to the mentioned literature review, it is obvious that several studies using AI models have been conducted and show promising performance, each model for a specific case. Despite the fact there is no exceptional model that justifies superiority to others, applying the knowledge of ensemble techniques could lead to more promising outcomes. The ensemble techniques have been used in various fields of engineering (Sharghi et al. 2018). Nourani et al. (2018) demonstrated a combination of different AI-based single models in an ensemble modeling parameter to predict the performance of a wastewater treatment plant. The obtained results showed that ensemble techniques outperform the AI-based models. Kim & Seo (2015) developed, respectively, ensemble methods using data mining techniques to simulate biological oxygen demand (BOD) and various water quality (WQ) parameters. However, since the pronouncement of the non-linear ensemble (NLE) approach, to the best of the authors’ knowledge, there is no published research indicating the application of different AI-based models integrated with several non-linear ensemble techniques (NETs) in WTPs. Hence, the present work is focused on the application of four types of AI-based models (i.e., general regression neural network (GRNN), HW, non-linear autoregressive with exogenous (NARX) neural network, least square support vector machine (LSSVM) model for multi-parametric modeling in Tamburawa WTP, Kano, Nigeria. For this purpose, various combinations of the input parameters based on their significance are imposed into the models. In order to improve the overall performance of the prediction, NLE modeling is also employed using the outputs of the single models. In this way, three different types of black-box models, GRNN, LSSVM, HW, and NARX ensemble kernels (GRNN-E, LSSVM-E, HW-E, and NARX-E) are used and the results are compared together. Other feasible alternative single models may also be used, such as genetic programming, ARIMA models (Olyaie et al. 2015; Nabavi-Pelesaraei et al. 2017) and for an ensemble, wavelet-ANN (Shamshirband et al. 2019) and non-linear neural network ensemble (Elkiran et al. 2019) were also employed.

Kano is the most populous state in Nigeria, in which drainage river water is conventionally treated in a plant exceeding the minimum standard WQ of WHO. The Tamburawa water treatment plant (TWTP) has a capacity of producing 150 ML potable water per day to cover communities in Kano city and the surroundings. The raw water from the source is pumped via pump station and enters a preliminary treatment unit where grits and some of the suspended solids are removed to avoid pump wear and pipe deterioration. The water is then clarified as shown in Figure 1(a), the clarifier retains and settles the water to remove settleable organic and floatable solids by gravity. Figure 1(b) shows the schematic flow chart of the operational process. The operational process comprises the stages of rapid mix, coagulation/flocculation, sedimentation, filtration, disinfection, and final treated water, which can be distributed to different sources such as domestic, commercial, and institutional (Gaya et al. 2017).

The obtained weekly data set from the TWTP for the year 2015 are measured raw and treated pH (pH_R and pH_T), turbidity (Turb_R and Turb_T) (μs/cm), conductivity (Cond_R and Cond_T) (mS/cm), total dissolved solids (TDS_R and TDS_T) (mg/L), suspended solids (SS_R and SS_T) (mg/L), hardness (Hardness_R and Hardness_T) (mg/L), chloride (Cl_R and Cl_T) (mg/L), and iron (Fe_R and Fe_T) (mg/L). Table 1 shows the descriptive statistical analysis used for this study of the data. The data set was divided into two parts, 75% for the calibration and the remaining 25% were used in the verification phase. As a very common and classic method of data splitting, we used simple random selection of train and test data samples. However, it is highly recommended that in order to have homogeneous data distribution in both training and test data sets, after splitting the data set, the statistical characteristics of both subsets should be checked to have almost the same statistics (mean, standard deviation, etc.) in both subsets to guarantee the generalization of the model. In order to see that the data splitting strategy does not affect the overall performance of the modeling, via the cross-validation method, such random data splitting is repeated several times to see whether there any large differences among the obtained performance metrics in different runs or not; if not, the model generalization is good and the quantity and quality of the data are appropriate.

Selection of the dominant inputs is one of the essential parts in AI-based modeling. As such, a suitable combination of inputs is determined (see Table 2). The holdout validation method was applied and considered as another version of k-fold cross-validation. In the validation process, different types of validation approach can be applied including cross-validation, which is called k-fold cross-validation, others are holdout, leave one out, and so on. The holdout method is also considered as the simple version of k-fold where the data are usually portioned randomly into two sets known as training and testing phase (Nourani et al. 2018).

The crux of selecting individual inputs, for example, hardness, turbidity, pH, and suspended solids parameters to simulate the performance of WTP is that these variables are usually utilized and considered when determining the performance of the treated water from the plant. The hardness is a common quality of water and contains dissolved compounds of calcium and magnesium and, sometimes, other divalent and trivalent metallic elements such as the concentration of hydrogen ions that indicates the acidic and basic level of the solution. It is considered to be of great practical importance as it influences most of the chemical and biochemical reactions (Elkiran et al. 2018). Measuring suspended solids (SS) in water is used for control of various treatment processes and for the examination of wastewater quality. SS is considered to be one of the major pollutants that contributes to the deterioration of WQ, contributing to higher costs for water treatment, and the general esthetics of the water. Turbidity mostly provides cover and food for pathogens. If not effectively removed, turbidity can cause an outburst of waterborne diseases (Gaya et al. 2017).

The stability analysis is an important issue in both data-driven algorithms and numerical solution of initial-value problems. It has been studied extensively in the past for a variety of engineering problems, especially for input selection approach. The motivation behind exploring the stability is to provide evidence that the selected features are relatively robust to slight changes in the data (Křížek et al. 2007). However, different stability selection approaches are applied in an effort to reduce the noise and dimensionality of such data sets. Kalousis et al. (2007) measured the stability in terms of Pearson and Spearman correlation. Nourani & Fard (2012) introduced the stability features selection approach using sensitivity analysis. Similarly, Křížek et al. (2007), Nourani et al. (2019), Nagana et al. (2019), and Meinshausen & Bühlmann (2010) reported different input stability selection approaches. In this paper, both the Pearson and Spearman correlation and non-linear stability of input variables selection approach were assessed prior to model development.

It should be noted that the quality and quantity of the intake water can be drastically changed through different periods of the year due to climate/natural or anthropogenic issues. However, as shown by Equation (2), in this study, we tried to learn the pattern between the input and output parameters using available data (only one-year data), so that having the inputs, the model can estimate the output parameters (WQ parameters in outlet). Therefore, when a severe change occurred in the intake water parameters, via the trained (learned) models, proportionally the output will reflect such severe changes.

It should be mentioned that if we have a very long period of observed data set to use for training the models, the efficiency of such developed data-driven methods will be more robust, especially when facing such severe conditions.

For the purpose of this research, the employed data-driven intelligence, GRNN, HW, NARX, and LSSVM models and a brief description of the mathematical concept for each of these models as well as the related citations are provided in the Appendix, Figures A1, A2, and A3, in more detail.

According to the ‘no-free lunch’ theorem, it is a plain and well-known fact that no single model proves the best for all kinds of data set. The properties of data such as normality, size, linearity, etc. affect the performance accuracy of any model. In addition, several studies have already shown that even for the same type of data set, the performance results may vary for different models (Nourani et al. 2019). Hence, employing multiple models and combining them has been found to be effective for a variety of problems. The ensemble technique has been used with promising success in several fields including hydro-environmental sciences, data mining and statistics as an approach to improve the prediction skill. The main goal for this technique follows the concept of enhancing the performance of the single model, by combining the results of the various individual models. As such, the process of combining various single models will be an ensemble modeling parameter (Nourani et al. 2018). In this study, four different NET were used, GRNN-E, HW-E, NARX-E, and LSSVM-E, to compare and improve the performance of single models.

As mentioned above, four black-box models, GRNN-E, HW-E, NARX-E, and LSSVM, are employed to perform the NLE modeling. In NLE techniques, the outputs of the individual models are built to form the input layer of the considered NLE, each of which is assigned to one neuron in the input layer. Sharghi et al. (2018) and Nourani et al. (2018) have used non-linear ANN ensemble and reported superiority of such technique over linear ensemble techniques. For the modeling procedure by GRNN-E, LSSVM-E, HW-E, and NARX-E, first the individual models are selected and the outputs of the individual models are used for the ensembling. Figure 2 shows a flowchart of the techniques employed for the ensemble modeling.

The main context of data-driven and predictive intelligence models for system identification (i.e., GRNN, LSSVM, HW, and NARX) are evaluated numerically via various acceptable performance accuracy. Afterwards, NLE approaches (i.e., GRNN-E, LSSVM-E, HW-E, and NARX-E) are used to enhance the accuracy of the individual models. The comparison of the individual models is presented in Table 3, according to Equations (3)–(6). As defined by Equation (2), three different models and input combinations were examined based on the number and types of input for the employed models in order to predict the treated hardness (Hardness_T) (mg/L), turbidity (Turb_T), pH (pH_T) (μs/cm), and suspended solids (SS_T) (mg/L).

It is essential to determine the appropriate range of the smoothing factor in any GRNN, due to its significant effect on the simulation ability of the model. The smoothing factor needs to be an average value because extremely large or small values will affect the regression result, so the smoothing factor was considered within the range of 0.01–1 (Ji et al. 2017). The LSSVM model with non-linear radial basis function (RBF) kernel function was created for different conditions. For this purpose, different γ and σ values, through a grid search, were tried to get the best modeling result (Nourani et al. 2017). HW and NARX were built using the system identification toolbox of MATLAB based on the configuration of piecewise linear function (range 10–20) containing input and output non-linearity predictors in the case of the HW model. While for the NARX model, a specify delay and number of terms in standard regressors were considered as 1 and 4–8, respectively.

The direct comparison of results for all of the models shows that the HW model outperformed the other models in the prediction of hardness (see Table 3). The assessment of the results also indicates that for the HW model, the best model was acquired using the third input combination (M3). This can be proved using the performance indicators as the basis of identifying an optimal model for HW ((DC = 0.8272), (DC = 0.8316), and (DC = 0.9181) for M1, M2, and M3, respectively). According to the obtained DCs, addition of hardness, Cl, and SS increased the performance of the prediction as indicated in the third combination. This finding corresponds with that identified by Dan'azumi & Bichi (2010).

The M1, M2, and M3 for the HW model produced RMSE values of 0.1208, 0.1269, and 0.0245, respectively, and MAE of 0.0276, 0.3639, and 0.5639. Another important indicator is MAPE, which provides a better way to determine the predicting model (Gaya et al. 2017). The smaller the MAPE value, the more accurate the estimation model. Table 3 shows that the HW model has the smallest MAPE values with regards to LSSVM, and NARX models with the exception of GRNN. It is noteworthy that several evaluation metrics indicate the merits of the HW over the other models. To continue with additional examination of the estimating models, a series of scatter plots for the three input combinations for the HW model are shown in Figure 3(a). From the plots it can be seen that M3 was able to obtain the best agreement between the observed and computed hardness values. Assessment based on the time series estimated hardness is essential to provide a deeper and more comprehensive analysis, as Figure 4(a) indicates the best computed hardness values for the four models for the verification phase. A visual examination of the time series shows that the computed values of HW were closer to the observed hardness than GRNN, LSSVM, and NARX models.

Table 3 also depicts the obtained results for pH computed by GRNN, LSSVM, HW, and NARX using three different input combinations (M1, M2, and M3) and the best model emerged as NARX using the third input combination (M3). As shown in Table 3, M3 has the lowest RMSE and MAE and highest DC values in the verification phase with merit and suitable accuracy for two other input combinations. Considering the results of the four models in predicting pH, HW was the second-best model followed by GRNN and lastly LSSVM. Further examination of the models shows that when all the variables are used as inputs in pH modeling, the accuracy of the model is higher. Figures 3(b) and 4(b) show the scatter and time series plots for the computed pH, respectively. Turbidity (Turb) as the measure of WQ is also simulated and the obtained results are presented in Table 3. From Table 3, the HW model outperformed all models with reasonable accuracy in both calibration and verification steps. According to the results, it can be justified that M2 with the combination of pH, Turb, Cond, TDS, hardness, and Cl demonstrated the best performance accuracy and therefore proved to be a reliable model for prediction of turbidity in TWTP. A survey of Turb results indicates that HW-M2 increases the prediction accuracy up to 20% and 5% with regards to HW-M1 and HW-M3, respectively. Comparison of results with respect to RMSE and MAE values indicates that the quantitative prediction of Turb can be arranged in the following order: HW > NARX > LSSVM > GRNN. Figures 3(c) and 4(c) show the scatter and time series plots for the Turb parameter in the verification phase, respectively. SS is considered to be one of the major pollutants that contributes to the deterioration of WQ, as such the obtained result for this variable is also presented in Table 3. It is noticed that the HW model is more satisfactory for predicting the SS in TWTP, supported by the values of DC, RMSE, MAE, and MAPE. Among the input combinations, M2 was found to be the best model and the simulated results showed a good level of satisfaction both in calibration and verification steps. Figures 3(d) and 4(d) show the scatter and time series plots for the SS parameter in the verification phase, respectively. Further investigation of SS results shows that HW-M2 increases the prediction accuracy up to 24% and 19% with regards to HW-M1 and HW-M3, respectively, according to computed DC. In addition, comparison of results with respect to RMSE and MAE values indicates that the quantitative prediction of SS can be arranged in the following order: HW > NARX > GRNN > LSSVM. The assessment of the different modeling paradigms (GRNN, HW, NARX, and LSSVM) with different input combinations (i.e., M1, M2, and M3) illustrates that the accurate predictions depend on the used model for a given input data set.

The comparison of multi-parametric simulation in this study proved that HW could serve as the best model for the simulation of hardness, Turb, and SS while the NARX model demonstrated high capability in predicting pH variable. It is apparent that system identification models (HW and NARX) verified the improvement of prediction capability better than GRNN and LSSVM models. Thus, the HW model clearly indicates the extent of effluent removal efficiency and plant performance in TWTP. As mentioned above and justified by Figure 4, there is no one exceptional model that delivers the best estimation performance. The best sets of model inputs were not the same for each of the explored predictive modeling techniques, demonstrating that the respective model types respond differently to different input variable sets and the data patterns/characteristics in the historical input data. Therefore, combining the outputs of these models in the form of an ensemble context could increase the generalization capability of the modeling by properly capturing the target and hence improving the prediction performance. As such, four ensemble techniques (GRNN-E, LSSVM-E, HW-E, and NARX-E) were employed to improve the predictive performance by combining the outputs of the single models.

As reported by Yaseen et al. (2016), no single model works best for all data sets, and different data characteristics such as data size, normality, linearity, and correlation all affect model performance considerably. Nourani et al. (2019) also reported that even for the same data set, different models may excel in different aspects. Hence, gathering the multiple models in an ensemble approach has been found to be effective for a variety of problems. Different NLEs such as GRNN-E, LSSVM-E, HW-E, and NARX-E performed as shown in Figure 5 and the NLE results are presented in Table 4.

Table 4 demonstrates that among the NLE modeling, GRNN-E outperformed the other models with considerable accuracy, due to its robustness in dealing with complex interactions which perform prediction without requiring large samples. The accuracy of GRNN-E as a type of ANN can also be attributed to the promising predictive skills of the overall ANN model. Peng & Kumar (2005) and Nourani et al. (2019) reported that in a few cases single models can be superior to the ensemble model in some instances, which was also observed by comparing Tables 3 and 4.

Figure 5 shows the radar chart showing the different percentage variation of MAPE. From Figure 5 it is clear that the MAPE value is less than 10% for all the best models, which demonstrated the high accuracy of the model. Similarly, in terms of DC, GRNN-E increased the accuracy of the modeling up to 30% for hardness and Turb, 34% for pH, and 37% for SS with regards to the single models.

In Figure 6, a Taylor diagram was used as the most widely recommended diagram for accuracy comparison (Kim et al. 2018; Zhu et al. 2019). The performances of the GRNN-E, HW-E, NARX-E, and LSSVM-E models were compared during verification phase, using the correlation coefficient (R) and the standard deviation (SD). From Figure 6 it can be seen that GRNN-E outperformed other models, because the observed points are closer to the computed points. This can also be proved by considering the high value of R which was attributed to the GRNN-E. Generally, if the SD of the computed values is higher than the SD of observed values, then it will result in overestimation and vice versa.

In this study, GRNN, HW, NARX, and LSSVM for modeling the treated hardness, turbidity, pH, and suspended solids were applied in Tamburawa WTP Kano, Nigeria. Subsequently, the NETs were also employed using the outputs of the single models to enhance the overall accuracy of the prediction. Weekly available data from the treatment plant were used and various statistical performance indicators were used to explore the predictive capability of the overall modeling techniques. The comparison results of multi-parametric simulation in this study showed that HW served as the best model for the simulation of Hardness_T, Turb_T, and SS_T while the NARX model demonstrated high capability in predicting pH_T variable. Yet, the HW and NARX as system identification techniques attained overall best predictive performance among the four modeling approaches. However, among the NLE models, GRNN-E proved of high merit and increased the accuracy of the best single models up to 30% for Hardness_T and Turb_T, 34% for pH_T, and 37% for SS_T with regards to the single models. Finally, such a model can serve as the background and be part of the portfolio of tools used by water engineers in decision-making at TWTP. The results also recommended that the application of other AI such as extreme learning machine (ELM), wavelet and the combination of hybrid models may also be combined in the proposed ensemble approach in order to integrate a set of models so as to create a new model which could yield higher precision than the used single models.

Just like any data-driven method and as the limitation of the study, the efficiency of the proposed model is markedly related to the quality and quantity of the available data, which most of the time is a challenge in developing countries. Therefore, the used data should be checked and when gathering more data in the future, the models should be updated using the new data.

The Supplementary Data for this paper is available online at http://dx.doi.org/10.2166/aqua.2019.078.