In the present study, three different artificial intelligence based non-linear models, i.e. feed forward neural network (FFNN), adaptive neuro fuzzy inference system (ANFIS), support vector machine (SVM) approaches and a classical multi-linear regression (MLR) method were applied for predicting the performance of Nicosia wastewater treatment plant (NWWTP), in terms of effluent biological oxygen demand (BODeff), chemical oxygen demand (CODeff) and total nitrogen (TNeff). The daily data were used to develop single and ensemble models to improve the prediction ability of the methods. The obtained results of single models proved that, ANFIS model provides effective outcomes in comparison with single models. In the ensemble modeling, simple averaging ensemble, weighted averaging ensemble and neural network ensemble techniques were proposed subsequently to improve the performance of the single models. The results showed that in prediction of BODeff, the ensemble models of simple averaging ensemble (SAE), weighted averaging ensemble (WAE) and neural network ensemble (NNE), increased the performance efficiency of artificial intelligence (AI) modeling up to 14%, 20% and 24% at verification phase, respectively, and less than or equal to 5% for both CODeff and TNeff in calibration phase. This shows that NNE model is more robust and reliable ensemble method for predicting the NWWTP performance due to its non-linear averaging kernel.
Adaptive neuro fuzzy inference system
Artificial neural network
American society of civil engineer
Influent biological oxygen demand
Effluents biological oxygen demand
Influent chemical oxygen demand
Effluents chemical oxygen demand
Feed forward neural network
Milligram per liter
Multi-layer feed forward
Nicosia wastewater treatment plant
Neural network ensemble (NNE)
Root mean square error
Support vector regression
Support vector machine
Simple averaging ensemble
Influent total nitrogen
Effluent total nitrogen
Total suspended solids
United Nations Educational, Scientific and Cultural Organization
United Nations Development Programme
Weighted averaging ensemble
Wastewater treatment plants
Wastewater treatment plants (WWTPs) consist of multiple equipment's and processes to treat wastewater which generally allows anthropogenic activities and industrial effluents to be disposed without danger to human health or undesirable damage to the natural environment (Gómez et al. 2017). According to the United Nations Educational, Scientific and Cultural Organization (UNESCO) (2015), wastewater treatment and environmental sanitation are of paramount importance for sustainable development and critical for human health ecosystems. A diluted mixture of different wastes from commercial, residential, industrial and other public environments is considered as wastewater or sewage, though its characteristics depend upon the discharge source but wastewater generally contains organic matter, inorganic matter and living organisms (Aydın Temel et al. 2018).
A WWTP is a highly complicated and dynamic system and so its proper operation and control is essential in order to safeguard the public and environmental health issues. The quality of raw and treated effluent has a great impact on the operation and performance of any WWTP (Ahmadi et al. 2017). In addition, the influent flow to a WWTP has a significant influence on the energy consumption and entire treatment process. It is difficult to measure some dominant variables; e.g. biological oxygen demand (BOD) requires 5-days' incubation, and this makes it hard to find and delay the timing process; it is also well known that high BOD concentration requires longer aeration time and supply of more oxygen. Thus, it is important to predict the effluent concentration at future time horizons in order to well manage the plant and control the effluent quality (Kim et al. 2006). Operational control of WWTPs is complex because of the complexity of mechanism in the treatment plant, influent quality and strength of the wastewater. Hence, modeling of such a complex system is quite difficult and most of the available traditional models are based on some rough expectations, linear approximations and assumptions (Gao et al. 2017; Hameed et al. 2017). The conventional processes used for determining the performance of WWTPs are based on the balance equations coupled with the mass flow rate equations for microbial growth and substratum consumption which involve nonlinear interaction, time consuming and complex nature (American Society of Civil Engineer, ASCE Task Committee 2000). This complex behavior of WWTPs is difficult to predict or simply explained by classical linear models and, as such, development of reliable and convenient modeling tools plays an essential role in simulation and monitoring the performance of WWTPs and describing the overall phenomena occurring in the entire system.
Therefore, non-linear artificial intelligence (AI) based models have been applied recently to overcome the problems associated with traditional linear methods, due to their accuracy, flexibility and promising application in different engineering fields (Govindaraju 2000; Nourani 2017). AI based models such as adaptive neural fuzzy inference system (ANFIS), feed forward neural network (FFNN), support vector machine (SVM), are relatively new black box methods used at various aspects of water and environmental engineering as well as assessment of WWTP operations. To assess the WWTP performance there are certain important parameters that have been focused in the most of previous studies conducted using AI based models. These parameters include BOD, chemical oxygen demand (COD), total suspended solids (TSS) and total nitrogen (TN) (Hameed et al. 2017). Guo et al. (2015) used the influents of pH, temperature, COD and SS to predict the concentration of TN effluent from the WWTP in Ulsan, Korea, by employing ANN and SVM models and concluded that AI models could be reliable methods for prediction of the effluent conditions of the WWTPs. Granata et al. (2017) employed SVM and regression tree in order to forecast the fluent concentration of BOD, COD, TSS and TDS in WWTP, the SVM showed a bit better performance than regression tree which is a kind of decision tree. Civelekoglu et al. (2009) applied ANN and ANFIS methods to model the COD removal in biological WWTP, the overall results indicated that ANFIS is a suitable model for prediction of the WWTP performance. Hamed et al. (2004) used the BOD and TSS values recorded at various positions as input parameters and outlet BOD and TSS as target variables to predict the performance of WWTP using ANN model. The results proved the ability of ANN model for predicting WWTP performance.
As the literature review shows, there is no unique model to be superior to others in all cases and the performances of different models may be different according to condition of each WWTP. Therefore, it is verified that the combination of outputs (from different models) through an ensemble method may lead to more accurate results. The idea of such an ensemble model has been already used at different fields of engineering, environmental and water quality modelling (Cloke & Pappenberger 2009; Sharghi et al. 2018). However, since the pronouncement of ensemble methods in engineering, to the best of the authors' knowledge, there is no published study in the technical literature indicating the application of AI based ensemble approach in WWTP modeling. The current study aimed at the following. (i) To develop and compare the potential of some AI based models (FFNN, SVM and ANFIS) and conventional multi-linear model (MLR) for prediction of the Nicosia WWTP performance considering four different combinations of input parameters. Other feasible alternative models may also be used (e.g. genetic programing, ARIMA models, Olyaie et al. 2017), but these models were adopted here due to their outstanding performances in various hydro-environmental studies. (ii) To establish and apply three ensemble techniques using the outputs of aforementioned single models in order to improve the overall efficiency of the prediction performance. In this way, simple linear averaging, weighted linear averaging and non-linear neural ensemble techniques are proposed to combine the outputs of the methods.
MATERIALS AND METHODS
Plant description and used data
The new WWTP of Nicosia (NWWTP) was planned to take care with 270,000 populations with the project horizon year 2025. The implementation of stage has been put in place and considered so as to sidestep general and extensive capacity surcharge of the consumer for unexploited capacity. The volume of stage 1 was conventionally established as 30,000 m3/day and stage 2, on the other hand, will be executed to accomplish the final design volume of 45,000 m3/day. The new plant has been planned in light of membrane bioreactor (MBR) technology. It has high condition of workmanship and state of art technology for a WWT and the new plant has been designed as an MBR plant with advanced biological nutrient removal. It is currently the second biggest WWTP in Europe that uses MBR technology which serves the needs of both Turkish Cypriot and Greek Cypriot (bio-communal). About 10 million m³ of treated water every year can be reused for agricultural purpose. Depending on the crop type and the rotation approach, approximately 500 hectares can be irrigated, so the treated water can significantly reduce the over-extraction of groundwater in the area, and enhancing water resources and water conservation. More than 3,000 tons of dry solids suitable for use as natural fertilizers are also produced every year. The NWWTP has anaerobic sludge digesters which are equipped and capable of producing electricity from biogas. The operation of the plant will therefore be partly powered by renewable energy (10–20% on average), reducing its carbon dioxide (CO2) emissions. Figure 1 shows the map of the study area and schematic of the NWWTP process; this process presents only the treatment process from raw sewage to treated effluents.
For this study, the available daily data were obtained from the new NWWTP (United Nations Development Programme (UNDP) 2014). The measurement of the selected parameters covers all the seasonal variations and consists of various sets of inputs and outputs parameters. The daily measured data set comprising of influent of pHinf, conductivity (Condinf), BODinf, CODinf, total nitrogen (TNinf) as the inputs of models, and three effluent values like: BODeff, CODeff and TNeff as targets over the period of 2015–2016. The available data set was divided into two parts: 75% of the data were employed for the calibration and the remaining data were used for the verification purpose. For the validation purpose there are several methods which can be implemented such as holdout, leave one out, cross validation and so on. The method we employed here is holdout or sometimes called leave-group-out. In the holdout method, the data are randomly assigned to two sets, usually called training and testing, and can be considered as the simplest version of k- fold cross validation (Sargent 2009; Tsioptsias et al. 2016).
It could be observed from Table 1 that the inflow characteristics of some parameters in WWTP are slightly higher than the design values, for example: BODinf, CODinf and TNinf have the design values of 538, 1,202 and 121 mg/L, respectively (UNDP 2014). It is clear that for all kind of data driven methods (e.g. AI methods) if the amount of dispersion (standard deviation) of data is low (this indicates the closeness of the data to the mean) it is expected to get less biased outputs from the models.
In the descriptive statistical analysis, correlation coefficient (R) as the most commonly used techniques was calculated to measure the strength and degree of linear relationship between two variables, which can be served as the preliminary indication of probable linear correlation between a set of variables. However, the weakness of the computed R values depicts that the application of conventional linear techniques to handle such non-linear complex interactions cannot be recommended and there is a great need to introduce more robust non-linear tools. Therefore, instead of some other studies which used linear correlation coefficient between input and output parameters to select the dominant inputs of non-linear, different combinations of input parameters are examined through the used methods (FFNN, ANFIS, SVM and MLR) in this study.
In this study, FFNN, ANFIS, SVM and MLR models were proposed for simulation of the performance of NWWTP. The used data were normalized and partitioned into calibration and verification sets to simulate BODeff, CODeff and TNeff. In addition, ensemble models were proposed using simple average, weighted average and neural average techniques. Different combinations of input parameters for black box AI based models (FFNN, ANFIS, SVM) and linear model (MLR) were examined in the modeling framework. The overall schematic of the proposed methodology is presented in Figure A1 at Appendix A (available with the online version of this paper).
Used data driven methods and efficiency criteria
As mentioned, three AI based methods of FFNN, ANFIS and SVM and one classic MLR approach were employed for single and ensemble modeling in this study. A brief description of mathematical concept for each of these models as well as the related citations are provided in Appendix B for more details (available with the online version of this paper). Furthermore, determination coefficient (DC) and root mean square error (RMSE) criteria. Other performance efficiency of the model can also be used such as bias or mean absolute error (MAE). For a good analysis of any model, the efficiency performance should include at least one goodness-of-fit (e.g. DC) and at least one absolute error measure (e.g. RMSE) (Legates & McCabe 1999).
Ensemble learning techniques
Ensemble learning is a machine learning to combine the process of multiple predictors in order to enhance the final performance (Sharghi et al. 2018). Ensemble techniques were proved to produce more accurate results than a single model. Ensemble techniques have already applied in several fields such as web ranking, classification and clustering, time series and regression modeling (Kazienko et al. 2013). In this study, two linear and one non-linear ensemble techniques were used as follows to improve the performance of the single models.
Technique 1: simple averaging ensemble (SAE)
Technique 2: weighted average ensemble (WAE)
DCi is the performance efficiency of ith single model.
Technique 3: non-linear neural ensemble (NNE)
In the neural ensemble techniques, non-linear averaging is performed by training another neural network. The input layer of the neural ensemble model is fed by the outputs of the considered models, each of which is assigned to one neuron in the input layer. In a neural ensemble model like single FFNN, considering tangent sigmoid as activation functions of hidden and output layers, the network can be trained using BP algorithm and the best structure and epoch number of the ensemble network can be determined through the trial–error procedure. It is clear that, other non-linear kernels (e.g. ANFIS, SVM, etc.) can be alternatively used for such a non-linear ensembling, but FFNN was used in this study since it is the most common AI based method.
RESULTS AND DISCUSSION
Results of single model predictions of BODeff, CODeff and TNeff
It is worth mentioning that, determining the number of hidden neurons, training epoch number and transfer functions are essential aspects in designing the FFNN model. Lavenberg Marquardt was chosen and used in this study as the BP training algorithm due to its fast learning and high-performance accuracy. Four different FFNNs for each output were trained considering different input combinations (as presented in Table 2) and the best architecture was determined through a trial–error process. FFNN (5-1) which include five inputs and one output neurons found to be the best for all three simulated variables as shown in Table 3. It is noticed that obtained results of FFNN are satisfactory for predicting the NWWTP performance, which are supported by the values of DC and RMSE displayed in Table 3, except for the simulation of BODeff. For the determination of appropriate ANFIS model, different types of membership functions were examined by trial–error process. ANFIS models were trained for each model using hybrid algorithms (Table 3). According to Table 4, ANFIS (5-1) was found to be the best model and the simulated results showed a good level of satisfaction. In addition, Table 3 confirms that increase in performance is considerable due to the increase in input parameters. ANFIS performance was increased in verification phase up to 10% for BODeff, with no or negligible increase for CODeff and TNeff (Table 3). This shows that for predicting the models of NWWTP, ANFIS is more recommended for BODeff modeling while both FFNN and ANFIS could be applied for the simulation of CODeff and TNeff. For SVM modeling, the optimum model was obtained by adjusting two parameters, i.e. squared kernel and regularization constant parameter until the desired output set was achieved. Table 3 shows the simulated results of BODeff, CODeff and TNeff by SVM model. Table 3 proves that, for predicting the NWWTP performance, the use of SVM model is recommended for simulation of CODeff and TNeff, with regards to BODeff which the result was not so reliable. In case of BODeff, SVM (4-1) was found to be the best while for CODeff and TNeff, SVM (5-1) could lead to better outcomes. MLR model was also applied as a conventional method to predict the performance of NWWTP. Four different models were used to simulate the BODeff, CODeff and TNeff (Table 3). The best model was determined according to the DC and RMSE criteria. The efficiency criteria of the BODeff modeling ranged between 0.5941–0.6212, 0.0077–0.0193 for DC and RMSE, respectively. However, for CODeff those ranged between 0.7013–0.8669, 0.0012–0.0065 and for TNeff ranged between 0.7055–0.8392, 0.0006–0.0034 for DC and RMSE, respectively. The best model was found to be MLR (5-1) for BODeff and CODeff while MLR (4-1) for TNeff. Presented results indicate the improved performance of SVM in verification phase up to 10%, 4% and 11% for BODeff, CODeff and TNeff modeling, respectively, over MLR model. It is apparent that SVM model slightly demonstrated the enhancement of prediction capability than MLR model, and the MLR also clearly indicates the extent of effluent removal efficiency and plant performance in NWWTP.
Table 3 also justifies that, when all the variables are fed into the models to simulate the outputs, the prediction turns to improve in terms of performance criteria. It can be observed from Table 3 that ANFIS is the best model among all applied models due to capability of the fuzzy concept to handle uncertainty in the process. Meanwhile, CODeff and TNeff provide reliable accuracy while BODeff is found to be the worst in all models. The fair BOD indicates the presence of organic matter and bacteria, the main focus of NWWTP is to reduce the BOD, COD and TN in the effluent before discharging to natural waters. However, for the simulation of BODeff in verification phase, ANFIS performance showed increase up to 10%, 15% 16%, for FFNN, SVM and MLR, respectively. For all models, the verification and calibration results were used to assess the accuracy and efficiency of the algorithm in terms of DC and RMSE. Nevertheless, in hierarchical comparison to other models, FFNN ranked second best, followed by SVM and finally MLR model. It is also clear that the ratio between the various components in wastewater has a significant influence on the selection and functioning of NWWTP processes. In NWWTP, COD/BOD ratio is high which indicates that substantial part of organic matter will be difficult to degrade biologically, leading to fair result of BODeff. The result of BODeff modeling also proved that the pollutions load is mostly contributed from the households and institutions with low significant contribution from the industrial catchment. Also, the variations and compositions of NWWTP are contributed by the amount of organic waste produced by domestic, institutional and commercial areas. Figure 4 shows the time series plot of the fitted models for three simulated outputs via ANN, SVM, ANFIS and MLR.
As seen in Figure 2, there are some missing data, mostly for COD. Considering the presented input layers in Table 2, different alternatives may be used as input parameters of models. For example, in the case of several missing data of COD, the input combination which does not include COD may be used for modeling. Of course, the performance will be lower than when we have COD data in input layer, but the data availability dictate this to the modeler.
Also, some appropriate methods may be used to fill in the missing data first and then to use the synthetic data in modeling, but since in this case the synthetic data may include larger amount of error, then the developed AI using such data may not lead to appropriate results (Nourani et al. 2012a, 2012b).
Figure 4 depicts plot of predicted TNeff by different methods highlighting two sample points (a) and (b). From this figure, it is clear that for sample point (a) ANFIS model could lead to a bit better performance than FFNN and SVM models. On the other hand, for sample point (b), FFNN and SVM models are better than ANFIS model. Therefore, although overall performance of one of the models may be better for whole time series, at different spans of time series, the performance of the models may be different. As such, at different conditions, different methods may lead to different outcomes and so it is a logical idea to ensemble the outcomes of different methods to get more accurate results for the future predictions.
Results of ensemble predictions for BODeff, CODeff and TNeff
The ensemble of outputs from FFNN, ANFIS, SVM, and MLR were carried out based on proposed SAE, WAE and NNE to improve the overall prediction accuracy of the single models. Table 4 shows the obtained results by SAE, WAE, and NNE techniques. The obtained DC and RMSE values for both calibration and verification phases show improvement in the modeling efficiency with regards to the single models. The performance of ensemble techniques depends on the accuracy of each individual model as each model has its own drawback and merit in the modeling process. The results also proved that, for the prediction of BODeff in NWWTP, ensemble methods could lead superior results with regard to single models (Table 4). This is because the integration of single model's outputs reduces the variance, bias and improves performance of the overall modeling. Despite a reliable result for all the ensemble techniques, NNE found to be more accurate, followed by WAE and lastly SAE. In the verification phase, SAE, WAE and NNE increased the efficiency performance of AI modeling up to 14%, 20% and 24%, respectively, for predictions of BODeff and up to about 5% for modeling both CODeff and TNeff parameters. This proved remarkable improvement in the prediction of BODeff which was found poor using single models. Ensemble methods aimed primarily not only to integrate a set of models but also to decrease the weaknesses of each single model and come up with the enhanced and composite model, which is feasible, and more reliable with high accuracy than single models. According to Table 4, the results of WAE slightly outperformed SAE due to the fact that weight is assigned to each parameter based on relative importance which is not in the case of simple averaging. The performance of NNE is better than two ensemble techniques in both calibration and verification steps, because of the robustness of NNE in handling non-linear interactions, and able to back propagate the produced error during calibration phase until the desire result is achieved. Figure 5 shows the results obtained by NNE as scatter plot and time series plots for BODeff, CODeff and TNeff versus observed values.
In non-linear method (e.g. proposed neural ensemble technique), the errors (or noise of data) of the single predictions may be non-linearly magnified. Therefore, for only a few (two or three) extreme values which include higher values and consequently may have a higher degree of errors, the non-linear method may not provide good agreement with the measured values. But this can be true only for two or three samples (as seen in Figure 5) and for most of other samples which include lower error, the errors are not significantly magnified and so, the overall performance (measured by RMSE and DC) will be okay. Actually to handle such problems, linear method is also embedded in the modeling (as already proposed in this study) but sometimes it is not enough and, in this case, a data preprocessing approach such as de-noising method (e.g. see Nourani et al. 2014) can be more helpful.
In this paper, the performance of NWWTP was modeled by different AI models of FFNN, ANFIS, SVM and a conventional MLR. Simple averaging, weighted and neural network ensemble techniques were subsequently employed to enhance the prediction performance of single models. For this purpose, daily data from NWWTP were obtained and DC and RMSE were used in order to determine the prediction performance. The comparison of single models showed that ANFIS was better than other single models in both calibration and verification phases. According to the results, SVM was found to be more reliable than the MLR model. Also, in verification step of BODeff modeling, the models showed more accurate performance up to 10%, 15% and 16% with regards to FFNN, SVM and MLR models, respectively. In the verification phase of ensemble predictions, SAE, WAE and NNE increased the efficiency of AI modeling up to 14%, 20% and 24%, respectively, in the BODeff and about 5% for both CODeff and TNeff predictions. Among ensemble techniques, NNE was found to be more robust and efficient method of combination and could improve the performance of AI modeling up to 24%. The benefit of the NNE was due to the fact that FFNN model has the ability of handling non-linear behavior in the system. According to the results obtained so far, firstly, single models should not be considered as reliable models for the simulation of BODeff in NWWTP, as it proved fair results for all AI models. Secondly, all AI and classical models employed in this study were found to be satisfactory and, therefore, recommended for the simulation of CODeff and TNeff. Thirdly, the NWWTP performance indicated high quality of treated effluent which can be used for irrigation and other re-use purposes and the ensemble results provide more reliable and promising results than the single models. Finally, the study may serve as the background for researchers carrying out further studies in NWWTP. The outcomes also suggested that for the application of these models in the real world, the uncertainty involved in the process could be addressed. As such, the application of other AI tools may also be combined in the proposed ensemble approach in order to integrate a set of models so as to come up with a new model which could produce higher accuracy and more reliable estimates than the single models.
In this study, the performance of WWTP was investigated according COD, BOD and TN parameters at outlet due to the availability of the data, but the method may be applied similarly (of course with the new trained AI structures) for other important parameters of wastewater such as ammonia.