Evolutionary modelling of municipal water demand with multiple feature selection techniques

Water demand forecasting is of crucial importance in water resource planning and management as it is a prerequisite for optimal allocation of available water resources (Qu et al. 2010). City managers and water utilities often rely on water demand forecasts to guide their decision-making on infrastructure investments as well as the scheduling and operation of water distribution systems. For example, long-term forecasts are imperative in providing new water supplies and upgrading the capacity of existing water treatment plants while short-term forecasts guide day-to-day operation of treatment plants and reservoirs to meet daily demands. Accurate water demand forecasts are therefore required for both short- and long-term infrastructure planning, operation and coordination. Moreover, the importance of water demand forecasting in realizing the sustainable development goals (SDGs) has been stressed by the United Nations (UN). The UN in its 2015 water development report called for improvement of water demand models as competing demands may lead to increasingly difficult allocation decisions and restrict the growth of sectors critical to sustainable development (UNESCO 2015). This implies that, amidst growing demands for fresh water across the globe, sustainable development can only be achieved if competing sources of demand are well defined to enable restoration of the balance between demand and supply. Water demand forecasting therefore provides useful information for promoting a more economical use of water resources and ensuring the sustainability of water distribution systems in the short, medium and long terms.

Accurate water demand prediction is reliant on the explanatory variables adopted in model development. Research has shown that model accuracy is a function of the impacts of each explanatory variable (Firat et al. 2009; Babel & Shinde 2011; Toth et al. 2018). However, many city managers, consultants and water utilities still assume that water demand will evolve simply as a function of per-capita demand and a prognosis of population, although the predictive power of such approaches is deficient under changing conditions (Toth et al. 2018). Babel & Shinde (2011) argue on the need to develop improved and city specific water demand models as water demand is influenced not only by population but also by various weather and socioeconomic variables as well as government policies and strategy-related factors, which are often location-based. Therefore, the need to carefully define, evaluate, understand and model the explanatory variables that directly and indirectly influence water demand is now acknowledged as crucial in obtaining accurate demand forecast. Table 1 presents a list of some relevant studies that have considered weather and socioeconomic variables in water demand forecasting.

Over the past few decades, many techniques have been used in forecasting water demand. These techniques mainly include traditional forecasting techniques such as multivariate regression and time series analysis (Babel et al. 2007), system dynamics modelling (Qi & Chang 2011), and more recently, advanced computational intelligence techniques like expert systems or agent-based models (Ali et al. 2017) and artificial neural networks (ANNs; Bennett et al. 2013). The application of ANN in water demand forecasting is becoming increasingly popular due to its superiority over traditional techniques and its ability to account for nonlinear patterns observed in real problems (Babel & Shinde 2011; Kofinas et al. 2014). ANN is capable of learning and analysing data attributes and, thereafter, implementing nonlinear approximation function without any initial assumption on the physics of the system being modelled or its data distributions (Faizollahzadeh Ardabili et al. 2018). As a result, ANN is now being adopted as an alternative to the traditional methods which are limited due to their linear pre-assumption of the form of the model (Kofinas et al. 2014). A review of the capability, implementation and application of ANN in water resources modelling including water demand forecasting is available in Ghalehkhondabi et al. (2017) and Oyebode & Stretch (2018a).

Water demand forecasting using ANN is characterized by some complexities. According to Kofinas et al. (2014), these complexities can be summarized as: (i) inability to adequately extrapolate outside the range of primary (training) data; (ii) diminishing forecast accuracy when lagged values of target variable are used as input; and (iii) disregarding the impacts of other explanatory variables affecting water demand due to the high correlation between future water demand and its historical values. In a comprehensive review of techniques used in forecasting water demand, Oyebode & Stretch (2018b) noted the non-inclusion of weather-based variables as inputs in most of the studies reviewed. It was further argued that, due to the non-inclusion of weather-based variables, most studies in the literature lack a climate variability perspective to water demand modelling. This jeopardizes the opportunity to put in place effective early warning systems and to implement adaptive interventions to deal with variations in water availability and the occurrence of extreme climate-linked events. The inclusion of climate-based parameters is likely to enhance the outcome of existing water demand forecasting models.

This study aims to suggest possible ways of addressing the limitations of ANN in water demand forecasting. First, it allows for the enhancement of demand forecasting models proposed in earlier works, by integrating new explanatory variables related to climate and socioeconomic variations. The study explores the capabilities of five feature selection techniques in providing the optimal set of explanatory variables required for accurate prediction of water demand. Furthermore, it investigates the ability of an evolutionary-based technique – differential evolution (DE) - in evolving an ANN model with optimal model complexity and accuracy. Research has shown that despite the prominence and successful applications of evolutionary algorithms in water resources (Maier et al. 2014), DE is yet to be fully explored in water demand forecasting (Oyebode & Stretch 2018a, 2018b). In this study, the ability of DE in training a multilayer feed-forward neural network is explored and, by doing so, a water demand forecasting model with optimal complexity and accuracy is developed.

This section presents a brief background on the modelling technique, training algorithm and feature selection techniques applied in this study.

The study investigates the ability of ANN to forecast monthly water demand considering the nonlinear and dynamic nature of input variables based on climate and socioeconomic factors.

In newer generations, each parent individual generates an offspring using a dual-staged approach. The initial stage involves creating a donor vector via mutation. In the second stage, a crossover operation is executed between the donor and parent vectors, resulting in an offspring. A parent and an offspring are subjected to a competition-based selection process (greedy selection) and only the superior proceeds to the succeeding generation.

To develop a model with high degree of accuracy and minimal complexity, it is important to select only a small number of variables with significant predictive features. Research has shown that the inclusion of irrelevant or redundant or noisy variables could increase model complexity, reduce model interpretability, heighten computational demands, and consequently lead to non-convergence (Bowden et al. 2005). Feature selection has been widely reported as a technique that can be beneficial to learning as it seeks to identify and possibly remove all the irrelevant and redundant attributes, thereby reducing the dimensionality of the data and the size of the hypothesis space accordingly (Hall 1999; Oyebode 2014). Feature selection achieves this aim by finding a minimum set of variables, such that the resulting probability distribution of the data classes is, to a great extent, close to the original distribution obtained using all variables (Azhagusundari & Thanamani 2013). Feature selection thus enables learning algorithms to execute faster and more effectively. The techniques utilized in evaluating the worth of features (variables) used in this study are briefly described in turn.

Information gain is thus founded on the premise that the information gained about the target variable after observing an independent variable is equal to the information gained about after observing ⁠. The limitation in using information gain is in its bias towards features with more values even when they are not more informative (Phyu & Oo 2016).

Principal component analysis is a feature selection which seeks to identify linear combinations of unique explanatory variables (referred to as principal components – PCs) capable of summarizing the data, with the aim of retaining maximum information during the process. Principal component analysis operates by transforming a given set of variables orthogonally such that the transformed variables are uncorrelated and independent of each other, especially if the initial variables are normally distributed (Hu et al. 2007).

This research focused on the City of Ekurhuleni, a metropolitan municipality, located in the Gauteng province of South Africa – the most populous province in South Africa with a population of approximately 14.7 million people (Stats-SA 2018). The City of Ekurhuleni was established in the year 2000 from the amalgamation of two existing regional entities, namely Kyalami Metropolitan and the Eastern Gauteng Services Council, thereby agglomerating a set of nine relatively small and fragmented towns (Figure 1). The City of Ekurhuleni currently accounts for about 26% of Gauteng's population and plays a dominant role in the national economy, contributing 8.8% to South Africa's Gross Added Value as of 2016 (IDP 2018). The city also has a Human Development Index (HDI) of 0.704, greater than the National value of about 0.653. However, it is at the epicentre of migration, resulting in increased pressure on limited water resources (IDP 2018). The Gauteng province has already used up its available water resources and is now importing more expensive water, via bulk purchase, from the Lesotho Highlands transfer scheme and feeding it into the Vaal dam (IDP 2018). Table 2 provides current figures relevant to Ekurhuleni Water Infrastructure.

The city management seeks to ensure that Ekurhuleni transitions from being a fragmented city to being a ‘Delivering City’ from 2012 to 2020, a ‘Capable City’ from 2020 to 2030, and lastly a ‘Sustainable City’ from 2030 to 2055 (IDP 2018). To achieve these milestones, a long-term development strategy referred to as the Ekurhuleni Growth and Development Strategy 2055 (GDS 2055) has been developed to systematically analyse Ekurhuleni's history and its development challenges, wherein it therefore outlines the desired growth and development trajectory. One of the strategic objectives and the key focus areas and/interventions is the promotion of urban integration and continued investment in water infrastructure to ensure security of supply. This is critical to attaining the state of a ‘Sustainable City’ and realizing the African Union Agenda 2063 and 2030 UN SDGs which include access to clean water and sanitation, innovation and infrastructure as well as reduced inequality. A reliable water demand forecasting model which considers not only population but also other factors related to the weather and socioeconomic profile of the city is therefore of vital importance. This would assist in the planning and management of the city's water resources, thereby fostering the achievement of its objectives.

One of the initial steps in water demand forecasting is identifying explanatory variables that directly and indirectly influence water demand. The identification of explanatory variables forms the basis upon which final input parameters are selected for model development. Details on key explanatory variables to be considered in water demand forecasting are available in Oyebode & Stretch (2018b). Based on availability, the explanatory variables considered in this study include monthly total rainfall ⁠, monthly average minimum and maximum temperatures (⁠ and ⁠), monthly average relative humidity (RH), monthly average wind speed ⁠, number of household connections ⁠, population ⁠, human development index and water consumption ⁠. Although the weather-based variables usually employed in water demand forecasting studies are temperature and rainfall, considering the semi-arid characteristics of the City of Ekurhuleni (and generally, South Africa), RH and wind speed were included as potential explanatory variables as they could influence outdoor water consumption (Huntra & Keener 2017). Moreover, previous studies conducted in areas with similar characteristics have reported the inclusion of these parameters in forecasting water demand (Firat et al. 2009; Babel & Shinde 2011; Huntra & Keener 2017).

Monthly data records for each variable were obtained from relevant government departments (South African Weather Service, Statistics South Africa (Stats SA), and the City of Ekurhuleni) for the period August 2010 to March 2018. Water consumption data were based on total monthly billed (revenue) water consumption of water users. Weather information was supplied from a representative weather station located in the OR Tambo International Airport. The number of household connections provides an indication of the number of dwelling units served by the authority while population represents the total number of people domiciled in the city. The HDI is a measure of the city's overall achievement in its socioeconomic dimensions including life expectancy, education and income levels. Yearly population and HDI data were transformed into monthly values for use in this study by linear interpolation.

The statistical properties and historical trends of the data collected in this study are presented in Figure 2 and Table 3, respectively. A surge in water consumption and huge variations thereafter can be observed between mid-2015 and March 2018. As clarified by the city's water services planning manager, in mid-2015, the city management implemented one of its strategic objectives which aimed at reducing water loss within the city's water distribution system. The implementation of this strategy entailed the installation of new water meters and repair of faulty water pipes to address high water losses which were mainly due to leaks, theft and metering inaccuracies. As a result, water initially categorized as non-revenue water (i.e. real and apparent water losses) was transformed into revenue water.

To identify feature subsets that can describe the water consumption data of the City of Ekurhuleni as good or better than the primary data set, the five feature selection techniques described above were investigated. The feature selection algorithms were implemented by means of a Ranker search method (Witten et al. 2016). The features selected by each of the techniques are presented in Table 4.

To investigate the performance of the feature selection techniques, a multilayer feed-forward ANN comprising three layers such as one input, one hidden and one output layer was developed. The feature subsets produced by each of the feature selection techniques were used as model inputs in turn. The baseline scenario was also implemented on the ANN. The optimal architecture of the models was established by incrementally changing the number of hidden layer processors from 1 to 10 using a single stepping function. The output layer consists of only one neuron, representing the target variable, water consumption, while a logistic sigmoidal-type activation function within [0, 1] interval was utilized in the hidden layer to rescale the inputs in the range [0.1, 0.9]. A linear activation function was used in the output layer to transform nonlinearities in the inputs into a linear space.

The ANN was trained using a classic DE algorithm (Storn & Price 1997). The crossover probability, ⁠, and mutation scale factor, ⁠, were used to govern the genetic operations during the algorithm run. Following the suggestion of Montgomery & Chen (2010), was set at ‘ multiplied by 10’, where is the number of weights and biases in the selected architecture. Adopting a stepping value of 0.1, the DE algorithm was subjected to sensitivity analysis by varying and incrementally within [0.5, 0.9] and [0.1, 0.5] intervals, respectively. This was aimed at determining the optimal parameter settings to govern the evolution process. The algorithm was thereafter run for 1,000 generations for each of the models. Early stopping (Raskutti et al. 2014) was integrated in the ANN models to address overfitting problems. Early stopping aims to identify the point where minimum error on the validation data set begins to rise and then halts training to prevent overfitting. Early stopping thus ensures that model performance balances model complexity with the errors observed during training and validation.

The methodological framework developed and implemented for this study is depicted in Figure 3.

The performance of the ANN models developed in the study was evaluated based on learning accuracy and model complexity, and the performance evaluation results are presented in Tables 5 and 6. Table 5 compares the performance of the ANN models in reproducing the actual water consumption at the City of Ekurhuleni, while Table 6 presents the optimal model architectures, optimal DE control parameters and ranks for each of the ANN models. The results show a highly competitive performance amongst the techniques employed in this study, with minimal errors (RMSEs) observed in all the ANN models. All the models were ranked based on their average performance across the three statistical measures and over the validation data sets. Overall, the ANN model developed using the Pearson correlation subset performed better than other techniques, producing the lowest error (RMSE) estimate of 4,172 mL. Similarly, the Pearson correlation-based ANN model produced the highest R² and NSE values at 0.9233 and 0.9001, respectively.

The ANN model developed using the Relief-F attribute technique produced the second-best performance, while those developed using the information gain, principal component analysis and symmetrical uncertainty came third, fourth and fifth, respectively. It is interesting to note that all the ANN models developed using the five feature selection techniques converged better during validation than training, implying that the models do not suffer from the ‘curse of dimensionality’ and overfitting which typically plagues ANN models (Adeyemo et al. 2018). This also suggests that the early stopping criterion was effective in preventing overfitting.

Contrastingly, the ANN developed using the baseline scenario had the worst model performance amongst the six ANN models. A slight overfit can be noticed in its training and validation results. This slight overfit could be due to overparameterization, suggesting parameter irrelevancy or redundancy in the full set of potential explanatory variables considered. The performance of the baseline scenario ANN models agrees with the argument of Phyu & Oo (2016) that, if feature selection techniques are employed, the consistency of the full set of attributes can never be higher than that of any subset of attributes. Notwithstanding the rank of the ANN model developed from the baseline scenario, its performance could be referred to as reasonable considering its model architecture (8-2-1), which seems to have the minimal complexity.

Sensitivity analysis performed on the DE control parameters shows that the optimal crossover and mutation probabilities were in the [0.7, 0.8] and [0.3, 0.5] intervals, respectively, across the six ANN models. This suggests a high exploratory search by the DE algorithm, which is often a product of continuous productive search (Montgomery & Chen 2010).

Figures 4 and 5 present plots of observed and forecasted water demands for the training and validation phases, respectively. The plots clearly show that all the models produced a good representation of the water demand pattern in the City of Ekurhuleni. Both the peaks and troughs including sharp spikes in the water demand pattern were reproduced by the feature selection-based models. Some constant values are, however, noticeable in the baseline scenario model during training, possibly due to the inclusion of irrelevant variables that have little or no significant influence on the learning process. The corresponding scatter plots also depict high accuracy and correlation as the observed and forecasted values are close to the line of equality in all the models. The best model representation and the best line of fit are produced by the Pearson correlation-based model.

Table 7 shows the contribution of each potential explanatory variable. The contribution of each variable was determined by a total count across the subsets derived from the five feature selection techniques. The results show that the number of household connections and population contributed the most to model performance, appearing in the subsets produced by the five feature selection techniques. This is followed by the minimum and maximum temperatures as well as HDI, which appeared in three of the five subsets. Although wind speed and rainfall are individually evident in only one of the subsets, their contribution is noteworthy as they, respectively, belong to the subsets that produced the most superior (Pearson correlation) and second-best (Relief-F attribute) performances. Similarly, RH appeared in the subsets of the third- and fourth-best models. These results suggest that, besides temperature, wind speed, rainfall and RH have some influence on water consumption in the City of Ekurhuleni. This may be explained by the climate of the study area which is characterized by wet, windy and humid summers, resulting in higher water use. Results from this study thus agree with the findings of Babel & Shinde (2011) and Huntra & Keener (2017), which found that RH, wind speed and rainfall could have some degree of influence on water consumption, especially in semi-arid regions. Generally, these results suggest that weather and socioeconomic status could have significant impacts on water demand, and thus incorporating them in water demand forecasting studies could result in a more reliable model that considers the impacts of weather and socioeconomic variations.

The capability of five feature selection techniques in finding the optimal subset of features for a water demand forecasting model has been investigated in this study. The performance of the subsets generated by the five feature selection techniques was compared to that of a baseline scenario comprising eight potential explanatory variables, totalling six scenarios. The aim was to develop an improved and reliable municipal water demand model that accounts for the impacts of weather and socioeconomic variations. HDI was introduced for the first time in water demand forecasting as a socioeconomic variable and used alongside weather-, population- and water demand-based variables. Using a combination evolutionary computation and artificial intelligence approach, DE-inspired ANN models were developed; one for each scenario. Results show that minimum and maximum temperatures as well as HDI were selected alongside population and number of household connections which are popularly used in water demand forecasting. Results further show that these three variables contributed significantly to the performance of three of the five models. Pearson correlation proved to be the most superior feature selection technique. DE showcased robustness in fine-tuning algorithm parameter values, thereby producing good performance in terms of the solution efficiency and quality. Generally, this study demonstrates that ANN water demand models can now account for the impacts of weather and socioeconomic variations by incorporating explanatory variables based on weather and socioeconomic factors. This study also suggests that the synergetic use of feature selection techniques, DE algorithm and early stopping criterion could be used to address the limitations of ANN, thereby improving model generalization and forecast accuracy as well as providing a climate variability perspective to water demand forecasting. The methodologies, principles and techniques behind this study foster sustainable development and thus could be adopted in planning and management of water resources. This study is limited to the use of historical water demand, weather and socioeconomic variables in predicting water demand. However, to enhance the applicability of the current ANN predictions, future research will focus on the impacts of other explanatory factors like land use, recharge and run-off on the city's water demand when the information becomes available.

The author wishes to express his gratitude to the City of Ekurhuleni, Statistics South Africa (Stats SA) and South African Weather Service (SAWS) for providing the data used for this study.