Application of ensemble empirical mode decomposition based on machine learning methodologies in forecasting monthly pan evaporation

Evaporation is probably the most complicated and difficult parameter to forecast among all the elements of the hydrological cycle due to the complex interactions among the hydrologic components of water surface, land, atmosphere process, and vegetation. Hence, proper forecasting of evaporation is a significant issue in water resources management and agriculture, particularly in semi-arid and arid areas. Because of temperature differences, this hydrological phenomenon is a nonlinear process which occurs in nature (Irmak et al. 2002; Trajkovic & Kolakovic 2010; Tezel & Buyukyildiz 2016; Malik et al. 2017). There are many parameters influencing pan evaporation (P_E) rates, including air temperature (T_A), solar radiation (S_R), sunshine hours (H_S), relative humidity (R_H) and wind speed (W_S). Although accurate estimation of P_E is of high importance, especially in studies related to integrated water resources management, the effects of various weather parameters on pan evaporation in different regions are still not well understood.

In general, there are two main approaches, namely direct and indirect procedures, for calculating and forecasting evaporation. Direct measurements of P_E are widely applied for predicting evaporation. Indirect methods such as energy-budget, water budgets, and Penman–Monteith techniques are based on various climate variables (e.g. TA, W_S and S_R), which are not easily accessible. It is not possible nowadays to extract a mathematical relationship that includes all of the physical processes due to the nonlinearity and complexity of the evaporation process. Several researchers therefore recommend the use of models (Tezel & Buyukyildiz 2016; Wang et al. 2017; Tan et al. 2018) that do not require a priori knowledge of the underlying physical processes or excessive data. Such models are considered to be more adequate (adaptable) in real-decision support systems compared to the physical-based methods (Kisi 2007; Ch et al. 2014; Deo & Şahin 2015). Considering that evaporation is highly nonlinear, these techniques are able to map the nonlinearities associated with evaporation and its input parameters to a high degree of accuracy.

In the last decade, machine learning (ML) methods such as Adaptive Neuro Fuzzy Inference System (ANFIS), artificial neural network (ANN), model tree (MT), gene expression programming (GEP), extreme learning machine (ELM) and support vector machine (SVM) have been successfully employed for solving a wide range of environmental and water engineering problems (Adamowski & Karapataki 2010; Chua et al. 2011; Sattar 2013; Ch et al. 2014; He et al. 2014; Ebtehaj et al. 2015; Deo et al. 2016; Najafzadeh et al. 2016; Li et al. 2017; Mouatadid & Adamowski 2017; Rezaie-Balf & Kisi 2017; Rezaie-Balf et al. 2017a, 2017b; Yu et al. 2017). Many studies have also been conducted using ML methods for predicting pan evaporation (Tabari et al. 2012; Lin et al. 2013; Kim et al. 2015; Tezel & Buyukyildiz 2016; Kisi et al. 2016; Malik et al. 2017; Wang et al. 2017), although the development of powerful approaches and methods with high levels of reliability to achieve accurate forecasts remains a major challenge. This is because time series P_E data are generally highly nonlinear and seasonal. Under such situations, applying the raw data to the model directly may not produce accurate results if the data are highly nonlinear. In this context, using a preprocessing technique may enhance the model's performance (Wu et al. 2010). In recent years, researchers using data pre-processing tools such as principal component analysis (PCA), continuous wavelet transform (CWT), moving average (MA), wavelet multi-resolution analysis (WMRA), maximum entropy spectral analysis (MESA), and singular spectrum analysis (SSA) have been successful in improving forecasting accuracy (Hu et al. 2007; Kişi 2009; Sang et al. 2009; Wu et al. 2009; Wu & Chau 2011; Nourani et al. 2013; Sang et al. 2013; Rezaie-Balf et al. 2017a, 2017b; Tiwari & Adamowski 2017; Wen et al. 2017).

More recently, new noise assisted data analysis techniques, empirical mode decomposition (EMD) and ensemble empirical mode decomposition (EEMD), were pioneered by Huang et al. (1998) and Wu & Huang (2009), respectively. The EEMD approach is an advanced form of traditional decomposition techniques (e.g. wavelet decomposition and Fourier decomposition), and is an intuitive, empirical and self-adaptive data processing tool which was developed for non-linear and non-stationary time series. Recently, several successful applications have been presented which utilized EEMD to forecast time series phenomena such as rainfall and runoff (Karthikeyan & Kumar 2013; Latifoglu et al. 2015; Wang et al. 2015; Zhao & Chen 2015; Barge & Sharif 2016), water quality parameters (Fijani et al. 2019), soil moisture (Prasad et al. 2018), and electricity demand (Al-Musaylh et al. 2018). For instance, Napolitano et al. (2011) carried out a study to explore several aspects of ANN in forecasting daily stream flow time series using EMD. They showed that using EMD analysis, the ANN can have a higher prediction accuracy and reliability. Likewise, Sang et al. (2012) presented a hybrid EMD and maximum entropy spectral analysis method. They proved that this method, in addition to improving the period identification, also can improve overall period identification by being able to discriminate period, trend, and noise. Wang et al. (2013) applied EEMD as an adaptive data analysis tool for decomposing annual rainfall time series in modeling rainfall-runoff process using a SVM model. They employed particle swarm optimization (PSO) to calculate optimal SVM control parameters. Similar to previous investigations, they showed their proposed model's efficiency and capability in rainfall-runoff forecasting. Moreover, a four-stage model, EEMD-based radial basis function neural network (RBFN) and linear neural network (LNN), was developed by Di et al. (2014) for hydrological time series forecasting. They considered six datasets and showed that their model outperformed the RBFNN, EMD-RBFNN, and EMD-WA-RBFNN-LNN methods.

The overall goal of the current study is to propose a new hybrid model to improve the monthly P_E forecasting. The SVM and MT techniques were investigated and improvements of the models' accuracy were tested by coupling them with the EEMD procedure. The data used are the monthly P_E data recorded at Siirt and Diyarbakir stations, located in Turkey. The rest of the present research is organized as follows. The next section provides a case study and data analysis, followed by a section describing the methodology of EEMD and each forecasting method (i.e. MT and SVM). Next, the statistical indicators applied in this study are provided. The penultimate section represents the forecasting results of the hybrid models and the comparison results, and the final section consists of a summary and conclusion of outline results.

To validate the performance of the proposed approach, the monthly climatic data and P_E from Siirt station (latitude 37° 55′ N, longitude 41° 56′ E, and altitude 896 m) and Diyarbakir station (latitude 37° 54′ N, longitude 40° 13′ E, and altitude 649 m), operated by the Turkish Meteorological Organization (TMO) were utilized. All the climatic data were obtained from TMO. It is the only legal organization that provides meteorological information in Turkey. TMO measures pan evaporation, which are not publicly available, using Standard A pans (World Meteorological Organization standards). The 187 datasets of climatic variables comprising wind speed (W_S), temperature (T_A), relative humidity (R_H) and solar radiation (S_R) of the aforementioned stations consisting of 33 years (1975–2008) are described in Table 1. Furthermore, data were not available for January, February, March and December and about 5% of the whole data were missing for May and April. Data were directly used in this study without any pre-processing (e.g. filling in missing ones). Seventy-five per cent of the whole data was used for training of the applied models and the obtained models were tested by using the remaining 25% data.

The location of the study area is illustrated in Figure 1. In this area, average annual precipitation is between 400 and 800 mm, falling mainly in the winter months of December to January. The mean values of the pan evaporation during spring and summer (main crop growing seasons) at Siirt and Diyarbakir are 8.81 and 7.47 mm, respectively. Summers are hot and dry and the annual temperature ranges from 16.0 to 46.8 °C (Keshtegar & Kisi 2017). According to the de Martonne (1926) aridity index, both Siirt and Diyarbakir stations have a dry, semiarid (semi-desert) climate.

Two ML algorithms, SVM and MT, were developed and coupled with EEMD as a data pre-processing procedure for forecasting monthly P_E in Siirt and Diyarbakir. In this section, the description of EEMD, SVM and MT are briefly presented.

The EEMD method, proposed by Wu & Huang (2009), is an empirical procedure used to represent a nonlinear and non-stationary signal from original data. This data pre-processing method is an improvement of the EMD, self-adaptive time-frequency transformation procedure, and does not rely on process information. EMD is used mainly for decomposing the original time series data into a finite and low number of oscillatory modes depending on the local characteristic time scale (Huang et al. 1998). The oscillatory modes are revealed by intrinsic mode function (IMFs) components, embedded in the data. These component signals are represented as a sum of zero-mean well behaved fast and slow oscillation modes (IMFs) which have two requirements (Huang et al. 1998; Wu & Huang 2009): (i) Throughout the whole length of data, the number of zero-crossings and the number of extrema must either be equal or at least differ by one; and (ii) At any point, the mean value of the local maxima and minima envelope is zero. According to these properties, some meaningful IMFs can be well defined. Generally, an IMF indicates a simple oscillatory mode, compared to simple harmonic function. Based on this definition, a shifting process of original time series can be briefly expressed as follows (Huang et al. 1998).

Step 1: Identify all extrema (local maxima and minima) points of the given time series y(t);

Step 2: Connect by spline interpolation, to create an upper envelope of local maxima points e_max(t), and a lower envelope e_min(t) of all minima points;

Step 5: If h(t) satisfies the two properties of IMF as indicated by a predefined stopping criterion, h(t) is indicated as the first IMF (written as c₁(t) where 1 is its index); If h(t) is not an IMF, y(t) is replaced with h(t) and iterate steps 1–4 until h(t) satisfies the two conditions of IMFs.

Support vector machines, firstly presented by Cortes & Vapnik (1995), is an effective tool for solving nonlinear problems by relating input variables to a defined output. The least square SVM is an advanced type of SVM, developed by Suykens & Vandewalle (1999), and is used for chaotic time series prediction. The SVM on the basis of the structural risk minimization (SRM) principle, minimizes the expected error in order to reduce the over-fitting. In SVM, in order to separate the data patterns, the input data are mapped into a higher dimensional feature space.

The MT is a state-of-the-art hierarchical algorithm, presented by Quinlan (1992), for providing the relation between input–output parameters. This algorithm is used to solve the problem by dividing the data space into several sub-problems (sub-spaces) and builds piecewise linear regression models for each sub-domain. The data records, in classification trees, are classified by sorting the tree from root node to some leaf nodes of the tree. The MT is based on the well-known CART algorithm (Breiman et al. 1984), which deals with continuous-class learning attributes. This algorithm is known to be one of the most efficient methods to present physically meaningful insights for a given phenomenon (Kisi et al. 2016). Figure 2 illustrates the tree-building process, within four linear regression models and the knowledge extracted from the structure for corresponding sub-domains. Furthermore, a general tree structure of MT approach is illustrated in Figure 2(b). In this figure, if X₁ and X₂ are less than 3 and 1.5, respectively, the fifth model should be considered as a form of Y = a₀ + a₁X₁ + a₂X₂.

In the present research, to assess the EEMD-SVM, EEMD-MT, SVM and MT models, several error indicators were used:

• 1.
  Nash–Sutcliffe Efficiency (NSE): This criterion is taken into account to evaluate the ability of hydrological models. The highest value of unity demonstrates a perfect fit between observed and predicted P_E where a negative value means that the model performs worse than the arithmetic mean of the time series (Nash & Sutcliffe 1970):

  

  (11)
• 2.
  Root mean square error (RMSE): RMSE shows the standard deviation of the forecasting errors or residuals. RMSE varies from zero for perfect estimates to large positive values for poor estimates (Hyndman & Koehler 2006):

  

  (12)
• 3.
  Performance index (PI): PI values vary from 0 to +∞, with lower values showing better accuracy. A PI value close to zero (e.g. PI ≤ 0.2) indicates that the model predicts the observed values well (Gandomi & Roke 2015):

  

  (13)
• 4.
  Willmott's index of agreement (WI): WI is a standardized measure for model prediction error and ranges from 0 to 1. The values close to 0 show poor accuracy while the values close to 1 reveal good performance (Willmott et al. 2012):

  

  (14)

The main goal of the EEMD-MT/SVM models is to forecast monthly P_E at different stations in Turkey. According to Figure 3, the workflow of the proposed approach contains three main steps to enhance the performance of proposed forecasting techniques that can be expressed as follows:

1. In Step 1, the EEMD procedure is firstly utilized to decompose the all original input/output time series y(t) into several IMF components C_i(t) (i = 1, 2, 3, … , n) and one residual component r_n(t).

2. Next, for each extracted IMF component and the residual component (for example IMF1), the SVM and MT are established as forecasting models to simulate the decomposed IMF and residual components, and to predict each component by the same sub-series (IMF1) of four input variables, respectively.

3. In Step 3, the forecasted values of all extracted IMF and residue components by the SVM and MT models are aggregated to generate the modelled P_E time series.

To summarize, the hybrid EEMD-SVM/MT forecasting tools establish the idea of ‘decomposition and ensemble’. The decomposition is used to simplify the forecasting process, and the ensemble is utilized to formulate a consensus forecasting on the original time series.

Next, the accuracy of the proposed models in forecasting of monthly P_E was assessed at both training and testing stages (Table 2). Based on the comparison between SVM and MT approaches, the MT is found to perform better than the SVM in terms of statistical metrics during training and testing stages at Siirt station. The quantitative evaluation results in Table 2 also show that EEMD-MT has a lower value of RMSE (0.765 mm) and PI (0.079 mm) compared to the EEMD-SVM model in the training stage for the Siirt station. Moreover, the EEMD-MT provided more accurate results in terms of NSE (0.89), WI (0.969), and LMI (0.699) than the SVM and MT.

In the testing stage, with respect to the measured value of P_E, it can be seen from Table 2, in terms of NSE and WI, that the EEMD-MT and EEMD-SVM models are able to model P_E at Siirt station well. The SVM model has the highest error in estimating P_E (RMSE = 2.663 mm; PI = 0.137 mm) compared to the other models. In addition, the EEMD-MT model improves the MT model with 44.71 and 49.50% reductions in term of RMSE and PI, respectively, and a 36, 8 and 42% increase in NSE, WI, and LMI, respectively.

Comparison of the modelled P_E is made against measured values in the scatter plots (Figure 5) for both training and testing stages. The figure illustrates the superiority of the EEMD-MT model during the test stage for Siirt station. The superiority of the EEMD-MT model over the SVM, MT, and EEMD-SVM models is also clear from Figure 6. These results support our hypothesis that the EEMD method is suitable for decomposing monthly P_E time series, and the idea of ‘decomposition and ensemble’ is feasible and the proposed EEMD-SVM/MT models can overcome drawbacks of the individual SVM and MT models by generating a synergetic effect in forecasting. Thus the monthly P_E data decomposed by using the EEMD procedure can improve the forecast results at Siirt.

Similar to the methodology used for Siirt station, four proposed models (i.e. SVM, MT, EEMD-SVM, and EEMD-MT) were developed for the purpose of forecasting monthly P_E at Diyarbakir station. Specifically, SVM and MT were adopted as benchmark models and the results compared with the conjunction models which used EEMD to pre-process the original monthly P_E data, adopting the phase-space reconstruction method to design the input vectors. The training and testing data sets were identical across all models. Table 3 indicates the performance results for the four models in terms of the four aforementioned statistical error metrics. The results show that the hybrid techniques have the lowest RMSE and PI values and the highest NSE and WI, outperforming the SVM and MT models, at both training and testing stages. In the training stage, the comparison of the SVM and MT models shows that the MT model has better accuracy (NSE = 0.882, WI = 0.953, and LMI = 0.755) in forecasting of P_E; however, these models' results are significantly improved by coupling with the EEMD technique. The EEMD-MT approach outperformed all other models (Table 3). By integrating the EEMD with SVM, RMSE and PI were reduced to 1.270 and 0.138 mm in the training stage, respectively. It can be seen from Table 3 that by coupling with EEMD, the RMSE and PI of single MT were reduced by 25.9–28.9 and 19.4–75.5% in the training and testing stages, respectively. Improvements in the forecast results were approximately 5.2, 6.1 and 4.5% respectively, for NSE, WI and LMI values at the training stage, and 1.7, 2.2 and 11.6%, respectively, in the testing stage.

The measured and forecasted values of P_E during training and testing stages are compared in Figures 7 and 8 for the four models. It can be seen from Figure 8 that SVM and MT models are able to follow the general trend of measured P_E quite well, although these models are not able to match the extreme high and low values accurately, especially at the latter part of the testing period. In fact, the SVM and MT models under-predict P_E for the testing data from 2006 onwards, whereas the EEMD-SVM/MT models are able to provide accurate estimates during this period of time (with the exception of 2008 where it is noticed that P_E is under-predicted by all models). In general, the results of this analysis illustrate that the proposed EEMD-SVM/MT models are able to obtain better results than the SVM/MT models, shown by the improvements in different evaluation measures.

To further evaluate the performance of the proposed models, the results were assessed in the spring and summer seasons only, since these are the main crop growing periods. Hence, further analysis was conducted for the period from May to October for 1975–2008 in both stations. Table 4 illustrates the results of the EEMD-SVM and EEMD-MT models in forecasting P_E from May to October for Siirt station. As can be seen in Table 4, the EEMD-MT has a lower error in terms of RMSE and PI for all months compared to the EEMD-SVM model. It is notable that the models' accuracy (NSE, WI and LMI) from May to July reduces but then increases for the months of August, September, and October.

In Diyarbakir station, the forecasted P_E values in both the spring and summer seasons with the EEMD-SVM/MT models have a similar trend to the Siirt station (Table 5). Also, it can be said that the EEMD-SVM has inadequate accuracy in most of the months and it could just be a drawback of this model. Moving to a more detailed analysis on seasonal forecasting of the proposed models, Figure 9 illustrates RMSE and PI indices along May–October for the aforementioned period. As can be seen in both stations, the EEMD-MT model has lower error compared to EEMD-SVM. This indicates that the EEMD-SVM model may not provide a complete solution to every case. This also shows that accurate prediction of monthly P_E is a complex issue.

Predicting a hydro-meteorological time series is often complicated and frustrating. The series may appear completely random; a volatile sequence showing no signs of predictability. Time series are full of patterns and relationships. Decomposition aims to identify and separate them into distinct components, each with specific properties and behavior. In this study, the EEMD procedure was applied to tackle the trends and random behavior of time series data which may impact on accuracy of P_E forecasting. This research attempted to improve SVM and MT models in the prediction of pan evaporation by coupling with the EEMD procedure. The P_E time series from two climatic stations in Turkey, Siirt and Diyarbakir, were utilized for validation of the proposed coupled models. Firstly, the SVM and MT models alone were applied to forecast monthly P_E. In order to enhance the forecasting accuracy of the typical models, coupled EEMD-SVM and EEMD-MT models were then used. For the coupled models, the original time series data were decomposed into eight IMFs and one residual for P_E modeling process. Input variables included T_A, R_H, W_S, and S_R. The performance of the proposed models was assessed in terms of NSE, RMSE, PI, WI, and LMI in the training and testing phases. At Siirt station, the EEMD-MT model provided more accurate results in terms of NSE (0.89), WI (0.97), and LMI (0.70) than the SVM (NSE = 0.46, WI = 0.81, and LMI = 0.34), MT (NSE = 0.65, WI = 0.89, and LMI = 0.49), and EEMD-SVM (NSE = 0.80, WI = 0.93, and LMI = 0.60) in the testing stage. For the Diyarbakir station, the highest P_E forecasting accuracy was achieved with the EEMD-MT model (NSE = 0.92, WI = 0.98, and LMI = 0.80) compared to others. The results showed that the SVM and MT models coupling EEMD generally provide better accuracy and are superior to the SVM and MT models alone in forecasting monthly P_E at both Siirt and Diyarbakir stations. The proposed models can be applied to different climate conditions and EEMD may be compared with other pre-processing methods such as CEEMDAN and improved CEEMDAN in future studies.