Evaluation of classification and decision trees in predicting daily precipitation occurrences

Precipitation is one of the most important input data to hydro climatology and hydrology systems. Precipitation studies and measurements are in most cases, necessary for the study of runoff, groundwater, flood, sediment, etc. Due to rapid population growth and increasing problems of access to drinking water in arid and semi-arid regions such as Iran, the importance of proper and providing reliable rainfall models is increasing day by day. Since precipitation depends on many factors such as temperature, humidity, and evapotranspiration, it is complicated to emulate mathematically and predict the precipitation occurrences in the temporal and spatial domain. The influence of tangible and intangible factors on event of rainfall occurrence is so high that it leads to a complex and chaotic structure. In addition to the dynamic and thermodynamic mechanisms for precipitation to occur at a suitable vertical velocity, the presence of sufficient moisture is also essential. Adequate and appropriate humidity is necessary not only during rainfall but also for a certain time. The connection between the precipitation system and humidity source must be established continuously to compensate for the humidity reduction and continues rainfall. This process will strengthen the system again, and this cycle will lead to significant rain with the help of geographical conditions. In precipitation forecasting, various methods have been proposed that these methods are divided into two general categories: dynamic and empirical models. Due to the computational complexity in dynamic methods and finding features and spatial and temporal relationships of historical precipitation data in empirical methods, in recent years intelligent systems based on data mining have been adopted in the modeling of hydrological processes, which has the potential of modeling complex nonlinear processes and can provide more accurate estimates of regional conditions using meteorological and hydrological data (Bhattacharya & Solomatine 2005).

In recent years, several studies have developed and examined different methods for predicting precipitation. Mandal & Choudhury (2014) predicted maximum daily rainfall using a series of probabilistic models and found that the probability of total monthly rainfall of more than 100 mm in summer and winter was about 75–85%. Ramsundram et al. (2016) compared the decision tree data-driven model with Artificial Neural Network (ANN) based rainfall prediction model. The results showed that the developed model with weakly correlated input data compared to ANN could be forecast future precipitation circumstances. Bahrami et al. (2017), by examining the effect of pre-processing methods on the performance of the ANN, concluded that the minimum-maximum pre-processing method has the best performance in predictions. Nourani et al. (2017) used a combination of decision tree and association rules in long-term rainfall forecasting, and the results showed that the confidence index was estimated to be above 60%. Dash et al. (2018) used artificial intelligence methods (AI) K-NN, ANN, Extreme Learning Machine (ELM) to predict seasonal rainfall; they proposed that AI approaches for predicting rainfall in both summer monsoon and post-monsoon have good potential. Diop et al. (2019), using a hybrid artificial intelligence model, integration of Multilayer Perceptron models (MLP) with whale optimization algorithm (MLP-WOA) predicted annual rainfall. They concluded that the accuracy of standalone MLP using MLP-WOA improved. In addition, other machine learning techniques have also been used in the prediction of precipitation. Balamurugan & Manojkumar (2019) used a machine learning-based approach in the study of short-term rain forecasting. The results implemented those machine learning methods compared with statistical methods had better results. Diez-Sierra & Jesus (2020) used atmospheric synoptic patterns in semi-arid climates with statistical and machine learning methods for predicting long-term rainfall; they demonstrated that selected hyperparameters had the most influence on the performance of machine learning methods. According to the studies conducted in precipitation forecasting, the importance of daily precipitation estimating is well understood. Mohammed et al. (2020) provided a comparative study among multiple linear regression (MLR), support vector regression (SVR), and lasso regression. However, the key issue is their black-box nature since only their predictions are presented, and the rules remain hidden within the black-box itself. In addition, the MLR, though it provides rules, is unable to capture the non-linearity in a data set, making it not suitable (Mohammed et al. 2020). Data mining is a relatively new methodology for finding meaningful relationships between large amounts of data utilizing pattern recognition techniques. Essentially these methods can be categorized in predictive and descriptive modeling, in which predictive modeling may be implemented to estimate particular numerical value. In contrast, descriptive modeling investigates data and focuses on the connections and the hidden relationship between them. Decision trees are the descriptive types of predictive modeling approaches. On the other hand, the decision tree explains its prediction in of clear-cut rules. In addition, in the decision tree, unlike neural networks, nominal data can also use. Consequently, decision tree modeling does provide an apt alternative in the prescriptive data-driven models for precipitation predictions. Decision rules, PART, K-nearest neighbor, J48, M5P, and random forest, etc. are as classification algorithms. PART regards as one of the leading classification algorithms that produce rules. By comparing data to each rule, decision trees create, and the best leaf changes to the new rule. This method combines ripper and C4.5 algorithms and is popular for classification purposes (Hussain et al. 2018). A literature review represents that in the field of rainfall predicting, various models such as machine learning methods, artificial neural network and, etc., with the strength, and weakness of each used. However, there are still some state-of-the-art models, such as logistic model tree (LMT), J48, random forest (RF) and PART, which rarely employee for prediction occurrence of precipitation. Therefore they need to be further compared investigated. Hence, this study aims to examine the probability of daily precipitation occurrence by implementing novel classification, and decision trees including LMT, J48, RF and PART due to the introduction of models with a straightforward and understandable structure for decision making. Although these models use very simple techniques, the field of diagnosis and prediction can work with complex methods as well as ANN models. Antecedent input parameters include precipitation (P), average relative humidity (RH_mean), minimum relative humidity (RH_min), maximum relative humidity (RH_max), average air temperature (T_mean), minimum temperature (T_min), maximum temperature (T_max), sunshine hours (−ssh) and average wind speed (S_w) from 2004–2019 used as the inputs. The daily meteorological data from Pars Abad station, as a case study, are utilized. For best feature selections, five scenarios based on cross-correlation function (CCF) and partial autocorrelation function (PACF) are developed to channel the optimal features to the respective models. To the best of the author's knowledge, the considered LMT, J48, RF, and PART models have not been applied previously for forecasting precipitation occurrence. The development, application and evaluation of the classification and decision trees algorithms as the prescriptive data-driven models for precipitation predictions are the key contribution of this study.

DT is used for data mining and it is one of the powerful and standard tools for classification and prediction that generates a set of rules in emulating precipitations. During the model training process, the data series is subdivided into homogeneous subsets in predicting or regulating an objective variable culminating into its tree regression structure. The process of creating a tree consists of three steps: dividing the nodes and assigning the nodes to the end classes. A decision tree combines a root, a series of branches and finally leaves. LMT, J48 and RF are famous and practical decision trees.

Here C is the number of classes and in this equation, for the least-squares to change in proportion to⁠, it must be ⁠. The structure of LMT showed in Figure 1.

The aim is to maximize the gain, dividing by overall entropy due to split argument by value j. Finally, the tree is pruned to solve the problem of overfitting.

Essentially, the main advantage of the PART method over other methods is that combining the two set patterns, it creates rules that do not require global optimization. PART is distinct and conquers classifier algorithm. The PART algorithm produces decision lists that are used as a set of rules in the list. As new data is added, it is compared to the existing rule and the clause is transferred if the matching rule does not exist. The partial decision tree formed by part results from the combination of C4.5 and repeated incremental pruning to produce an error reduction (RIPPER) algorithm. In this method, the data set is divided back into a partial tree then the test selected and divided to subsets. The subsets are developed based on the average entropy. This process continues until a subset expands and reaches the leaf and in the next steps, other subsets that have not yet risen are selected (Frank & Witten 1998). The best leaf is introduced as a rule (Shawkat & Smith 2006). The tree-building algorithm showed in Figure 3.

Pars Abad is the second-largest city in Ardabil province in north-western Iran which is on the border of Iran and the Republic of Azerbaijan. Pars Abad is located in a flat plain with a warm climate. The meteorological data for this study were from Pars Abad station and the geographical location of Pars Abad station is 72.6 meters above sea level at 39° 36′ north latitude and 47° 46′ east longitude (Figure 4). It should be noted that in this study Pars Abad is only a case study. In this study, daily meteorological data of 16 years related to the period of 2004–19 were used. Daily data included precipitation (P), average relative humidity (RH_mean), minimum relative humidity (RH_min), maximum relative humidity (RH_max), average air temperature (T_mean), minimum temperature (T_min), maximum temperature (T_max), sunshine hours (−ssh) and average wind speed (S_W). The statistical characteristics of implemented meteorological parameters in Pars Abad station are shown in Table 1. Following Table 1, P has the greatest skewness. S_w also shows the skewed distribution. Other studied parameters show normal distributions since they have considerably low skewness values. The data set was divided into two parts. In the first part, 70% of the total data were used for training (2004–2015) and the remaining 30% of data were used to test the studied models (2016–2019). Then, based on rainfall data, the days of the year were divided into two classes: rainfall days and non-rainfall days, and meteorological data were analyzed in the form of 5 series of data with delays of 1–4 days to predict the occurrence and non-occurrence of daily rainfall. The process of the present study illustrated in the Figure 5.

Here, is the predicted value obtained from models, is the observed value and N is the number of data set.

In this study, the feasibility of using meteorological parameters; RH_mean, RH_min, RH_max, T_mean, T_min, T_max, P, −ssh, and S_W in different combinations to predict the occurrence of daily precipitation is tested (Table 1). To this end, we first defined five scenarios by delaying meteorological data from 1–4 days and for each scenario the input parameters were selected based on their correlation coefficient with precipitation. Moreover, for the feasibility of computations, 7–9 input parameters were chosen for each lag. For example, for the first lag, all nine input parameters were selected as scenario one, but in lag 2, instead of using 18 input parameters resulted, i.e., lags 1 & 2 for all the nine variables, based on cross-correlation function (CCF) and partial autocorrelation function (PACF), seven best inputs with higher correlations were selected to have low latency models. This process was continued for all lags and finally, 5-input combinations were established, as mentioned in Table 3. Furthermore, since the correlation of RH_max(t−1) was high, it was used in all scenarios. Moreover, the absolute value of the correlation coefficient of meteorological parameters was the criteria used for selecting input parameters. As a result, five scenarios were developed based on the CCF and PACF for calibration and validation of the studied methods (Figure 6).

Because there is not any direct method for dividing the training and testing data into data-driven approaches, scholars have used different proportions, such as 67% of data for training (Qasem et al. 2019; Kargar et al. 2020), or 70% for training (Asadi et al. 2020; Samadianfard et al. 2020) and as high as 80% for training (Zounemat-Kermani et al. 2019) has been used. Therefore, for developing the LMT, J48, RF, and PART algorithms, the data were divided into 70% training data (2004–2015) and 30% testing data (2016–2019). Then, based on rainfall data, the days of the year were divided into two classes: rainfall days and non-rainfall days, and meteorological data were analysed in the form of 5 series with delays of 1–4 days and the combination of parameters in each scenario was done based on the cross-correlation function (CCF) and partial autocorrelation function (PACF) for calibration and validation of the studied methods to predict the occurrence or non-occurrence of daily rainfall (Table 3). The input parameters in Table 3 were selected based on the higher values of correlation coefficients in different lags. In other words, while considering different lags, 7–9 meteorological parameters with higher correlation coefficients were selected as input parameters. Moreover, the computations were completed with Weka software, which contains a large number of machine learning and data mining techniques that will enable the comparison of machine learning techniques and allow users to access features such as visualization and analysis of many data mining algorithms (Witten & Frank 2000).

Our motivation for conducting the present study is to introduce the capability of classification and decision tree models to anticipate the occurrence of daily rainfall with the help of effective meteorological variables in the form of multi-day lags. The daily scale data of meteorological station, namely Pars Abad was used to expand LMT, J48, RF, and PART methods. The evaluation of the decision tree-based modeling approaches in predicting daily precipitation levels is presented here. Table 4 explains the details of the criteria for implemented LMT, J48, RF, PART, and methods in predicting precipitation conditions based on a series of daily meteorological data. It reveals that in Scenario 1 with one day lag, PART method with weighted average of precision = 0.770, Recall = 0.806, F-measure = 0.764, MCC = 0.242, AUC = 0.681 and PRC = 0.759 had the best performance compared to the other three methods. In the same scenario, RF with precision = 0.741, Recall = 0.784, F-measure = 0.750, MCC = 0.180, AUC = 0.691 and PRC = 0.773 was recognized as the method with the weakest performance. Then, in Scenario 2 and 3, the PART method had the highest precision and Recall, but in Scenario 4 and 5, the J48 method had the highest precision and again, the PART had the most Recall. In general, the highest and lowest precision was in Scenario 3, the PART method registered precision = 0.788, achieved the highest precision, and LMT with precision = 0.732 had the lowest precision among all defined scenarios. Furthermore, Table 5 shows the results of the utilized indicators, including Accuracy (%), Kappa, RMSE, and MAE for all studied methods. The results of different algorithms used in this study to predict the occurrence or non-occurrence of precipitation showed in Table 5. According to the results in Table 5, the PART method had the highest accuracy of the patterns used in precipitation predictions, while the J48 method had the lowest accuracy. In Scenario 1, the PART method with Accuracy = 80.5503%, Kappa = 0.2007, RMSE = 0.3879, and MAE = 0.2856 was selected as the superior method in predicting precipitation conditions one day later. Moreover, the LMT with Accuracy = 80.4878%, Kappa = 0.1895, RMSE = 0.3845, and MAE = 0.3047 had a better performance for predicting precipitation and J48 and RF methods were in the next position to predict the probability of precipitation in scenario 1, respectively. In Scenario 3 and 5, the RF method was more suitable for predicting rainfall conditions due to its high accuracy, F- measure and MCC compared to other methods. Although daily precipitation was predicted in all methods with more than 78% accuracy, the PART method was recognized as the superior one in all scenarios due to its accuracy of nearly 80%. Additionally, in scenario 2 the J48 method with Accuracy = 78.3615, Kappa = 0.1999, RMSE = 0.4245 and MAE = 0.2796 had the weakest performance comparing with other methods. So, the results indicate that the PART method, rule-based, compared to the other three methods based on tree classification, has a significant ability to estimate the days on which precipitation occurred or not. Therefore, PART, LMT, RF, and finally J48 methods have the first to fourth position, in predicting occurrence or non-occurrence daily rainfall. The results of this study are consistent with the findings of Mahtabi et al. (2018), who reported that using meteorological data from 1–5 days ago, daily precipitation was predicted with an accuracy of more than 80% and the performance of DT was the best. Also, the relative humidity and maximum daily temperature had the highest correlation of the occurrence of daily precipitation. The results are also agree with Agnihotri & Mohapatra (2012), which examined the forecast of daily summer monsoon rains in the Karnataka region of India and concluded that the highest correlation of daily rainfall is with relative humidity parameters and minimum daily air temperature.

To evaluate the susceptibility of the models in prediction more clearly, Figure 7 displays a detailed diagram of the weighted average accuracy of the algorithm criteria for all four methods. As shown in Figure 7, the PART method showed higher capability in predicting precipitation in most scenarios due to its accuracy of nearly 80%.

The PART method with more input data, estimates the occurrence of precipitation with higher accuracy, while the performance of this method decreases when the number of input variables is low. For future studies, it is suggested that the methods used in this study be evaluated for different areas (arid or humid) with different lags. In this study, we used only a rule-based model in predicting the occurrence of daily precipitation, which ironically provided the best results among other methods, however in general it cannot be concluded that the performance of all rule-based models will be better than models based on tree classification or not. So, it is recommended that in future studies, the performance of other rule-based models be examined.

In the current study, the daily precipitation occurrences in Pars Abad station was predicted with the application of classification and decision tree models including PART, J48, LMT, and RF using meteorological parameters. To this end, five scenarios were defined by delaying meteorological data from 1–4 days and for each scenario the input parameters were selected based on the CCF and PACF functions. A robust performance evaluation of the models was carried out using accuracy and Kappa. Comparison of the results showed that using daily meteorological data, rainfall up to 4 days can be predicted with more than 78% accuracy. Among five selected scenarios, Scenario 1 with the PART method with accuracy of 80.5% was determined to be the best case. The performance of PART method was more suitable than the other methods, because in all five scenarios, the accuracy of the PART was higher than 80%. In all scenarios, the relative humidity and maximum daily temperature had the most significant influence in predicting daily precipitations. Conclusively, based on the obtained results, the PART method was recommended in predicting precipitation occurrences using the previous meteorological data.

Data cannot be made publicly available; readers should contact the corresponding author for details.