Forecasting carbon dioxide emissions: application of a novel two-stage procedure based on machine learning models

ANFIS

adaptive neuro-fuzzy inference system

ANN

artificial neural network

ARIMA

autoregressive integrated moving average

BLSTM

bi-directional long short-term model

CART

Classification And Regression Tree

CO₂

carbon dioxide

CV

coefficient of variation

G20

Group of Twenty

GDP

gross domestic product

GM

grey model

HHO

Harris hawk optimization

KF

kalman filter

KLS

acronym of KF, LSTM, and SVM

KKT

Karush–Kuhn–Tucker

LSTM

long short-term model

MAE

mean absolute error

MAPE

mean absolute percentage error

ML

machine learning

RBF

radial basis function

Relu

rectified linear unit

RF

random forest

Ridge

ridge regression

RMSE

root mean squared error

SD

standard deviation

SOM

self-organizing map

SVD

singular value decomposition

SVM

support vector machine

SVR

support vector regression

According to the Global Carbon Budget 2021 (Friedlingstein et al. 2022), carbon dioxide (CO₂) emitted to the atmosphere attributed to human activities keeps growing at an unprecedented rate. Of all CO₂ emissions during the last 70 years, 70% were emitted after the 1960s, and 33% were emitted in this millennium. The astronomical amount of CO₂ emitted is exacerbating global warming. Consequently, considerable impacts on the ecological environment are introduced including crop production, rainfall, water scarcity, ocean acidification, etc., which, conversely, causes severe harm to human beings (Valipour 2017; Bhatt & Hossain 2019; Rehman et al. 2021).

In the last 10 years (2012–2021), around 89% of total anthropogenic CO₂ emissions had originated from fossil fuel combustion (Friedlingstein et al. 2022). Consumption of fossil fuels, such as coal, oil, and natural gas, is a substantial factor that affects CO₂ emissions. Therefore, it is imperative to promote the transformation of energy consumption structure from fossil fuels to renewable energy such as hydropower (Kuriqi et al. 2019, 2020, 2021; Malka et al. 2022), solar energy (Rabaia et al. 2021; Djaafari et al. 2022), and wind energy (Sadorsky 2021).

China takes its share of responsibility in tackling the climate change issue. At the 75th United Nations General Assembly in 2020, China stated its dual carbon commitment to achieve carbon peaking before 2030 and carbon neutrality before 2060. To make informative decisions in achieving these ambitious goals, it is necessary to make an accurate forecast of CO₂ emissions coming from fossil fuel consumption.

There is rich literature addressing the issue of CO₂ emission forecast, with various forecasting methods proposed. Classical methods such as linear regression (Al-Mulali et al. 2016; Wang et al. 2019), grey models (Lu et al. 2009; Lin et al. 2011; Lotfalipour et al. 2013), and the time-series models, e.g., autoregressive integrated moving average (ARIMA) (Nyoni & Bonga 2019; Yang & O'Connell 2020; Ning et al. 2021), are frequently used. It has been widely concerned that the performance of linear regression can be compromised by the violation of the independence or normality assumptions (Gallo et al. 2014). Grey models focus on dealing with small data and poor information. The forecasting performance of grey models may be decreased when there are many independent variables or data. Moreover, the often-used grey model GM(1,1), together with ARIMA, is, by definition, unworkable for the particular forecasting task when multiple independent variables are included.

In contrast, machine learning (ML) methods are more flexible and show great potential in modeling nonlinear and complex patterns. For example, Stamenković et al. (2015) use a back-propagation neural network and a general regression neural network to forecast emissions of methane. It is observed that both neural network models are more effective than multiple linear regression. Saleh et al. (2016) utilize support vector regression (SVR) to predict the expenditure of CO₂ emissions based on energy consumption in Yogyakarta and acquire a low prediction error. Acheampong & Boateng (2019) establish an artificial neural network (ANN) model to forecast the carbon emission intensity for Australia, Brazil, China, India, and the USA. Their study also identifies the most sensitive variable to each country's carbon emission intensity.

Many studies suggest that a combination of two or more models exhibits better-forecasting performance than a single model in various applications. For instance, Mardani et al. (2020) propose a multi-stage method to forecast CO₂ emissions in G20 (Group of Twenty) countries. They use singular value decomposition (SVD) to forecast missing data, self-organizing map (SOM) to cluster data, and ANN and adaptive neuro-fuzzy inference system (ANFIS) to forecast CO₂ emissions based on the gross domestic product (GDP) and energy consumption. Compared against methods using only ANN, ANFIS, or multiple linear regression, the multi-stage approach yields the lowest forecast error. Wang & Zhu (2021) use the Johansen cointegration test and the neural network autoregression model to forecast China's CO₂ emissions based on assumptions of three levels of GDP growth rate, which results in a different amount increase in natural gas consumption and production. Morshed-Bozorgdel et al. (2022) find that a two-level ensemble of ML algorithms captures the high variation in wind speed. Farzin et al. (2022) use a bi-directional long short-term model (BLSTM) and the Harris hawk optimization (HHO) algorithm to optimize the model's hyperparameters to forecast the underground water table in Iran. They find that their BLSTM–HHO method is superior to benchmark methods regarding assessment criteria such as mean absolute error (MAE), root mean square error (RMSE), and forecast variance.

Li (2020) develops the KLS (acronym of KF, LSTM, and SVM) method, which is a fusion of Kalman filter (KF), long short-term memory (LSTM), and support vector machine (SVM), to forecast China's carbon emissions. In the meantime, ridge regression (Ridge) is used to select independent variables. The KLS method integrates time-series forecast and variable selection and has been confirmed to be more accurate than the four existing methods. Geevaretnam et al. (2022) apply three ML models, namely random forest (RF), SVM, and ANN to forecast global CO₂ emissions and forecast performance was compared in terms of MAE, RMSE, and mean absolute percentage error (MAPE). It reveals that SVM produces the lowest forecast errors.

Accordingly, we will consider a hybrid of ML models to improve the forecast accuracy of CO₂ emissions in China.

We observe that, in the literature of CO₂ emission forecast, few studies are explicitly discussing the time domain of their dependent and independent variables (Saleh et al. 2016; Acheampong & Boateng 2019; Mardani et al. 2020). So it is possible that some of them train models on data sets in the form of ⁠, where T is the observation length of the time series. Then, for the testing set, the value of independent variables at time t + h, namely⁠, is substituted into the trained model to forecast the value of the dependent variable at time t + h to get ⁠. Such a procedure is suitable when investigating the dynamic links between independent variables and dependent variables. But it is inappropriate and unnecessary to forecast CO₂ emissions by giving independent variables at the same time domain. Because the released data of production-based CO₂ emissions are calculated by multiplying activity data (i.e. fuel consumption data in the industrial sector) by corresponding emission factors by the type of fuel (Liu et al. 2020; Friedlingstein et al. 2022). That means, when independent variables, i.e. activity data at time t + h are released, the CO₂ emission data at time t + h are also released.

In practical applications, policymakers are more interested in forecasting CO₂ emissions at time t + h through independent variables at time t. That is, models should be trained on data sets in the form of ⁠, where the forecasted value, ⁠, is obtained by given ⁠. There is a time gap h in the dependent variable, so data sets in the form of will be called lagged data sets thereafter. For instance, Hou et al. (2022) aim to forecast China's short-term CO₂ emissions at time t + w via independent variables at time t. It converts multivariate time-series data into labeled sample data using the sliding window method and different shallow learning approaches are tested to find the best one. Faruque et al. (2022) also train several deep-learning models on lagged data sets in their second step to forecast Bangladesh's CO₂ emissions.

We will explore the functional relationship between and ⁠. However, when forecasting CO₂ emissions at time t + h using independent variables at time t, the forecast errors may increase in contrast with using independent variables at time t + h. To our knowledge, there is no work focused on lowering the forecast errors caused by the time gap, h, in the CO₂ emission forecast field. We propose a novel two-stage forecast procedure, adapting the procedure in Patel et al. (2015) by making necessary modifications to reduce the forecast errors. The first stage forecasts independent variables at time t + h with their historical values up to time t, i.e. is calculated via ⁠. The second stage forecasts CO₂ emissions at time t + h with forecasted independent variables at time t + h, i.e. ⁠. We make an effort to contribute to the existing knowledge in the following ways:

• Clarify the necessity to train models on lagged data sets when forecasting CO₂ emissions.

• Propose a novel two-stage forecast procedure based on ML models to reduce the forecast errors caused by the time gap in the dependent variable of lagged data sets. It is shown that four hybrids of ML models in the two-stage procedure almost all have lower forecast errors than four individual ML models in the single-stage procedure, respectively.

• Develop a forecasting tool for CO₂ emissions from a real-case data set in China. A model with high accuracy could provide a guiding tool for the formulation of China's carbon emission reduction policy.

There are several novelties of the proposed two-stage forecast procedure compared to that of Patel et al. (2015):

• The historical values of the independent variables are used to forecast their future counterparts in the first stage. Thus, this method introduces no exogenous variables to update independent variables and greatly simplifies the forecasting procedure of Patel et al. (2015).

• Patel et al. (2015) forecast stock prices in the financial field, while we attempt to apply a two-stage forecast procedure based on ML models to data sets dealing with macroeconomics and climatology. We expand the application domain of the two-stage forecast procedure.

• Four ML models, i.e. SVR, RF, Ridge, and ANN, are applied in this article, while Patel et al. (2015) only tried SVR, RF, and ANN.

Four supervised ML models, namely SVR, RF, Ridge, and ANN, are introduced as the building blocks of our forecast procedure. Denote a training set as ⁠, where D is the number of features and n is the sample size.

The value of -insensitive loss function is zero when the deviation between the observed value and the forecasted value is no greater than ⁠. Equation (4) makes SVR robust to outliers and can prevent overfitting to some extent. It is also verified that SVR has excellent performances in regression and time-series forecast applications (Smola & Schölkopf 2004).

Setting ⁠, a grid search method (Mselmi et al. 2017; Papouskova & Hajek 2019) is conducted to determine the optimal parameters that give the smallest test error by choosing from all possible value combinations in Table 3.

Table 3

Tested values of SVR parameters

Parameters .	Tested values .
Kernel function	Polynomial kernel, RBF
Polynomial kernel,	{1,2,3,4,5}
Regularization parameter,	100 numbers equally spaced between [0.01,10]

Parameters .	Tested values .
Kernel function	Polynomial kernel, RBF
Polynomial kernel,	{1,2,3,4,5}
Regularization parameter,	100 numbers equally spaced between [0.01,10]

View Large

Ensemble learning takes advantage of the power of multiple ML models combined together to train on the same set of observations. Therefore, ensemble models can significantly improve forecast accuracy compared to individual ML models. RF is one of the most popular decision tree-based ensemble models (Ramasubramanian & Singh 2017) due to its high forecasting accuracy (Boulesteix et al. 2012; Belgiu & Drăguţ 2016; Ouedraogo et al. 2019). RF randomly fits multiple decision trees and then takes the average of each tree's forecast as forest's forecast. RF used in our research is implemented as follows:

• 1.

  Create bootstrap samples by sampling n elements from a given training set with replacement.

• 2.

  Use the Classification And Regression Tree (CART) algorithm (Loh 2011) to train a decision tree with randomly selected features on each bootstrap sample.

• 3.

  Calculate the forecasted value of each trained decision tree on the test set and average them as the ultimate forecasted value.

The number of features (⁠⁠) and the number of decision trees (⁠⁠) are hyperparameters that need to be determined through experiments. A comprehensive number of experiments are carried out by varying the parameter values as shown Table 4.

Table 4

Tested values of RF parameters

Parameters .	Tested values .
Number of features,	{3,4,5,6,7,8,9}
Number of decision trees,	{10, 20, 30, …, 200}

Parameters .	Tested values .
Number of features,	{3,4,5,6,7,8,9}
Number of decision trees,	{10, 20, 30, …, 200}

View Large

Ridge adopts a regularized loss function to compress the linear regression coefficients resulting in reduced variance but increased bias of coefficient estimators. Compared to linear regression, this method can avoid overfitting and the ridge estimator is preferably effective at enhancing the least-squares estimate when there is multicollinearity (Arashi et al. 2019).

Next, the outputs of the first stage, i.e. the forecasted values of nine independent variables at time t + h, serve as the inputs of the second stage in Figure 5. The output of the second stage is the forecasted value of CO₂ emissions at time t + h, which is the same as the output of the single-stage procedure in Figure 4. This indicates that models used in the second stage need to fit the relationship between nine forecasted independent variables at time t + h and CO₂ emissions at the same time domain.

Briefly, the first stage of the two-stage procedure updates the value of the independent variables at time t to time t + h, and the second stage forecasts CO₂ emissions at time t + h through forecasted independent variables at time t + h.

We build a model on each training set and test it on the corresponding testing set, respectively. The final forecast error is averaged over the forecast errors of the eight testing sets.

It is clear that SVR–RF vs. RF with h = 2 is the only scenario where the two-stage procedure shows a slightly larger average forecast error than the single-stage procedure. In all other cases, the average forecast errors of the two-stage procedure are smaller than that of the single-stage procedure.

Figures 8 and 9 reveal that except for the RF and SVR–RF scenarios, the average forecast errors of all models display a slowly increasing trend as h moves up. But there is an apparent reduction in forecast error in the two-stage procedure compared with the single-stage procedure, which indicates that the two-stage forecast procedure proposed in this study indeed improves the forecasting accuracy of CO₂ emissions.

In addition, among these four two-stage models, SVR–ANN exhibits the smallest forecast error, whereas SVR–RF gives the greatest forecast error. The forecast errors of RF and SVR–RF show significant fluctuation as h increases. This can be explained by the characteristics of the RF algorithm, which consists of decision trees. According to Figures 1 and 2, four out of nine independent variables resemble the increasing trend of CO₂ emissions. Therefore, the decision tree may be randomly fitted by a single independent variable during training, which is insufficient for the forecasting task. This also implies that models involving decision trees may not be suitable for the data in this paper.

Finally, error reduction rates for the four two-stage models relative to their corresponding single-stage models ranging from 1 to 8 are averaged, respectively, resulting in Table 7. It indicates that the performance of SVR–Ridge and SVR–SVR is significantly improved than Ridge and SVR, respectively. The performance of SVR–ANN is moderately improved than ANN and SVR–RF is slightly improved than RF. This demonstrates the advantages of the proposed two-stage procedure.

In conclusion, SVR–Ridge gives the best performance improvement than Ridge, but SVR–ANN has the lowest forecast error among all models. Therefore, it is appropriate to forecast China's CO₂ emissions via SVR–ANN.

In addition to the hybrids of SVR with the four ML models, alternative hybrids, i.e. RF–SVR, RF–RF, RF–Ridge, RF–ANN; Ridge–SVR, Ridge–RF, Ridge–Ridge, Ridge–ANN; and ANN–SVR, ANN–RF, ANN–Ridge, ANN–ANN, are also tested in the study. The resulting figures of RMSE, MAE, and reduced rates are provided in the Supplementary Material. It is observed that the two-stage models all have better prediction performance than their corresponding single-stage models.

Nine relevant independent variables have been adopted to forecast production-based CO₂ emissions in China. In the literature, it is found that some studies may not train their models on lagged data sets, and some train models on lagged data sets but ignore the forecast errors caused by the time gap, h, in the dependent variable. We develop a novel two-stage procedure based on ML models in order to reduce the forecast errors identified. The hybrids of ML models in the two-stage procedure are compared against the corresponding individual models in the single-stage procedure.

Our experiments point out that the average forecast errors of the four two-stage methods, SVR–SVR, SVR–RF, SVR–Ridge, and SVR–ANN, are almost consistently smaller than their single-stage versions. Other combinations of SVR, RF, Ridge, and ANN are also compared against the corresponding single-stage procedures. It is observed that our two-stage procedure can improve forecasting performance. An accurate forecast of CO₂ emissions can help policymakers to schedule carbon emission reduction plans more efficiently and promote the realization of the carbon peaking and carbon neutrality goals.

The window width used in the first stage of the two-stage procedure is fixed as 3. It may be worth trying more possibilities to give more accurate forecasts. Since ML models have the advantages of handling big data with many inputs, another direction for future study is to consider more independent variables that affect CO₂ emissions, such as consumer price index and environmental policy stringency (Ahmed & Ahmed 2018).

Finally, the two-stage procedure is also suitable for other forecast tasks. For example, it is possible to forecast the consumption of different types of fossil fuels and renewable energy, which are listed before.

All relevant data are available from an online repository or repositories.

The authors declare there is no conflict.