Assessment of the performance of CMIP5 and CORDEX-SA models over the drought-prone Bundelkhand region, India

Coupled Model Intercomparison Project Phase 5 (CMIP5) and the further upgraded CMIP6 general circulation models (GCMs) under the World Climate Research Program (WCRP), Geneva, are now in use worldwide. These models have more accuracy compared to old GCMs to predict future climate variables. Coordinated Regional Climate Downscaling Experiment (CORDEX) is the latest platform providing a structure for the evaluation and comparison of downscaling model performance and establishing a set of measures to generate climate predictions for use in impact studies on a regional scale. WCRP CMIP5 GCM outputs are driving the CORDEX climate change experiments.

Many studies have been carried out regarding the applicability of recent GCMs and regional climate models (RCMs) in the Indian climate and have concluded the best models based on their observed historical data. Mishra et al. (2014) have taken GCMs of CMIP5 and compared them with the data of CORDEX RCMs. The study concluded that, statistically, bias-corrected GCMs impart better results than the CORDEX RCMs for most parts of India. Some of CORDEX RCMs provided a better picture of future data. Choudhary et al. (2018) also concluded the best five CORDEX RCMs for India and two from the five best models are the same as those found to be the best in Mishra et al.'s. (2014) study. Both studies also concluded that the data of any platform could be used after suitable methods of bias correction. Similarly, Singh et al. (2017) and Mishra et al. (2018) also suggested applying the bias correction approaches before using the CMIP5 GCMs and CORDEX RCMs data for India. Bias correction methods like linear scaling (LS), general quantile mapping (GEQM), gamma quantile mapping (GAQM), and power transmission (PT) could be applied to bias-correct the model data (Homsi et al. 2020).

Most of the studies were unable to conclude a clear picture of climate models in the central part of the country. Thus, the best of five CMIP5 GCMs and corresponding CORDEX RCMs have been chosen from the literature survey for India and applied to the Bundelkhand region in central India. The quantile mapping (QM) method has been used to bias-correct the modeled daily data based on the distribution function approach, which efficiently removes historical biases comparative to the observations (Themeßl et al. 2012).

The Bundelkhand region covers 13 districts in the states of Madhya Pradesh (MP) and Uttar Pradesh (UP) in central India and lies between 23°08′ N to 26°30′ N latitude and 78°11′ E to 81°30′ E longitude. This region is one of the severe drought-prone areas in central India with more significant climate variability. The location map of the Bundelkhand region is shown in Figure 1.

The total area of the Bundelkhand region is 71,619 km². The total population of Bundelkhand is 18.3 million (Gupta et al. 2014), and 82% of the total population depends on rain-fed agriculture.

Observed daily precipitation data for 35 years (1971–2005) have been collected from the Indian Meteorological Department (IMD) for 82 rain gauge stations of Bundelkhand. Historical daily precipitation data from 1971 to 2005 have been chosen to evaluate the performance of the models. Data of the five best CMIP5 GCMs were collected from the Earth System Grid Federation (ESGF) (https://esgf-index1.ceda.ac.uk/search/cmip5-ceda and https://cera-www.dkrz.de/WDCC/ui/cerasearch/), and data of the CORDEX South Asia RCMs were obtained from the Center for Climate Change Research (CCCR), Indian Institute of Tropical Meteorology (IITM) (http://cccr.tropmet.res.in/home/data_portals.jsp). Table 1 shows a description of CMIP5-GCM and CORDEX-RCM models used in this study.

Data from the five best CORDEX RCMs and their driving CMIP5 GCMs is collected for the Bundelkhand region of central India. Historical daily precipitation data from 1971 to 2005 have been chosen to evaluate the performance of the models using the observed daily precipitation data obtained from the IMD for the region. The flow chart of the methodology is represented in Figure 2.

The following steps are involved in achieving the objectives of this study.

The grid sizes of GCMs and RCMs data consist of different resolutions. Hence, before using these data, it is necessary to prepare them and rectify errors by using bias corrections and regridding/remapping processes. The following steps are taken into consideration while preparing the projected data.

Models’ data have been checked for technical and numerical bugs. Metadata integrity was also noted while reviewing the data.

This study deals with half-degree resolution datasets. Thus, it requires spatial interpolation of modeled data on a reference grid. Regridding has been done by Climate Data Operators (CDO) from the Max Planck Institute, which gathers various algorithms for interpolation used by the scientific community. Thus, daily rainfall has been remapped at 0.5° × 0.5° resolution using bilinear interpolation, which is easy to program and apply when the source and destination grids are rectilinear.

CORDEX RCM models and their driving GCMs have skills in simulating future climate, but systematic biases have been shown when statistically compared to climatological observations. There are many types of approaches to adjust climate model outputs such as linear scaling, delta methods, quantile mapping method, distribution mapping method, etc. The data are bias-adjusted by the quantile mapping method using observed daily precipitation.

The Nash–Sutcliffe efficiency ranges from ⁠. One refers to the perfect match of the modeled to the observed data. Efficiency less than 0 means that the prediction or simulations are not reliable and the observed mean is more accurate. The performance rating table (Moriasi et al. 2007) is shown in Table 2.

Observed daily precipitation data of 35 years (1971–2005) are used to examine the performance of the best five CORDEX-RCMs and their driving GCMs for the Bundelkhand region. The performance has been checked based on the mean annual precipitation (MAP), mean seasonal precipitation (MSP) for the months of June, July, August, and September (JJAS) and mean non-seasonal precipitation (MNSP). Arithmetic and weighted average of both the RCMs and their driving GCMs have been evaluated to see their ensembles. The ArcGIS tool is utilized to spatially represent the data by the kriging interpolation. The weighted average is calculated based on the results of their skill score test performed to examine the accuracy of the models.

The results of the MAP showed a satisfactory bias for all of the models. Bias-corrected GCM, MPI-ESM-MR indicated the best correlation between observed and modeled annual mean, as shown in Figure 3. Based on NSE and r² performance indicators, MAP results were suitable for all of the models for future climate projections.

NSE and r² were performed to see the correlation of modeled MAP, MSP, and MNSP results with their observed ones. r² could not discriminate between the accuracy of the models, while NSE indicated better results to find the best of all the models. Results of NSE showed that the bias-corrected GCMs performed well as compared to the CORDEX RCMs (Table 3). GCM-EC-EARTH and GCM-GFDL-ESM-2M were found closest to the mean observation.

MSP results indicated a better picture of model performance. Only the model, EC-Earth (CORDEX, as well as its driving GCM), showed the best correlation with the observed precipitation. The spatial distribution of all the models, along with the observed, is given in Figure 4. CORDEX-RCMs (except EC-Earth) failed to claim satisfactory performance. Some bias-corrected GCMs found better correlation in the upper Bundelkhand region (ACCESS1-0, GFDL-ESM-2M, and MPI-ESM-MR). At the same time, the ensemble mean of MSP also showed the least bias for bias-corrected GCMs where RCMs did not perform well. The weightage of the weighted ensemble of GCMs and RCMs were distributed based on their skill score test.

No relevant relationship was found for MNSP results with observed data for any of the climate models. Some parts of the lower Bundelkhand showed a relationship for GCM and RCM of EC-Earth, and some for the GCM of MPI-ESM-MR, which can be seen in Figure 5. Ensemble results of MNSP also failed to claim any satisfactory relationship with the observed data.

There was minimal variation found in MAP results. Most of the models were found closer to the mean observed precipitation. It is necessary to find the most suitable of the selected models over the Bundelkhand region in central India. Thus, the skill score method was chosen to see the model performance, as discussed in the Methodology section. SS was applied for all the bias-corrected CORDEX RCMs and their driving GCMs for MAP, MSP, and MNSP scales.

A combined skill score showed a wide variation among all the models. On the MSP scale, bias-corrected GCM, GFDL-ESM-2M was found to be the best model showing the highest SS, while the lowest value of skill was seen for the RCM of GFDL-ESM-2M, which is shown in Table 4. Model GFDL-ESM-2M of bias-corrected GCM claimed the best skill score for all the timescales (MAP, MSP, and MNSP). The reduction in skill indicated the significant deviation of the model from the ensemble. Hence, the bias-corrected GCM, GFDL-ESM-2M experiment was found closest to the mean observation line with a highest ‘performance’ skill of 19.17, a ‘convergence’ skill of 33.29, and a combined skill of 407441.04.

The upper part of Bundelkhand is more affected by severe drought than its lower part due to the consistent climate variation between them. Thus, a separate skill test was also performed for the upper part (part of UP) and the lower part (part of MP) of Bundelkhand. The bias-corrected GCM, GFDL-ESM-2M experiment was found closest to the mean observation line for both parts of Bundelkhand in central India. The highest values of combined skill for GCM, GFDL-ESM-2M were found for the upper part (2078676.08) as well as for the lower part (1399846.65) of the Bundelkhand, which can be seen in Figure 6.

Table 5 demonstrates the SS on the MNSP scale for all of Bundelkhand. Of all the bias-corrected CORDEX RCMs and their driving GCMs, GFDL-ESM-2M of bias-corrected GCM performed best with the combined score of 26.06.

The mean seasonal precipitation represented better climate accuracy of the climate models based on rainfall observations on annual, seasonal, and non-seasonal timescales. On the mean non-seasonal scale, results showed poor performance, while on the annual scale, most of the models performed with better accuracy. CORDEX RCMs demonstrated poor performance as compared to their driving bias-corrected GCM results. Models EC-EARTH and GFDL-ESM-2M of bias-corrected GCMs showed the minimum bias and best correlation for MSP as well as for MAP scale, and with some accuracy for the MNSP scale. Based on the combined skill score test, model GFDL-ESM-2M of bias-corrected GCM was claimed as the best experimental model to predict future precipitation over the Bundelkhand region in central India. It is noticed that bias-corrected GCMs depicted a better climate than the CORDEX RCMs. The study revealed that from the five best RCMs and their driving GCMs, bias-corrected GCM-GFDL-ESM-2M could be utilized to predict the future scenario of precipitation in the Bundelkhand region in central India. It is also noticed that the accuracy of climate models for precipitation could be best judged based on their mean monsoon observations.

The focus of the current study was to check the performance of climate models in signifying precipitation for a central region of India. Further, various hydrological indicators like stream flow, drought, and evaporation could be evaluated to discover their behavior in the study region (Wu & Chau 2013; Taormina & Chau 2015; Ali Ghorbani et al. 2018). Drought could be evaluated using specific machine learning approaches or satellite-based monitoring (Zhao et al. 2018; Shamshirband et al. 2020) to forecast future water crisis in the drought-prone region of Bundelkhand, India.

All relevant data are included in the paper or its Supplementary Information.