Evaluation of global climate models for the simulation of precipitation and maximum and minimum temperatures at coarser and finer resolutions based on temporal and spatial assessment metrics in mainland of China

The Intergovernmental Panel on Climate Change (IPCC) states that human activity has caused global warming of 1.0 °C over the past 100 years and is likely to reach 1.5 °C between 2030 and 2052 (Masson-Delmotte et al. 2018). Climatic extremes such as temperature and precipitation extremes are sensitive to climate warming (Fischer et al. 2014; Ji & Kang 2015; Kharin et al. 2018; Lorenz et al. 2019), and these extremes are often associated with profound effects on ecosystems (Terando et al. 2012; Wheeler & Von Braun 2013; Gu et al. 2020; Zhang et al. 2022). It is necessary to get reliable future climate change information to mitigate the adverse impacts of climate change.

The state-of-the-art GCM simulations from Coupled Model Inter-comparison Project Phase 6 (CMIP6) are expected to improve the representation of the Earth's climate system developed by different institutions around the world (Eyring et al. 2016). The quantification of climate change impacts is often achieved by using global climate model (GCM) simulations with downscaling or bias correction techniques (Cook et al. 2020; Hirabayashi et al. 2021; Xu et al. 2021; Adib & Harun 2022).

Some studies evaluated the performance of variables in GCMs according to their performance over historical periods, using various methods such as the reliability ensemble averaging method (Giorgi & Mearns 2003), relative entropy (Shukla et al. 2006), Bayesian approach (Min & Hense 2006), probability density function (Perkins et al. 2007), hierarchical ANOVA models (Sansom et al. 2013), clustering (Knutti et al. 2013), correlation (Jiang et al. 2015; Xuan et al. 2017), symmetrical uncertainty (Salman et al. 2018), multiple spatial metrics (Ahmed et al. 2019) and Taylor diagram (Ngoma et al. 2020). Furthermore, the performance of GCMs is evaluated at different temporal scales, daily (Perkins et al. 2007), monthly (Srinivasa Raju et al. 2017), seasonal (Ngoma et al. 2020) and annual scales (Murphy et al. 2004). In addition to temporal scales, a number of studies ranked GCMs re-gridded based on spatial average or all the grid covering the study area (Salman et al. 2018; Abbasian et al. 2019).

Ojha et al. (2014) applied a variable convergence score to evaluate 10 atmospheric variables associated with downscaling precipitation and found higher consistency across GCMs for pressure and temperature, and lower consistency for precipitation and related variables in India. Abbasian et al. (2019) used performance criteria including mean deviation, RMSE, NSE, r, Kolmogorov–Smirnov (KS) statistic, Sen's slope estimator and Taylor diagram to evaluate precipitation and temperature, and found that most GCMs perform well in simulating the annual and seasonal temperature but poorly in simulating precipitation, especially at the seasonal scale over Iran. Cui et al. (2021) re-gridded daily temperature and precipitation for the period 1961–2012 in 29 GCMs from CMIP6 with bilinear interpolation and found good performance in simulating the sign of the trends in extreme indices but underestimated their magnitudes and misrepresent spatial patterns. Yang et al. (2021) compared temperature and precipitation in 20 GCMs from CMIP6 with gridded observation data for the period 1995–2014 and found that CMIP6 models show a good ability to capture the climatological distributions of temperature and precipitation, with better performance for temperature than precipitation over China.

Accordingly, this study focuses on the assessment of the performance of daily precipitation and maximum and minimum temperatures from GCMs in temporal and spatial aspects from 1961 to 2014 in mainland China. Before the application of climate variables, spatio-temporal indicators are usually used to evaluate the temperature and precipitation of climate patterns after interpolation (Abbasian et al. 2019; Ahmed et al. 2019). When climate variables are applied to the impact study of climate change, it needs to be interpolated first. The use of spatial and temporal indicators to evaluate the performance of climate variables from GCM and primary climate variables input into the hydrological model is important for the reprocessing of the climate variables output from the climate model into the hydrological model. However, seldom do studies focus on the performance of climate variables at the scale of raw GCMs. Through the spatio-temporal analysis of the climate variables before and after interpolation and after deviation correction, the performance of precipitation and temperature from the raw climate model to the input of the hydrological model was evaluated in this study. The rest of the paper is arranged as follows: Section 2 presents the study area and datasets, and Section 3 presents a brief introduction to the methodology, including the temporal and spatial metrics, interpolation and bias correction methods. Section 4 presents the results, followed by the discussion and conclusion in Section 5. The spatial and temporal evaluation methods and the evaluation results of corrected climate variables from raw, interpolated and bias-corrected GCM could be helpful in the selection of GCMs in hydrologic impact studies, and this would be applied to the study of the hydrological response in climate change.

This study used a gridded meteorological dataset (⁠⁠) over China for the period of 1961–2014. This dataset contains three climate variables, including daily precipitation, and daily minimum and maximum temperatures, which are downloaded from the China Meteorological Data Sharing Service System (http://www.cma.gov.cn/) to represent the observed data. This gridded dataset came from 2472 in situ observation gauge stations across China and was interpolated using the thin plate spline method of GTOPO30 (Global 30 Arc-Second Elevation) data sampling (Zhang et al. 2015). This dataset can well simulate the daily precipitation and temperature of the historical period in China and is often used as observational data (Wan et al. 2021; Yin et al. 2021).

Daily precipitation and maximum and minimum temperatures in 21 GCMs (Table 1) extracted from the CMIP6 data center (https://esgf-node.llnl.gov/projects/cmip6/) were selected from 1961 to 2014 in GCMs.

In this study, GCMs for daily precipitation and maximum and minimum temperatures were first ranked separately (individual ranking) using temporal and spatial performance measures, KS statistic and spatial metrics (SPAtial Efficiency (SPAEF)). The bias correction method was used for the GCMs and ranked as below. The procedure used for the research is outlined as follows:

• The resolution of observation is at finer resolution, and the observed daily precipitation and maximum and minimum temperatures for the period 1961–2014 are remapped to the original grid of the GCM-simulated resolution by the mean grid method for the observation at coarser resolution. The data from raw GCMs are interpolated to finer resolution with inverse distance weighted interpolation (Shepard 1968).

• KS statistic and spatial metrics (SPAEF) are individually applied to daily precipitation and maximum and minimum temperatures for the period 1961–2014 at both finer and coarser resolution.

• The goodness-of-fit (GOF) estimated by KS and SPAEF for daily precipitation, and maximum and minimum temperatures is used to rank the GCMs separately at both finer and coarser resolution.

• The bias correction method is used for the GCMs at finer resolution, and the ranks are used for them and the difference between GCMs in different resolution and metrics are compared.

The temporal (KS) and spatial metrics (SPAEF) were individually applied from 1961 to 2014 to daily precipitation and maximum and minimum temperatures, respectively. Later, the GOF values of each day were temporally averaged to obtain a value for the entire study area. The details of the two metrics are given in the following.

The values of SPAEF are between −∞ and 1, where a value closer to 1 indicates a higher spatial similarity between the observations and GCM simulations (Koch et al. 2018).

We ranked the GCMs at the resolution of raw GCMs (coarser resolution) and observation data (finer resolution), with the relative error in the daily precipitation and maximum and minimum temperatures.

One multivariate bias correction method (two-stage quantile mapping, TSQM), which combines a single-variable bias correction method with a distribution-free shuffle approach (Guo et al. 2019), is used to correct daily precipitation and maximum and minimum temperatures in GCMs at finer resolution. The TSQM method (Guo et al. 2019) involves two steps: firstly, the daily bias correction (DBC) method (Chen et al. 2013) is used to correct biases for the three climate variables separately, and then, a distribution-free shuffle approach is used to construct the inter-variable correlations by rearranging the sequences of bias-corrected series. Also, the DBC method involves two steps: firstly, the biases in precipitation occurrence are corrected by determining the precipitation threshold for simulated precipitation amounts, and then, the biases in temperature (or wet-day precipitation amounts) between the reference simulation and observation, and the same biases are removed for the future simulation.

SPAEF and KS between observed (cma) and GCM-simulated daily mean precipitation, maximum and minimum temperatures in China at the coarse and finer resolutions and after bias correction were estimated for the period 1961–2014. Table 2 shows the GOF values that depict the performance of each GCM in simulating cma mean daily precipitation. In Table 2, the ranks of GCMs corresponding to SPAEF and KS are shown with brackets. The GOF values near 1(0) refer to better performance of the GCM of interest in SPAEF (KS). For example, ACCESS-ESM1-5 has a GOF value of 0.979 for SPAEF and is, hence, regarded as the best GCM, whereas MIROC-ES2L can be regarded as the poorest GCM, which has a GOF value of 0.862 in terms of SPAEF. Also, CNRM-ESM2-1 is regarded as the best GCM with a GOF value close to 0, and FGOALS-g3 is the poorest one with a GOF value close to −0.002 in terms of KS at coarse resolution. The GOF values and ranks at finer resolution and that after bias correction can also be interpreted in the same manner.

Table 2 shows the ranks attained by GCMs corresponding to temporal and spatial metrics for daily precipitation at coarse and finer resolutions and those after bias correction. The daily precipitation from raw GCM and that after bias correction perform better than that of interpolated GCM, the maximum SPAEF is 0.979, 0.925 and 0.798, and KS is 0, 0 and 0.101 for raw, bias-corrected and interpolated GCMs, separately. For example, BCC-CSM2-MR attained ranks 14, 18 and 16 for SPAEF but 17, 5 and 18 for KS at coarse and finer resolutions and after bias correction. MIROC-ES2L secured rank 21 for SPAEF in both coarse and finer resolutions but 7 after bias correction. The SPAEF for GCMs at coarse resolution is the largest one, and the KS for GCMs is the smallest one in the coarse resolution for daily precipitation.

Figure 2 illustrates the ranks of daily precipitation at coarse and finer resolutions and those after bias correction based on KS. The daily precipitation after bias correction shows the best in the ECDFs, while daily precipitation in the raw GCMs and that after interpolation overestimate the precipitation. Also, the boxplots also show that the mean of daily precipitation from bias-corrected GCMs shows the best performance than that from raw and interpolated GCM.

Table 3 illustrates the ranks of daily maximum temperature at coarse and finer resolutions and those after bias correction based on the temporal and spatial metrics. The GOFs are the largest for SPAEF and the smallest for KS in raw GCMs in comparison with the GCMs interpolated and that after bias correction. For instance, CNRM-ESM2-1 has a GOF value of 0.985 for SPAEF and is hence regarded as the best GCM in terms of SPAEF in raw GCMs, whereas MRI-ESM2-0 and EC-Earth3 are the best GCMs for finer resolution and that after bias correction with the GOF values of 0.463 and 0.89. On the other hand, MIROC-ES2L, HadGEM3-GC31-LL and INM-CM5-0 can be regarded as the poorest GCM, which have GOF values of 0.764, −4.146 and −0.181, respectively, in terms of SPAEF in coarse and finer resolutions and those after bias correction. The GOF values and ranks for KS can also be interpreted in the same manner. The SPAEF for GCMs at coarse resolution is the largest one for daily maximum temperature like precipitation, but the KS for GCMs interpolated but before bias correction is the smallest one for daily maximum temperature.

Table 4 illustrates the ranks of daily minimum temperature at coarse and finer resolutions and those after bias correction based on the temporal and spatial metrics. The GOFs are the largest for SPAEF and the smallest for KS in raw GCMs in comparison with the GCMs interpolated and that after bias correction. For instance, CNRM-ESM2-1 has a GOF value of 0.985 for SPAEF and is hence regarded as the best GCM in terms of SPAEF in raw GCMs, whereas MRI-ESM2-0 and EC-Earth3 are the best GCMs for finer resolution and that after bias correction with the GOF value of 0.463 and 0.89. On the other hand, MIROC-ES2L, HadGEM3-GC31-LL and INM-CM5-0 can be regarded as the poorest GCM, which have GOF values of 0.764, −4.146 and −0.181 in terms of SPAEF in coarse and finer resolutions and those after bias correction, whereas BCC-CSM2-MR, ACCESS-CM2 and EC-Earth3 are the best ones and HadGEM3-GC31-LL, IPSL-CM6A-LR and INM-CM5-0 are the poorest ones for coarse and finer resolutions and that after bias correction in KS. The SPAEF for GCMs at coarse resolution is the largest one, but the KS for GCMs interpolated but before bias correction is the smallest one for daily minimum temperature like daily maximum temperature.

This study not only studied the spatio-temporal performance of climatic variables after interpolation and bias correction but also compared the performance of climate variables of raw GCM, so as to illustrate the impact of reprocessing. It assessed the temporal and spatial accuracy of 21 CMIP6 GCMs in simulating daily precipitation, minimum and maximum temperature at coarse and finer resolutions and those after bias correction over China for the period 1961–2014. The GOF of KS is calculated for the temporal assessment metric, and SPAEF is used for the metric assessment metric. Ahmed et al. (2019) and Abbasian et al. (2019) have evaluated the precipitation and temperature from annual and seasonal scales after interpolation, but researchers seldom studied the performance in daily scale and compared the performance before and after interpolation. This work is important for hydro-climate research for their input is daily scale. The following conclusions were drawn from this study:

• The GOF of SPAEF in raw resolution shows the greatest at coarser resolution, which mean that daily precipitation, minimum and maximum temperature in raw GCMs have the highest similarity to those in re-gridded observational cma, especially in the spatial pattern. The inverse distance weighted interpolation may decrease the accuracy spatial aspects for all the three variables and temporal in precipitation but not for daily maximum and minimum temperatures.

• Almost all the GCMs overestimate the daily precipitation at both coarse and finer resolutions, but closer at finer resolution after bias correction. The performance of GCMs in daily minimum and maximum temperatures shows better than that in daily precipitation, where GCMs show a difference in temporal and spatial assessment metrics. It is of great importance to select the GCM before application with different situations.

• The GCMs after bias correction at finer resolution have the best similarity to the observation in spatial patterns but the poorest similarity in the trend of ECDFs in daily minimum and maximum temperatures. EC-Earth3-Veg (UKESM1-0-LL), EC-Earth3 (EC-Earth3) and MPI-ESM1-2-LR (CNRM-ESM2-1) is the best GCM in SPAEF (KS) at finer resolution after bias correction.

Furthermore, the process of inverse distance weighted interpolation will bring more uncertainty to variables from GCMs, especially in spatial aspects. Also, the TSQM method performs well in representing spatio-temporal precipitation and spatial temperature, but poorly in temporal temperature.

This work was partially supported by the National Natural Science Foundation of China (Grant No. 52109007), the Natural Science Foundation of Chongqing, China (Grant No. cstc2021jcyj-msxm2426) and the Scientific and Technological Research Program of Chongqing Municipal Education Commission (Grant No. KJQN2021007).

Y.L. conceived the original idea, designed the methodology, collected the data, performed the simulations, contributed to the interpretation of the results, and wrote and revised the paper. P.P. reviewed and edited the paper. W.J. reviewed and edited the paper.

All relevant data are included in the paper or its Supplementary Information.

The authors declare that there is no conflict.