Global climate models (GCMs) are the state-of-the-art tool for understanding climate change and predicting future. However, little research has been reported on the latest NEX-GDDP-CMIP6 product in China. The purpose of this study was to evaluate the simulated performance and drought capture utility of the NEX-GDDP-CMIP6 over China. First, the simulation skills of the 16 GCMs in NEX-GDDP-CMIP6 was evaluated by the 'DISO', a big data evaluation method. Second, the DISO framework for drought identification was constructed by coupling the Correlation Coefficient (CC), False Alarm Rate (FAR) and Probability of Detection (POD). Then, it was combined with SPI and SPEI to evaluate the drought detection capability of NEX-GDPD-CMIP6. The result shows that: (1) NEX-GDPD-CMIP6 can reproduce the spatial distribution pattern of historical precipitation and temperature, which performs well in simulating warming trend but fails to capture precipitation's fluctuation characteristics. (2) The best performing model in precipitation is ACCESS-CM2 (DISO 1.630) and in temperature is CESM2 (DISO 3.246). (3) The 16MME performs better than the best single model, indicating that multi-model ensemble can effectively reduce the uncertainty inherent in models. (4) The SPEI calculated by 16MME identifying drought well in arid, while SPI is recommended for other climate classifications of China.

  • Sixteen multi-mode ensembles outperform the best-performing single climate model.

  • Enlighteningly, CC, FAR and POD are coupled to construct a DISO framework for assessing drought identification capabilities.

  • The drought identification ability between the SPEI and the SPI calculated from NEX-GDDP-CMIP6 over China varies greatly, with the SPEI performing well in an arid region and inferior to the SPI over other climate classifications.

The globe has been warming rapidly in recent decades. Along with climate change, droughts caused by extreme weather are widely occurring over the world. The community is increasingly concerned about climate change research (Yuan et al. 2019). Global climate models (GCMs) are complex Earth system models, which can simulate atmospheric, oceanic, land surface and carbon cycle changes in the climate system (Nashwan & Shahid 2020). The GCMs play an important role in reproducing climate change processes, revealing evolutionary mechanisms and making future climate projections (Yang et al. 2018).

The World Climate Research Program (WCRP) launched the Coupled Model Intercomparison Project (CMIP) (Abro et al. 2020) to address climate change. CMIP6 is the latest version of the CMIPs, and numerous studies have confirmed the ability of CMIP6 to reproduce climate change, but the coarser spatial resolution limits its application to some extent (Eyring et al. 2016; Yang et al. 2020; Chen & Yuan 2021). To portray the impacts of climate change in a more granular way, the National Aeronautics and Space Administration (NASA) implemented the NEX-GDDP-CMIP6 (NASA Earth Exchange Global Daily Downscaled Projections 6) (Thrasher et al. 2022). To our knowledge, few relevant evaluation studies for NEX-GDDP-CMIP6 GCMs in China have been found. In order to enhance confidence in future climate projections, a comprehensive assessment of NEX-GDDP-CMIP6's performance in reproducing historical climate change is necessary.

Whether the spatiotemporal characteristics of the GCM are similar to the observations is the first question to be determined (Khan et al. 2018). Various statistical metrics are used to describe the performance of GCMs. However, single statistical metrics cannot reflect GCM performance from multiple perspectives. In order to reveal the comprehensive performance of GCMs, Hu et al. (2019) proposed DISO (Distance between Indices of Simulation and Observation), which assesses the performance of GCMs in 3D perspective by measuring the spatial distance between GCMs and observations (Hu et al. 2019; Zhou et al. 2021; Lei et al. 2022). DISO overcomes some disadvantages of Taylor diagrams and provides an intuitive way to measure differences between various GCMs in the same assessment system. This paper intends to evaluate the simulation performance of NEX-GDDP-CMIP6 over China by DISO.

The ability of GCMs to reproduce the spatial and temporal characteristics of climate change is important, but their accuracy in identifying various hazards should not be neglected, especially in detecting droughts (Wei et al. 2019). Drought is widely recognized as one of the most severe and pervasive disasters in the world, with devastating effects on agriculture, ecology, water security and society. Therefore, precise monitoring of drought is crucial to ensure ecological security (Wei et al. 2019; Yuan et al. 2019). Previous studies were mostly limited to the meteorological simulation performance of climate products (Li et al. 2021), and only a few scholars focused on it to detect drought ability. Wei et al. (2019) evaluated the drought monitoring utility of IMERG satellite precipitation products by correlation coefficient (CC), false alarm rate (FAR) and probability of detection (POD). Hao & AghaKouchak (2014) evaluated the utility of the United States Drought Monitor Data (USDM) using the Multivariate Standardized Drought Index (MSDI). The above studies only take a single perspective and the comprehensive drought identification performance of climate products has not been quantified. We heuristically couple CC, FAR and POD to establish an integrated drought identification DISO framework to measure the drought capture performance of the NEX-GDDP-CMIP6 over China.

This paper intends to address the following questions: (1) Can NEX-GDDP-CMIP6 better reproduce the spatial distribution patterns and temporal evolution trends of historical precipitation and temperature over China? (2) How well does the NEX-GDDP-CMIP6 simulate precipitation and temperature over China? (3) Does NEX-GDDP-CMIP6 have powerful drought capture capability over China?

The continent of China is vast and spans multiple latitudes. Its unique geographical location and complex topography create significant differences in climatic characteristics. Therefore, we adopted the approach of Fu et al. (2019) and divided the study area into four different climatic zones – arid (AR), semi-arid (SAR), semi-humid (SHD) and humid (HD), based on the 200, 400 and 800 mm annual precipitation contours that serve as cutoff thresholds (Figure 1). Due to limited observed data availability, Taiwan Province was not included in the study.
Figure 1

Four climate classifications and annual average precipitation (mm) in China.

Figure 1

Four climate classifications and annual average precipitation (mm) in China.

Close modal

NEX-GDDP-CMIP6 (https://portal.nccs.nasa.gov/datashare/nexgddp_cmip6/) is the latest version of the NASA Earth Exchange Global Daily Downscaled Projections. It is based on the output of CMIP6 with downscaling and bias correction to generate a high-resolution dataset covering the historical and future periods. NEX-GDDP-CMIP6 provides a finer spatial resolution (0.25° × 0.25°), which can more accurately reflect climate change, and more details can be found in Thrasher et al. (2022). NEX-GDDP-CMIP6 overcomes some shortcomings of NEX-GDDP-CMIP5 and can better reflect the climate change process, but its performance in China has not been evaluated. Therefore, we selected historical (1961–2014) monthly precipitation and temperature data for 16 GCMs in NEX-GDDP-CMIP6 with the aim of assessing their simulated performance over China (Table 1).

Table 1

Information on NEX-GDDP-CMIP6 climate models

No.Source IDInstitutionCountryGrid (latitude × longitude)
ACCESS-CM2 CSIRO-ARCCSS Australia 0.25° × 0.25° 
ACCESS-ESM1-5 CSIRO Australia 0.25° × 0.25° 
BCC-CSM2-MR BCC China 0.25° × 0.25° 
CESM2 NCAR United States 0.25° × 0.25° 
CESM2-WACCM NCAR United States 0.25° × 0.25° 
CMCC-CM2-SR5 CMCC Italy 0.25° × 0.25° 
CMCC-ESM2 CMCC Italy 0.25° × 0.25° 
CNRM-CM6-1 CNRM-CERFACS France 0.25° × 0.25° 
IITM-ESM CCCR-IITM India 0.25° × 0.25° 
10 MIROC6 MIROC Japan 0.25° × 0.25° 
11 MPI-ESM1-2-HR MRI-M DWD DKRZ Germany 0.25° × 0.25° 
12 MPI-ESM1-2-LR MRI-M AWI DKRZ Germany 0.25° × 0.25° 
13 MRI-ESM2-0 MRI Japan 0.25° × 0.25° 
14 NorESM2-LM NCC Norway 0.25° × 0.25° 
15 NorESM2-MM NCC Norway 0.25° × 0.25° 
16 TaiESM1 RCEC-AS Taiwan, China 0.25° × 0.25° 
No.Source IDInstitutionCountryGrid (latitude × longitude)
ACCESS-CM2 CSIRO-ARCCSS Australia 0.25° × 0.25° 
ACCESS-ESM1-5 CSIRO Australia 0.25° × 0.25° 
BCC-CSM2-MR BCC China 0.25° × 0.25° 
CESM2 NCAR United States 0.25° × 0.25° 
CESM2-WACCM NCAR United States 0.25° × 0.25° 
CMCC-CM2-SR5 CMCC Italy 0.25° × 0.25° 
CMCC-ESM2 CMCC Italy 0.25° × 0.25° 
CNRM-CM6-1 CNRM-CERFACS France 0.25° × 0.25° 
IITM-ESM CCCR-IITM India 0.25° × 0.25° 
10 MIROC6 MIROC Japan 0.25° × 0.25° 
11 MPI-ESM1-2-HR MRI-M DWD DKRZ Germany 0.25° × 0.25° 
12 MPI-ESM1-2-LR MRI-M AWI DKRZ Germany 0.25° × 0.25° 
13 MRI-ESM2-0 MRI Japan 0.25° × 0.25° 
14 NorESM2-LM NCC Norway 0.25° × 0.25° 
15 NorESM2-MM NCC Norway 0.25° × 0.25° 
16 TaiESM1 RCEC-AS Taiwan, China 0.25° × 0.25° 

Precipitation and temperature observations (1961–2014) of CN05.1 (Wu & Gao 2013) were selected as references to compare and validate the output of NEX-GDDP-CMIP6. The CN05.1 dataset was obtained based on the interpolation of 2,416 meteorological station observations and has been widely used as an evaluation reference for GCMs output (Xin et al. 2020). CN05.1 has the same spatial resolution as NEX-GDDP-CMIP6, which makes the analysis and comparison possible.

Climate model evaluation

We chose the conventional CC, relative bias (BIAS), mean absolute error (MAE) and root mean square error (RMSE) to quantitatively evaluate the error characteristics of NEX-GDDP-CMIP6. The optimal value of CC is 1, and the rest of the indicators are 0. Among them, the CC portrays the linear fit degree between the evaluation object (NEX-GDDP-CMIP6) and observation, the BIAS describes the error, the MAE reflects the dispersion and the RMSE measures the systematic deviation. These indicators are calculated by the following formula:

CC:
(1)
BIAS:
(2)
MAE:
(3)
RMSE:
(4)
where n is the length of the data time series; and are the values of the observation and the evaluated objects at time i, respectively; and and are their average values on the time series.

Drought index

Standardized Precipitation Index

The Standardized Precipitation Index (SPI) (Mckee et al. 1993) is a dimensionless meteorological drought index. It considers only precipitation elements and normalizes precipitation at a specific time by probability distribution function (e.g., gamma distribution) to measure drought degree (Faiz et al. 2018). The SPI is easy to calculate on the one hand and on the other, it is flexible in time scale. The SPI can describe cumulative drought conditions by identifying precipitation deficits at different time scales. In addition, the SPI is spatially comparable and has been widely used for drought identification over the world. Since the gamma distribution fits seasonal precipitation well and outperforms other probability distributions (e.g., the normal or lognormal distribution), it can estimate drought conditions more accurately. Therefore, this paper uses the gamma distribution to estimate the SPI, and the specific calculation process of the SPI is as in the following equations:
(5)
where x is the precipitation in a certain time period; Γ is the gamma distribution function; and α and β are the shape and scale parameters of Γ function.
(6)
where F(x) is the cumulative probability distribution of precipitation in a certain period.
Normalize the cumulative probability distribution F(x) to obtain the SPI:
(7)
(8)
where c0 = 2.515517, c1 = 0.802853, d1 = 1.432788, d2 = 0.189269 and d3 = 0.001308.

Standardized Precipitation Evaporation Index

Variations in temperature largely influence drought, and the Standardized Precipitation Evaporation Index (SPEI) is another suitable choice for drought detection because it contains information on potential evapotranspiration. The SPEI was proposed by Vicente-Serrano et al. (2010), which considers the effect of potential evapotranspiration on drought in the context of global warming. The SPEI adds an evapotranspiration component to describe the water balance and replaces precipitation with the difference between precipitation and potential evapotranspiration as an input variable (Faiz et al. 2020). Since the SPEI calculation process has the addition of the evapotranspiration component, which compensates the disadvantage of the singularity of meteorological variables in the SPI calculation, it has been widely used in climate change research in recent years. For the specific calculation process, refer to Vicente-Serrano et al. (2010).

Drought detection capability assessment

To assess the performance of NEX-GDDP-CMIP6 in capturing drought occurrence, the POD and FAR (Gourley et al. 2012) were applied in addition to the traditional CC. The POD and FAR range from 0 to 1, with optimal values of 1 and 0, respectively. The POD indicates the ratio of drought events correctly monitored by the assessment object (drought index calculated by NEX-GDDP-CMIP6) to those monitored by the reference (drought index calculated by CN05.1). The FAR represents the ratio of drought events incorrectly monitored by the assessment object to those monitored by the reference. These metrics are calculated by the following equation:
(9)
(10)
where H denotes the number of drought events detected simultaneously in the reference and assessment subjects. M indicates the number in which drought occurrences are detected in the reference and not detected in the assessment subject. F indicates the number of droughts that were not detected in the reference but were detected in the assessment object.

Distance between Indices of Simulation and Observation (DISO)

Taylor diagrams (Taylor 2001) are the most common way to assess the performance of climate products. However, it inevitably has inherent drawbacks: (1) statistical indicators are only available in 2D space and (2) the comprehensive performance of the model cannot be quantified (Hu et al. 2019; Zhou et al. 2021). The DISO (Hu et al. 2019) algorithm was designed to overcome the drawbacks that existed in Taylor diagrams. First, DISO has a higher dimensionality than Taylor diagrams, providing a three-dimensional view to comprehensively assess model performance. Second, DISO provides a quantitative measure of model performance that is easy to interpret and can be used to quantify and compare the performance of different models. In addition, DISO can evaluate the performance of a model on multiple metrics at the same time, which can reflect the performance of the model in different aspects. Figure 2 provides a schematic representation of the DISO principle: A, B and C (usually CC, MAE and RMSE) form the X, Y and Z axes of the 3D coordinate system. Assume that the assessment subject is Si, whose statistical indicators are xi, yi and zi. Since the observation (OBS) compares with itself, resulting in CC = 1, MAE = 0 and RMSE = 0, the position of the OBS in the 3D coordinate system is (1,0,0). Placing the assessment object in the same spatial coordinate system with its position Si(xi, yi, zi), the 3D distance from Si to OBS is denoted as DISOi, and the shorter the spatial distance, the better the assessment object performed. It should be noted that DISO is a dimensionless index, which is calculated as in the following equation:
(11)
Figure 2

Schematic diagram of DISO.

Figure 2

Schematic diagram of DISO.

Close modal
The ability of GCMs to reproduce precipitation and temperature is the first problem to be explored. Another essential part of the evaluation process is whether NEX-GDDP-CMIP6 can effectively identify drought. We coupled CC, MAE and RMSE to construct a DISO for the GCMs performance assessment, as shown in Equation (12). Furthermore, Zhou et al. (2021) noted that the DISO assessment system applies to any discipline in the natural sciences (e.g., mathematics, biology and chemistry) and social sciences (e.g., sociology, psychology and economics). Therefore, this paper inspiringly extends it to the assessment of drought identification capability. Coupling CC, FAR and POD constructs the DISO for drought capture capability assessment, as shown in Equation (13):
(12)
(13)

Spatial and temporal characteristics of CMIP6 high-resolution data

Yang et al. (2020) showed that an MME can effectively improve the simulation performance of climate models. Therefore, 16MME of NEX-GDDP-CMIP6 (16MME, calculated from 16 climate models in NEX-GDDP-CMIP6 by the weighted average method) is included in the evaluation of this study. Figure 3(a) shows the mean annual precipitation time patterns of each model, 16MME and CN05.1 observations over China. During the period 1961–2014, the precipitation of NEX-GDDP-CMIP6 differed significantly, with all models underestimating precipitation to varying degrees. 16MME fails to reproduce the precipitation fluctuation trend and underestimates the mean annual precipitation by 40 mm. Despite the bias, the fluctuation range of the observed series (thick red line) is almost within the value interval of 16 models (the shaded area), indicating that the precipitation output of NEX-GDDP-CMIP6 is not significantly different from the observation and the error is within an acceptable range.
Figure 3

Trend lines for mean annual precipitation and temperature (1961–2014) for China. Shaded areas show the range of models, trend lines for models with one simulation are shown as thin lines, multi-model ensembles (thick blue lines) and observations (thick red lines). Please refer to the online version of this paper to see this figure in colour: http://dx.doi.org/10.2166/nh.2023.140.

Figure 3

Trend lines for mean annual precipitation and temperature (1961–2014) for China. Shaded areas show the range of models, trend lines for models with one simulation are shown as thin lines, multi-model ensembles (thick blue lines) and observations (thick red lines). Please refer to the online version of this paper to see this figure in colour: http://dx.doi.org/10.2166/nh.2023.140.

Close modal

The mean temperature evolution trends over time are broadly consistent for all models (including 16MME), capturing the variability and general trends observed in the historical period (Figure 3(b)). The simulated temperature magnitudes do not differ significantly among models but all overestimate the temperature. 16MME overestimates the annual mean temperature by 0.5–1.5 °C. The simulations of both precipitation and temperature are biased, and the mechanism to improve this error remains a challenge.

CN05.1 precipitation decreases from the southeast to the northwest (Figure 4(a)), showing significant climatic classification characteristics. 16MME well reproduces the spatial distribution pattern of monthly precipitation overall, but inevitably there were deviations in local areas (e.g., the Himalayas) (Figure 4(b)). 16MME generally underestimates monthly precipitation over China, while a considerable wet bias in central Xinjiang and southwestern Tibet can be larger than 300% (Figure 4(c)). The 16MME exhibits relatively low bias in the HD and SHD (ranging from −25 to 5%) but suffers from significant deviations (ranging from −50 to 30%) in the AR and SAR (Figure 4(d)). The relatively limited performance of 16MME in simulating precipitation patterns on the Qinghai–Tibet Plateau and Xinjiang may be related to the complex climate and topography.
Figure 4

Spatial distribution of observed (a) and 16MME (b) monthly mean precipitation in China and percentage bias (c, d) between them.

Figure 4

Spatial distribution of observed (a) and 16MME (b) monthly mean precipitation in China and percentage bias (c, d) between them.

Close modal
For monthly temperatures, which show a spatial distribution pattern mainly related to altitude and latitude, i.e., the average temperature decreases with increasing altitude or latitude (Figure 5(a)). 16MME well reproduces the spatial distribution characteristics of temperature and accurately depicts regional features (e.g., Tianshan Mountains and Sichuan Basin) (Figure 5(b)). 16MME generally overestimates monthly temperatures over China, however, showing widespread underestimates in the Hexi Corridor and the Qinghai–Tibet Plateau (Figure 5(c)). The deviation characteristics among different climate classifications are acceptable (ranging from 0.3 to 0.7 °C) (Figure 5(d)). The bias of 16MME on the Qinghai–Tibet Plateau is severe, which may be related to the lack of temperature information at high altitudes.
Figure 5

As in Figure 4, but for temperature.

Figure 5

As in Figure 4, but for temperature.

Close modal

Climate simulation DISO score for NEX-GDDP-CMIP6

The models differ significantly in capturing monthly precipitation across climate classifications (Figure 6). The CC between models and CN05.1 was relatively higher in the SAR and SHD (>0.723), but in other regions (i.e., HD and AR) it was ranged from 0.396 to 0.673, with weaker correlations. The MAE and RMSE in AR were relatively small and concentrated, while they were relatively large and dispersed over HD. The DISO of the models precipitation ranged from 0.856 to 3.085, revealing a huge heterogeneity among climate classifications. Based on the DISO ranking, it is discovered that the best-performing models among climate classifications were MIROC6 (AR and SAR), TaiESM1 (SHD) and ACCESS-CM2 (HD), respectively. For China, the models that ranked first and last in terms of comprehensive performance were ACCESS-CM2 (DISO 1.630) and MPI-ESM1-2-HR (DISO 1.715) (Table 2).
Table 2

Average CC, MAE, RMSE and DISO of 16 GCMs in China, including precipitation and temperature

Climate modelPrecipitation
Temperature
CCMAERMSEDISOCCMAERMSEDISO
ACCESS-CM2 0.643 0.850 1.282 1.630 0.978 2.057 2.552 3.280 
ACCESS-ESM1-5 0.632 0.878 1.313 1.674 0.978 2.053 2.542 3.269 
BCC-CSM2-MR 0.633 0.866 1.292 1.652 0.976 2.114 2.631 3.377 
CESM2 0.631 0.885 1.326 1.688 0.977 2.027 2.532 3.246 
CESM2-WACCM 0.618 0.893 1.339 1.709 0.977 2.033 2.531 3.248 
CMCC-CM2-SR5 0.627 0.875 1.304 1.669 0.970 2.243 2.667 3.564 
CMCC-ESM2 0.633 0.875 1.307 1.668 0.976 2.129 2.652 3.403 
CNRM-CM6-1 0.615 0.889 1.343 1.710 0.975 2.115 2.644 3.387 
IITM-ESM 0.617 0.872 1.317 1.682 0.976 2.061 2.575 3.300 
MIROC6 0.646 0.864 1.284 1.638 0.975 2.119 2.652 3.397 
MPI-ESM1-2-HR 0.614 0.895 1.343 1.715 0.976 2.096 2.604 3.345 
MPI-ESM1-2-LR 0.623 0.876 1.312 1.677 0.977 2.088 2.579 3.320 
MRI-ESM2-0 0.624 0.885 1.336 1.698 0.977 2.076 2.568 3.304 
NorESM2-LM 0.630 0.882 1.326 1.687 0.977 2.080 2.587 3.321 
NorESM2-MM 0.625 0.891 1.339 1.703 0.977 2.089 2.598 3.336 
TaiESM1 0.625 0.880 1.315 1.681 0.971 2.238 2.666 3.560 
Climate modelPrecipitation
Temperature
CCMAERMSEDISOCCMAERMSEDISO
ACCESS-CM2 0.643 0.850 1.282 1.630 0.978 2.057 2.552 3.280 
ACCESS-ESM1-5 0.632 0.878 1.313 1.674 0.978 2.053 2.542 3.269 
BCC-CSM2-MR 0.633 0.866 1.292 1.652 0.976 2.114 2.631 3.377 
CESM2 0.631 0.885 1.326 1.688 0.977 2.027 2.532 3.246 
CESM2-WACCM 0.618 0.893 1.339 1.709 0.977 2.033 2.531 3.248 
CMCC-CM2-SR5 0.627 0.875 1.304 1.669 0.970 2.243 2.667 3.564 
CMCC-ESM2 0.633 0.875 1.307 1.668 0.976 2.129 2.652 3.403 
CNRM-CM6-1 0.615 0.889 1.343 1.710 0.975 2.115 2.644 3.387 
IITM-ESM 0.617 0.872 1.317 1.682 0.976 2.061 2.575 3.300 
MIROC6 0.646 0.864 1.284 1.638 0.975 2.119 2.652 3.397 
MPI-ESM1-2-HR 0.614 0.895 1.343 1.715 0.976 2.096 2.604 3.345 
MPI-ESM1-2-LR 0.623 0.876 1.312 1.677 0.977 2.088 2.579 3.320 
MRI-ESM2-0 0.624 0.885 1.336 1.698 0.977 2.076 2.568 3.304 
NorESM2-LM 0.630 0.882 1.326 1.687 0.977 2.080 2.587 3.321 
NorESM2-MM 0.625 0.891 1.339 1.703 0.977 2.089 2.598 3.336 
TaiESM1 0.625 0.880 1.315 1.681 0.971 2.238 2.666 3.560 
Figure 6

DISO 3D distribution and histograms for 16 GCMs precipitation in four climate classifications: (a) arid, (b) semi-arid, (c) semi-humid, and (d) humid.

Figure 6

DISO 3D distribution and histograms for 16 GCMs precipitation in four climate classifications: (a) arid, (b) semi-arid, (c) semi-humid, and (d) humid.

Close modal
The information on models' temperature is given in Figure 7. There was little difference in CC between models, closely related to CN05.1 (ranging from 0.953 to 0.981). It indicates that all models can well reproduce the linear variation trend of temperature. The average MAE and RMSE of all models were smaller in SAR and HD and more significant in AR and SHD. The best DISO-performing models among climate classifications were ACCESS-ESM1-5 (AR), ACCESS-CM2 (SAR), CESM2-WACCM (SHD) and CESM2 (HD). For China, the top and last models in terms of overall performance were CESM2 (DISO 3.246) and CMCC-CM2-SR5 (DISO 3.564) (Table 2). The models' temperature outputs were strongly correlated with CN05.1, with a small absolute error interval, indicating that they can simulate temperature more accurately.
Figure 7

As in Figure 6, but for temperature.

Figure 7

As in Figure 6, but for temperature.

Close modal
16MME has a relatively strong correlation in SHD and SAD with CC more than 0.8, while it has a weak correlation across other climatic classifications (ranging from 0.6 to 0.97) (Figure 8(a)). The CC of 16MME was relatively dispersed over AR (ranging from −0.6 to 0.97), indicating that 16MME performs poorly in reproducing precipitation trends. 16MME suffers from the negative MAE in the Qinghai–Tibetan Plateau even though it has some over-estimation of the wetting trend in SHD and HD (Figure 8(b)). The RMSE shows a decreasing trend from southeast to northwest (Figure 8(c)). 16MME has relatively better performance in AR with DISO less than 0.75, while it has limited performance over HD with DISO values more than 2.4 (Figure 8(d)). Excessively deviated MAE (−4.8 mm) and anomalously high RMSE (6.8 mm) and DISO (8.4) are observed in the Himalayas, which are probably influenced by the sparse instrumentation and complex topography.
Figure 8

Spatial distribution of CC, MAE, RMSE and DISO for 16MME precipitation, and radar maps based on the regional average value of climate classifications.

Figure 8

Spatial distribution of CC, MAE, RMSE and DISO for 16MME precipitation, and radar maps based on the regional average value of climate classifications.

Close modal
The temperature between 16MME and CN05.1 was strongly correlated over China (>0.982), and the spatial distribution differs greatly from precipitation (Figure 9(a)). MAE and RMSE are relatively small in most areas of China, showing a spatial distribution pattern that increases with latitude or altitude (Figures 9(b) and 9(c)). 16MME has little difference in DISO over climate classifications, and the main difference is sporadically distributed in the northwest (Figure 9(d)). Unlike the stepped spatial distribution exhibited by precipitation, 16MME is more stable in simulating temperature among climate classifications.
Figure 9

As in Figure 8, but for temperature.

Figure 9

As in Figure 8, but for temperature.

Close modal

The best-performing models for precipitation and temperature in China were ACCESS-CM2 (DISO 1.630) and CESM2 (DISO 3.246), while 16MME outperformed them. This phenomenon was observed simultaneously among climate classifications, indicating that the best single model was inferior to the 16MME. The multi-model ensemble is effective in reducing the uncertainty inherent in models, in agreement with Yang et al. (2020). Since 16MME has the best simulation performance, it was used to calculate the drought index to evaluate the drought identification capability of NEX-GDDP-CMIP6.

NEX-GDDP-CMIP6 drought capture DISO score

Drought identification capability of CN05.1

Before measuring the ability of NEX-GDDP-CMIP6 to identify drought, an evaluation of drought-capturing performance on the reference dataset (CN05.1) is essential. We calculated SPI-12 and SPEI-12 of China based on CN05.1 (Figure 10). In addition, the major drought years in history were extracted from the Chinese Meteorological Disasters Dictionary and compared with the drought index series one by one. The SPI and the SPEI failed to identify the droughts in 1970, 1974, 1975 and 1988 (shaded in red) but identified most drought years (shaded in gray). Despite the differences between the SPI and the SPEI, both have excellent drought identification utility in China (Figure 10). Therefore, the drought index calculated by CN05.1 can be used as a reference to assess NEX-GDDP-CMIP6's drought capture capability.
Figure 10

Historical drought years identified by CN05.1 data, with gray shading representing correct detection and red shading representing a failure to detect. Please refer to the online version of this paper to see this figure in colour: http://dx.doi.org/10.2166/nh.2023.140.

Figure 10

Historical drought years identified by CN05.1 data, with gray shading representing correct detection and red shading representing a failure to detect. Please refer to the online version of this paper to see this figure in colour: http://dx.doi.org/10.2166/nh.2023.140.

Close modal

The SPI and the SPEI of 16MME were calculated and statistical analysis with reference data was performed. The drought capture capability, fit degree and error characteristics of NEX-GDDP-CMIP6 were quantified by CC, POD, FAR and DISO. The POD and the FAR were counted with SPI/SPEI ≤ −0.5 as the drought condition.

The SPI between the two data sources was strongly correlated in HD, SHD and SAR, while poor in AR, especially in Xinjiang, with high instability and even negative correlation (Figure 11(a)). The CC of the SPEI gradually increases from the southeast coast to the northwest inland, i.e., it performs best in the AR (Figure 11(b)). The correlation of the SPI is relatively poor in AR but outperforms the SPEI among other climate classifications (Figure 11(c)). In HD and AR, the SPI tends to detect drought incorrectly, while the opposite is true in SHD and SAR (Figure 11(d)). Especially in areas with a negative correlation (Southwest Xinjiang), the FAR exceeds 0.8 and even reaches 1. The FAR of the SPEI performs well in AR, but it is prone to misestimate drought over other climate classifications, especially in northwestern Tibet and northeastern China (Figure 11(e) and 11(f)). The SPI had little difference in the FAR over HD, SHD and SAR but failed to detect drought correctly in northwestern Xinjiang (Figure 11(g)). In AR, the SPEI accurately detects almost all droughts, while in the remaining climate classifications, the average efficiency of POD is only 0.6 (Figure 11(h)). The SPI and the SPEI identified the occurrence of most drought events (mean POD > 0.7), but the capture capacity differed significantly among climate classifications (Figure 11(i)).
Figure 11

Spatial distribution of CC, MAE, RMSE and DISO between drought indices calculated from different data sources and boxplots by climate classification.

Figure 11

Spatial distribution of CC, MAE, RMSE and DISO between drought indices calculated from different data sources and boxplots by climate classification.

Close modal
The SPI suffers from relatively poor DISO in HD and AR, while performing well in the southwest and northeast (Figure 11(j) and 11(l)). The SPEI's DISO performance in AR was excellent. Meanwhile, very poor DISO was found in the Himalayan and Yellow River sources (Figure 11(k)). The drought indices' DISO over the climate classification varied greatly (Figure 12(a)). The spatial distance between the SPEI and the reference is shortest in AR (0.235), while among other climate classifications, the SPI outperformed the SPEI, especially in SAR and SHD.
Figure 12

DISO 3D distribution and histograms of 16MME drought capture capacity in four climate classifications.

Figure 12

DISO 3D distribution and histograms of 16MME drought capture capacity in four climate classifications.

Close modal

Overall, the comprehensive performance of the SPI and the SPEI calculated by 16MME for drought identification over China is acceptable. It is suitable for grid-scale drought detection. We recommend using the SPEI for drought monitoring in AR, while in other climate classifications, it has higher DISO spatial heterogeneity and is relatively inapplicable, so the SPI is recommended in these regions. The selection of appropriate drought indices according to local conditions is beneficial for accurate drought identification, but the climate output of NEX-GDDP-CMIP6 should be used cautiously for drought monitoring in places with poor DISO performance (e.g., the Himalayas and southwestern Xinjiang).

DISO applicability analysis

Taylor diagrams based on 2D space are currently the most common method to measure climate models' performance. Hu et al. (2019) pointed out the shortcomings of Taylor diagrams and proposed the DISO. It measures the comprehensive performance of the evaluated object through a 3D perspective. However, DISO is suitable for assessing the performance of single climate elements among models, but is inapplicable among different climate elements. For example, as the precipitation time scale expands, its absolute error gradually accumulates (year > month), which in turn leads to the expansion of DISO. In contrast, the MAE and RMSE of temperature did not change significantly due to the time-scale expansion. In addition, for precipitation and temperature, the units of MAE and RMSE could not be unified.

Using CN05.1 as a reference, Hu et al. (2019) evaluated the simulation accuracy in China for datasets including GPCC V7, CRU TS 4.01, WM 4.01, ERA-20C, ERA-20CM and CERA-20C. DISO can quantify and differentiate the simulation capability of the above models in China well, indicating the good applicability of DISO in China. In addition, DISO has better evaluation capability than the Taylor diagram. Therefore, it is a more ideal choice for evaluating the performance of climate models in China. Taylor diagrams have been widely used in climate model assessment studies around the world. DISO, also a mathematical statistical model, is an improvement on the Taylor diagram, and therefore, we believe that DISO is suitable for evaluating the simulation capability of climate models around the world. It should be noted, however, that DISO has not yet been applied elsewhere in the world. In a follow-up study, we hope to demonstrate the widespread applicability of DISO.

Zhou et al. (2021) pointed out that DISO applies to big data assessment in the natural and social sciences, and this paper extends its application to drought capture capability assessment. Unlike the first assessment objective, the drought capture capacity is comparable among drought indices. On the one hand, the SPI and the SPEI are standardized drought indices, roughly ranging from −4 to 4. They have the same classification criteria for drought severity (thresholds −0.5, −1, −1.5 and −2 for mild, moderate, severe and extreme drought, respectively). On the other hand, the drought capture capacity framework consists of dimensionless indices (CC ranges from −1 to 1, FAR and POD are both 0 to 1), which are different from MAE and RMSE. In summary, the difference in drought capture capacity between drought indices can be measured by constructing a suitable DISO framework.

Attribution analysis of drought capture capacity

The SPI and the SPEI calculated based on the 16MME have significant spatial heterogeneity in drought capture capability. Except for AR, the drought capture utility of SPI performed well in most regions, which may be related to the scarcity of meteorological observation stations and the poor accuracy of model simulation in precipitation among the AR region. In addition, 16MME underestimates precipitation over China (Figure 3(a)) and the SPI based on it misestimates drought conditions, leading to drought capture capacity variability among climate classifications. The SPEI generally performs inferior to the SPI over China but possesses a strong drought capture ability in AR. 16MME overestimates temperature, which leads to a significant increase in potential evapotranspiration. The SPEI calculation is based on the difference between accumulated precipitation and potential evapotranspiration, which is scaled up in the 16MME simulation. As can be seen in Figure 6, the FAR performs poorly over China except in AR (Figure 6(d)), which indicates that the 16MME overestimates the drought conditions and largely misreports drought conditions that do not exist. In conclusion, the overall drought monitoring capability of NEX-GDDP-CMIP6 over China is acceptable, but the applicable drought index needs to be selected according to the climate classification.

Uncertainties and limitations

The main uncertainty in this study comes from the observations and the multi-model ensemble process. The spatiotemporal range and resolution of CN05.1 fit well with the NEX-GDDP-CMIP6 product, which is the main reason for choosing it as the reference dataset. Although CN05.1 has been used in many studies (Xin et al. 2020), it inevitably has some systematic biases. The higher resolution will introduce greater uncertainty in the interpolation process on the one hand, and on the other, the scarcity of observatories at high altitudes and AR seriously affects the data quality. In future studies, the introduction of multi-source data is considered to comprehensively evaluate the performance of models. We construct the 16MME by weighted averaging, which may lead to homogenization of model performance and does not reflect the ability of the models to simulate climate extremes. In future studies, more reliable methods such as variable weight ensemble averaging and Bayesian model averaging are considered to be introduced.

There are three main categories of systematic biases inherent in climate models: (1) the internal physical simulation process of the climate model and its response to external forcing; (2) the mismatch between the internal phase changes of the climate model and the observations; and (3) the inability to reveal the influence of the grid-scale topography on the models. The statistical downscaling approach of NEX-GDDP-CMIP6 ignores the effects of land use and topography, and the increased resolution fails to reduce the bias between models and observations. It is recommended that dynamic downscaling methods that consider physical mechanisms be introduced in future studies to reduce the models' inherent uncertainty. This study examined the two most commonly used drought indices and many other potentially excellent-performing drought indices were not considered. Therefore, it is recommended that future studies include additional drought indices to provide a more comprehensive view on the performance of NEX-GDDP-CMIP6.

This paper evaluates the simulated performance and drought capture capability of NEX-GDDP-CMIP6 over China based on the DISO and draws the following conclusions.

  • (1)

    None of the climate models capture the fluctuations of China's annual average precipitation well, but they can effectively reproduce the strong warming trend in recent decades. The 16MME underestimates China's annual average precipitation with 40 mm and overestimates the temperature by 0.5–1.5 °C. The 16MME can reproduce the spatial distribution pattern of China's monthly average precipitation and temperature very well, but inevitably there are large errors in areas with high altitudes or few stations (central Xinjiang and the Qinghai–Tibet Plateau).

  • (2)

    The ability of the climate models to capture monthly precipitation among climate classifications varies widely. The best model for simulating precipitation over China is ACCESS-CM2 (DISO1.630) and the worst is MPI-ESM1-2-HR (DISO1.715). The performance of the temperature simulations did not differ significantly among climate classifications, with the top- and bottom-ranked models being CESM2 (DISO 3.246) and CMCC-CM2-SR5 (DISO 3.564).

  • (3)

    16MME can simulate precipitation trends in HD and SHD relatively better, while climatological trends in temperature are well reproduced over China. The MAE and RMSE of precipitation decrease from the southeast coast to the northwest, while the absolute error of temperature increases with elevation and latitude. The best-performing model for precipitation and temperature over China is inferior to the 16MME, which indicates that multi-model ensembles improve the performance of models effectively.

  • (4)

    The CC of the SPI performed poorly in AR and possesses the acceptable performance over other climate classifications, while the SPEI presents a strong correlation only in AR. The SPI based on 16MME is more effective in monitoring drought in HD, SHD and SAR, while the SPEI is recommended for AR areas. Our results emphasize the importance of selecting appropriate drought indices when conducting climate change studies.

This work was supported by the General Project of National Natural Science Foundation of China (Grant No. 52079036 and U2243203). The authors are grateful to the Editor and anonymous reviewers for their constructive comments.

All relevant data are available from an online repository or repositories (https://www.nccs.nasa.gov/services/data-collections/land-based-products/nex-gddp-cmip6).

The authors declare there is no conflict.

Eyring
V.
,
Bony
S.
,
Meehl
G. A.
,
Senior
C. A.
,
Stevens
B.
,
Stouffer
R. J.
&
Taylor
K. E.
2016
Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization
.
Geoscientific Model Development
9
(
5
),
1937
1958
.
Faiz
M. A.
,
Liu
D.
,
Fu
Q.
,
Sun
Q.
,
Li
M.
,
Baig
F.
,
Li
T.
&
Cui
S.
2018
How accurate are the performances of gridded precipitation data products over Northeast China?
Atmospheric Research
211
,
12
20
.
Faiz
M. A.
,
Liu
D.
,
Fu
Q.
,
Naz
F.
,
Hristova
N.
,
Li
T.
,
Niaz
M. A.
&
Khan
Y. N.
2020
Assessment of dryness conditions according to transitional ecosystem patterns in an extremely cold region of China
.
Journal of Cleaner Production
255
,
120348
.
Fu
X. J.
,
Hao
Z. C.
&
Cao
Q.
2019
Study of changes of dry-and-wet climate zone in China from 1961 to 2017
.
Journal of China Three Gorges University (Natural Sciences)
41
,
21
26
.
Gourley
J. J.
,
Erlingis
J. M.
,
Hong
Y.
&
Wells
E. B.
2012
Evaluation of tools used for monitoring and forecasting flash floods in the United States
.
Weather and Forecasting
27
(
1
),
158
173
.
Hao
Z.
&
AghaKouchak
A.
2014
A nonparametric multivariate multi-index drought monitoring framework
.
Journal of Hydrometeorology
15
(
1
),
89
101
.
Hu
Z.
,
Chen
X.
,
Zhou
Q.
,
Chen
D.
&
Li
J.
2019
DISO: A rethink of Taylor diagram
.
International Journal of Climatology
39
(
5
),
2825
2832
.
Lei
X.
,
Xu
W.
,
Chen
S.
,
Yu
T.
,
Hu
Z.
,
Zhang
M.
,
Jiang
L.
,
Bao
R.
,
Guan
X.
,
Ma
M.
,
Wei
J.
,
Feng
A.
&
Gao
L.
2022
How well does the ERA5 reanalysis capture the extreme climate events over China? Part I: Extreme precipitation
.
Frontiers in Environmental Science
10
,
921658
.
Li
Y.
,
Yan
D.
,
Peng
H.
&
Xiao
S.
2021
Evaluation of precipitation in CMIP6 over the Yangtze River Basin
.
Atmospheric Research
253
,
105406
.
McKee
T. B.
,
Doesken
N. J.
&
Kleist
J.
1993
The relationship of drought frequency and duration to time scales
. In:
Proceedings of the 8th Conference on Applied Climatology
. Vol.
17
, No.
22
, pp.
179
183
.
Nashwan
M. S.
&
Shahid
S.
2020
A novel framework for selecting general circulation models based on the spatial patterns of climate
.
International Journal of Climatology
40
(
10
),
4422
4443
.
Taylor
K. E.
2001
Summarizing multiple aspects of model performance in a single diagram
.
Journal of Geophysical Research: Atmospheres
106
(
D7
),
7183
7192
.
Thrasher
B.
,
Wang
W.
,
Michaelis
A.
,
Melton
F.
,
Lee
T.
&
Nemani
R.
2022
NASA global daily downscaled projections, CMIP6
.
Scientific Data
9
(
1
),
1
6
.
Vicente-Serrano
S. M.
,
Beguería
S.
&
López-Moreno
J. I.
2010
A multiscalar drought index sensitive to global warming: The standardized precipitation evapotranspiration index
.
Journal of Climate
23
(
7
),
1696
1718
.
Wu
J.
&
Gao
X. J.
2013
A gridded daily observation dataset over China region and comparison with the other datasets
.
Chinese Journal of Geophysics
56
(
4
),
1102
1111
.
Xin
X.
,
Wu
T.
,
Zhang
J.
,
Yao
J.
&
Fang
Y.
2020
Comparison of CMIP6 and CMIP5 simulations of precipitation in China and the East Asian summer monsoon
.
International Journal of Climatology
40
(
15
),
6423
6440
.
Yang
X.
,
Wood
E. F.
,
Sheffield
J.
,
Ren
L.
,
Zhang
M.
&
Wang
Y.
2018
Bias correction of historical and future simulations of precipitation and temperature for China from CMIP5 models
.
Journal of Hydrometeorology
19
(
3
),
609
623
.
Yang
X.
,
Yu
X.
,
Wang
Y.
,
He
X.
,
Pan
M.
,
Zhang
M.
,
Liu
Y.
,
Ren
L.
&
Sheffield
J.
2020
The optimal multimodel ensemble of bias-corrected CMIP5 climate models over China
.
Journal of Hydrometeorology
21
(
4
),
845
863
.
Yuan
X.
,
Wang
L.
,
Wu
P.
,
Ji
P.
,
Sheffield
J.
&
Zhang
M.
2019
Anthropogenic shift towards higher risk of flash drought over China
.
Nature Communications
10
(
1
),
1
8
.
Zhou
Q.
,
Chen
D.
,
Hu
Z.
&
Chen
X.
2021
Decompositions of Taylor diagram and DISO performance criteria
.
International Journal of Climatology
41
(
12
),
5726
5732
.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying, adaptation and redistribution, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/).