Evaluation of the CMIP5 general circulation models for simulating the precipitation and temperature of the Koshi River Basin in Nepal

Climate change is a key driver of sustainability in the Himalayan region (Wester et al. 2019). The Himalayan region is very sensitive to environmental change, since it affects important aspects of the environmental services provided by these regions, such as the water supply to lowlands (Buytaert et al. 2010). The mountains of the Himalayan region play a vital role in the regulation and distribution of water resources and contain the headwaters of 10 major river systems: the Amu Darya, Brahmaputra, Ganges, Indus, Irrawaddy, Mekong, Salween, Tarim, Yangtze, and Yellow, which provide services to 1.3 billion people downstream (Eriksson et al. 2009). The snow and ice stored in the headwater regions of these major rivers sustain seasonal water availability in the downstream areas through snowmelt runoff (Immerzeel et al. 2009; Nepal et al. 2014). The Himalayan region is covered with snow throughout the entire year. Due to climate change, there is snowmelt as well as changes in precipitation and temperature, potentially affecting the hydrology of headwater basins in the Himalayan region. Climate change impact studies on Himalayan river basins are carried out using different climate models such as general circulation models (GCMs) and regional climate models (RCMs) (Lutz et al. 2016; Rajbhandari et al. 2016). However, the selection and identification of climate models for Himalayan river basins, which represent both the existing and future climate, remains challenging. Thus, this study aims to evaluate and identify suitable climate models for the Koshi River Basin located in the Himalayan region. The identification and evaluation methodology adopted in this study is not only specific to this particular basin but can be implemented for other basins in the Himalayan river basins as well.

GCMs represent the numerous atmospheric processes of the global climate system and are the main tools for estimating future climate patterns and studying the changes in precipitation and temperature patterns. The various GCMs are continuously improved to enable the output to be used for different climate change studies in Coupled Model Intercomparison Project Phase 5 (CMIP5). However, there are many unanswered questions, such as ‘Which GCM is the most suitable?’ or ‘How should a proper GCM be selected?’, thus increasing the dilemma for the majority of GCM users in selecting the one most suitable for their study (Lin & Tung 2017). The selection of climate model can vary depending upon its objective and future projection. There are many approaches for model selection such as (a) including all the models/ensembles with available data and simply calculating the average of all predicted outcomes (Seager et al. 2007), (b) using a past performance approach, focusing on the model's capacity to simulate past and present climate (Pierce et al. 2009; Biemans et al. 2013), (c) the envelope approach, whereby GCMs/ensembles are selected at each climatic extreme based on annual means (Lutz et al. 2016), and (d) the multi-criteria decision-making method used in the selection of GCMs for future climate projection (Shiru et al. 2019; Homsi et al. 2020). Compromise programming is one of the multi-criteria decision-making tools for the selection of GCMs and is found to be an ideal and optimal solution compared to the others (Srinivasa Raju et al. 2017; Ahmed et al. 2019).

The evaluation of GCMs prior to their selection in climate change studies is very important. Climate model performance varies from area to area and model to model over a certain period (IPCC 2013). Evaluating the performance of GCMs helps to simulate several hydro-climatic variables in hydrological modelling studies. In many studies, simple and significant statistical performance indicators have been suggested for evaluating GCMs (Preethi & Kripalani 2010; Lupo et al. 2015). A few other studies suggest skill score and other performance metrics like the correlation coefficient (CC), normalised root-mean-square deviation (NRMSD), and root-mean-square error (RMSE) (Macadam et al. 2010; Sun et al. 2015). Recently, the evaluation of GCMs has been attempted using the strength values of individual models (Johnson et al. 2011; Sperber et al. 2013). Various studies have also used weighting schemes of multi-model ensembles with varying outcomes (Fordham et al. 2011; Sreelatha & Anand Raj 2019).

The Koshi River Basin is a transboundary river basin and covers the geographical areas from the Himalayan region to the Terai region of Nepal. The evaluation of GCMs in such a river basin is quite challenging due to poor data availability on climate patterns such as snow cover, snowmelt, floods, and droughts. Some studies on climate change and water resources in the Himalayan region have mainly focused on the impact of global climate change on the glacier regime, hydrology, floods, and droughts (Akhtar et al. 2008; Lutz et al. 2016; Zhao et al. 2018; Shamshirband et al. 2020), but the lack of data from these remote and inaccessible areas combined with the complex response of glaciers to warming means that considerable uncertainty exists regarding the results.

Despite similar studies in the existing literature on climate projections for the Koshi River Basin based on a few GCMs, there is a lack of research on the selection of GCMs based on statistical performance in the basin. A recent study (Kaini et al. 2019) used the envelope-based approach combined with past performance to select GCMs for the Koshi River Basin. The current study mainly focuses on the selection of GCMs to simulate the precipitation and temperature of the Koshi River Basin. Since the climate in the Koshi River Basin varies from the mountain region to the Terai region, the GCMs for this study are selected according to the climate indices. The first objective of this study is to evaluate the GCMs using performance indicators and climate indices, while the second objective is to evaluate the performance of the selected GCMs using the ranking technique.

The Koshi River Basin, located in the eastern part of Nepal, is a transboundary river basin between China, Nepal, and India (Figure 1). The total catchment area of the river at its confluence with the Ganges is 74,030 km² (FMIS 2012), 43% of which lies in China, 42% in Nepal, and the remaining 15% in India (FMIS 2012). The Koshi River Basin is divided into three physiographic zones: the Trans-Himalayan area to the north, High Himalayan, and Middle Mountains area in the centre and low-lying plain areas to the south (Rajbhandari et al. 2016). The Koshi River drains most of the eastern part of Nepal and has three main tributaries: Tamor, Arun, and Sunkoshi in the eastern, middle, and western parts of the basin, respectively. The annual temperature and annual precipitation in the Koshi River Basin range from −3 to 28 °C and 32.5 to 925.25 mm. The climatic seasons in the basin can be classified into pre-monsoon (March–May), monsoon (June–September), post-monsoon (October–November), and winter (December–February). Glaciers and snow contribute significantly to the runoff of the major rivers including the Koshi (Immerzeel et al. 2009). Accumulated snow acts as a reservoir, releasing water after melting. Snow-covered areas are estimated to represent about 3.26% of the Koshi River Basin.

The meteorological data for this study were collected from the Department of Hydrology and Meteorology (DHM) in Nepal. There are 35 rainfall stations and 14 temperature stations in the Koshi River Basin. For ungauged area, six rainfall and temperature stations from the gridded datasets, such as Asian Precipitation – Highly Resolved Observational Data Integration Towards Evaluation (APHRODITE) precipitation data (0.25° × 0.25°) and Climate Prediction Centre (CPC) (CPC_NOAA) temperature data (0.5° × 0.5°), were used as the observational data (Table 1).

Twelve GCMs were selected from the pool of CMIP5 including precipitation and temperature outputs from 1980 to 2005. These models were selected based on the literature review and studies relating to the Koshi River Basin (Agarwal et al. 2016; Lutz et al. 2016; Rajbhandari et al. 2016; Kaini et al. 2019). Table 2 provides an overview of the models, their home institution, and resolution. Further details can be found on the CMIP5 website (https://pcmdi.llnl.gov/mips/cmip5/).

The overall methodology framework is depicted in Figure 2. The aim of the study is to select the best GCMs out of the 12 to simulate the precipitation and temperature in the Koshi River Basin under different climate indices. Performance indicators were used to examine how well the simulated data from the GCMs compared with the observed data. Four indicators from the literature review were considered, namely the CC, NMRSD, absolute NMRSD (ANMRSD), and average absolute relative deviation (AARD). First, the seven climate indices from ETCCDI were selected based on frequency, intensity, and duration to evaluate the performance of the GCMs (Table 3). The climate indices were calculated for precipitation and temperature from 1980 to 2005 using RCLIMDEX software.

Secondly, the seven climate indices were calculated, and the weights were determined using the entropy method. The entropy method calculates the weights of the performance indicators and then applies maximum and minimum normalisation methods. Finally, the compromise method was employed to rank the GCMs based on their performance. The specific calculation methods are as follows:

The smaller the estimated NMRSD value, the better the model performance.

Similar to the NMRSD, smaller AARD values indicate better performance, with zero being the ideal value.

The weight of each indicator is calculated using Equations (5) and (6) and normalised using Equation (7). The maximum normalisation was used for the CC, whereas the minimum normalisation was applied to the NMRSD, ANMRSD, and AARD. For every climate index, the performance indicator weight was calculated (Supplementary Tables S2 and S3). The weights from the entropy method were used in the compromise programming.

Precipitation is shown to vary for the observations and 12 GCMs in the mean annual cycle of the period from 1980 to 2005 over the whole basin and three physiographic zones of the Koshi River Basin, as presented in Figure 3. The annual rainfall of the whole Koshi River Basin was 3,965 mm, whereas the maximum rainfall falls in the High Himalayas and Middle Mountain areas of the basin. The Koshi River Basin is highly influenced by the wet season (May–September), the 80% of the total annual rainfall falls in the wet season (Table 4). The 12 GCM values of rainfall for the whole basin and three physiographic zones tend to underestimate the observed values, especially for the wet season.

The maximum and minimum temperatures for the observations and 12 GCMs in the mean annual cycle of the period from 1980 to 2005 over the whole and three physiographic zones of the Koshi River Basin are also presented in Figure 3. The annual maximum temperature of the whole basin was 19.7 °C, whereas high values of the maximum temperature fall in the low-lying plain areas of the basin. The maximum temperature shows the progressively decreasing trend from low-lying plain areas to Trans-Himalayas for both wet and dry seasons. The 12 GCM values tend to overestimate the maximum temperature compared to the observed data for both wet and dry seasons (Table 5).

Similarly, the annual minimum temperature of the whole basin was 8.4 °C, whereas the minimum temperature can be found in the Trans-Himalayas areas of the basin for both dry and wet seasons. The minimum temperature also shows the decreasing trend from low-lying plain areas to Trans-Himalayas. During the dry season, the minimum temperature of Trans-Himalayas can fall up to −14.0 °C. The 12 GCM values tend to underestimate the minimum temperature compared to the observed data for both wet and dry seasons (Table 6).

Significant variability was exhibited in the precipitation and temperature projections made by different GCMs. Each GCM has different strengths and weaknesses; some perform better over tropical regions and others over mountainous regions. It is important to select the model which performs best in the region of interest. The 12 GCMs were analysed using performance indicators and climate indices for rainfall and temperature, respectively. The performance of GCMs under different climate indices, varied according to the model used, as demonstrated by the performance indicators.

The 12 GCMs were evaluated based on the climate indices and performance indicators (Figure 4). A CC value near to 1.0 indicates good model performance. Among all 12 GCMs, MPI-ESM-MR and NorESM1-M showed good performance according to the cold spell duration index (CSDI) only, while all GCMs exhibited a satisfactory performance for frost days (FD) (Figure 4(a)). The smaller the NMRSD value, the better the model performance. The consecutive dry days (CDD) and simple daily intensity index (SDII) show smaller NMRSD values for all 12 GCMs, indicating a satisfactory performance, whereas for FD, all GCMs exhibited large NMRSD values, indicating a poor performance (Figure 4(b)). The CDD, SDII, and warm spell duration index (WSDI) show smaller ANMRSD values for all 12 GCMs, indicating a satisfactory performance. Similarly to the NMRSD, the performance of ANMRSD was also very poor for FD (Figure 4(c)). The FD and very wet days (R95p) show smaller AARD values for all 12 GCMs, indicating very good performance with values of 0.04 for MIROC-ESM and 0.05 for MIROC5 (Figure 4(d)).

Similarly, the 12 GCMs were evaluated based on the climate indices and performance indicators for three different physiographic zones of the Koshi River Basin (Supplementary Figures S1–S3). The performance of 12 GCMs varies within different physiographic zones of the Koshi River Basin, Nepal. For low-lying plain areas, IPSL-CM5A-MR and CSIRO-MK-3.6.0 show the poor performance for R95p and WSDI with CC values of −0.01 and −0.02. NorESM1-M shows the satisfactory performance for CSDI (Supplementary Figure S1a). ACCESS 1.0 shows the poor performance for consecutive wet days (CWD) with an NMRSD value of 0.59. The SDII, FD, and CDD show smaller NMRSD values for all 12 GCMs, indicating good performance (Supplementary Figure S1b). For FD, all GCMs exhibited large ANMRSD values, indicating poor performance, whereas CDD, CSDI, and SDII show the small values of ANMRSD, indicating the very good performance of all 12 GCMs (Supplementary Figure S1c). Most of the GCMs like ACCESS 1.0, NorESM1-M, MIROC5, and IPSL-CM5A-MR show the higher values of AARD for CWD, indicating poor performance. Overall, 12 GCMs show good performance with smaller values of AARD for SDII, R95p, and CDD (Supplementary Figure S1d).

For High Himalayas and Middle Mountains, IPSL-CM5A-MR shows the poor performance for R95 and WSDI with a CC value of −0.16. Again, IPSL-CM5A-MR shows the satisfactory performance for CSDI (Supplementary Figure S2a). NorESM1-M shows the poor performance for FD with an NMRSD value of 36.69. The SDII, CDD, and CWD show smaller NMRSD values for all 12 GCMs, indicating good performance (Supplementary Figure S2b). Again, NorESM1-M shows the poor performance for FD with an ANMRSD value of 37.25. The SDII, CSDI, and CDD show smaller NMRSD values for all 12 GCMs, indicating good performance (Supplementary Figure S2c). IPSL-CM5A-MR shows the poor performance for CWD with an AARD value of 2.07. Overall, 12 GCMs show good performance with smaller values of AARD for FD and R95p (Supplementary Figure S2d).

For Trans-Himalayas, BCC-CSM1.1 shows the poor performance for SDII with a CC value of −0.18. MPI-ESM-MR shows the satisfactory performance for CSDI (Supplementary Figure S3a). MPI-ESM-LR shows the poor performance for CSDI with an NMRSD value of 3.65. The FD, CDD, and CWD show the smaller NMRSD value for all 12 GCMs, indicating good performance (Supplementary Figure S3b). Again, FD and CDD show the smaller ANMRSD value for all 12 GCMs, indicating good performance (Supplementary Figure S3c). Similarly, AARD values for all 12 GCMs for FD and R95p are small, indicating excellent performance. Overall, all 12 GCMs show the very good performance for FD in Trans-Himalaya's areas of the Koshi River Basin.

Compromise programming is used to rank the performance of GCMs. For each GCM, the is calculated using Equation (8). An with the minimum value is considered to be the best model. The minimum values for different climate indices vary, and the ranking of GCMs differs according to variations in the climate indices. For the SDII, the minimum value of L_p(a) is 0.47 for ACCESS 1.0. This means that ACCESS 1.0 is ranked in the first position (Table 7). Similarly, FD in MIROC5, R95p in CSIRO-Mk3.6.0, warm spell daily index (WSDI) in CanESM2, CSDI in NorESM1-M, CDD in BCC-CSM1, and CWD in MIROC-ESM are ranked first based on the climate indices. The ranking of the model varies within the different physiographic zones of the Koshi River Basin. For low-lying plain areas, CSIRO-Mk3.6.0, MIROC5, MRI-CGM3, NORESM1, IPSL-CM5A-MR, and MIROC-ESM are ranked first based on the climate indices (Supplementary Table S5). Similarly for High Himalayas and Middle Mountains, NorESM1-M ranked first for three climate indices i.e., SDII, WSDI, and CSDI. This indicates that NorESM1-M performed well in the different climate of Middle Mountain zone of the Koshi River Basin (Supplementary Table S6), and for Trans-Himalayas zones, CSIRO-Mk3.6.0 ranked first for FD and R95 which indicates this model simulates well for intensity-based climate indices (Supplementary Table S7). Sreelatha & Anand Raj (2019) ranked 36 CMIP5 GCMs using compromise programming for the Telangana region in Southern India. The models MIROC5, CNRM-CM5, HadGEM2-A0, ACCESS 1.0, and BCC-CSM1.1 were observed to be ranked as the first five most suitable, based on average temperature (T_avg). Moreover, Srinivasa Raju et al. (2017) used compromise programming to rank 36 CMIP5 GCMs in India, based on the performance of maximum (T_max) and minimum (T_min) temperature simulations. According to their evaluation, MIROC5 and MIROC4 h models from CMIP5 were shortlisted for both T_max and T_min. Deepthi et al. (2020) evaluated the performance of 38 GCMs using the entropy method and compromise programming and observed that the ensemble of MPI-ESM-P, CNRM-CM5-2, and CNRM-CM5 was suitable for the prediction of precipitation in the Upper Godavari sub-basin of India.

The CMIP5 GCMs, such as IPSL-CM5A-MR, BCC-CSM 1.1, CanESM2, CSIRO-MK3-6-0, and MRI-CGM3, are mostly used for the study of future climate patterns and the impact of climate change in the Koshi River Basin (Lutz et al. 2016; Rajbhandari et al. 2016; Kaini et al. 2019). Figure 5 shows the rankings for all 12 GCMs and climate indices in the Koshi River Basin, while Figure 6 shows the frequency of GCM rankings 1, 2, 3, 4, 5, and 6 for different climate indices. This statistical analysis shows the repetitive occurrence of different climate indices, helping to identify the best GCM for each index. The WSDI and CSDI evaluate the climatic extremes in air temperature. The air temperature in the Koshi River Basin varies greatly depending on the region. Extreme high temperatures are found in low-lying plain areas, while extreme minimum temperatures are found in the Trans-Himalayas. Based on the evaluation, NorESM1-M, CanESM2, CNRM-CM5, and MPI-ESM-MR are the most suitable models for predicting the climatic extremes of air temperature in the low-lying plain areas or Trans-Himalayan region of the Koshi River Basin. The CDDs and CWDs are used to evaluate the climatic extremes in precipitation. Therefore, based on the above results, the CNRM-CM5 and CSIRO-MK3.6.0 are the most suitable models for predicting the rainfall pattern in the Koshi River Basin. The intensity-based indices R95p, SDII, and FD are used to evaluate the intensity of precipitation and temperature. Similar to air temperature, the intensity of precipitation varies greatly depending upon the region. Based on the results, IPSL-CM5A-MR, CanESM2, and CSIRO-MK3-6-0 are suitable for use in predicting the intensity of rainfall and temperature for all three different zones of the Koshi River Basin. Rajbhandari et al. (2016) projected the rainfall and temperature in the different physiographic zones of the Koshi River Basin of Nepal using CanESM2, CSIRO-MK3.6.0, IPSL-CM5A-LR, and GFDL-ESM2G models under RCP4.5 and RCP8.5 scenarios. The future projections show a 14% increase in rainfall during the summer monsoon season by 2050. But the basin will also experience a decrease in rainfall in low-lying plain areas. The basin is likely to experience warming throughout the year. The greatest warming is expected over the high-altitude regions, while the low-lying plain areas are projected to have low warming.

Despite the existence of numerous research studies on the Koshi River Basin, this study has the added benefit of identifying the most suitable GCMs for future climate prediction in the basin.

This study focuses mainly on the performance evaluation of 12 CMIP5 GCMs in simulating the observed precipitation and temperature over the Koshi River Basin. The performance was assessed using four statistical performance indicators (CC, NMRSD, ANMRSD, and AARD) with seven different climate indices (CDD, CWD, SDII, R95p, FD, CSDI, and WSDI). Furthermore, the weights of the statistical performance indicators were determined using the entropy method. The aforementioned analysis and results demonstrate that compromise programming has the ability to identify suitable GCMs for the Koshi River Basin. This study provides the optimal models for use in different climate change and hydrology studies on the Koshi River Basin. The main results of this study are as follows:

• The GCM performance varies within the different zones of the Koshi River Basin. Some GCMs perform well for low-lying plain areas but poorly for another zone during the same period.

• The GCM performance is different for each climate index. Some GCMs perform well for one climate index but poorly for another during the same period.

• The CDD and CWD are extreme precipitation-based climate indices. According to the analysis, GCMs, such as CanESM2 and CSIRO-MK3.6.0, performed better for CWD and CDD for all physiographic zones of the Koshi river basin. This indicates that CanESM2 and CSIRO-MK3.6.0 models can represent the climate of the overall region and can provide better simulation during extreme precipitation events.

• The WSDI and CSDI are extreme temperature-based climate indices, affecting the snow and ice accumulation/melt process which is an important factor in the upstream of Himalayan river basins (Kaini et al. 2019). According to the analysis, GCMs, such as BCC-CSM 1.1, CanESM2, NorESM1-M, and CNRM-CM5, perform better in the WSDI and CSDI, indicating better simulation for extreme temperature events in Himalayan river basins.

• The FD, R95p, and SDII are intensity-based climate indices. According to the analysis, GCMs, such as IPSL-CM5A-MR, CanESM2, and CSIRO-Mk3.6.0, exhibit better performance. Hence, these models can also provide better simulation during R95p and FD.

• Overall, the IPSL-CM5A-MR, CanESM2, CNRM-CM5, BCC-CSM 1.1, NorESM1-M, and CSIRO-Mk3.6.0 GCMs perform better in most of the climate indices and in all three physiographic zones of the Koshi River Basin. Hence, these models can be used for future climate prediction, the study of climate change impact on hydrology, and the multi-model ensemble method in the Koshi River Basin, Nepal.

The authors express their sincere gratitude towards the Department of Hydrology and Meteorology (DHM) in Nepal for providing the necessary data to facilitate the successful completion of this research work.

All relevant data are included in the paper or its Supplementary Information.