Statistical refinement of the North American Multi-Model Ensemble precipitation forecasts over Karoon basin, Iran

Nowadays, the importance of precipitation forecasting has resulted in the development of various dynamic models to predict global precipitation at hourly, daily, monthly, and seasonal time-scales (Dehban et al. 2020). The North American Multi-Model Ensemble (NMME) is one of the most famous general circulation models that provide effective seasonal precipitation forecasts (Slater et al. 2019; Becker et al. 2020; Roy et al. 2020). However, the NMME models' predictions are accompanied by some uncertainties due to initial assumptions, model limitations, and inaccurate forecast ensembles (Rayner et al. 2005; Wu et al. 2011; Tao et al. 2014). As a result, researchers have attempted to improve the predictions by using different statistical postprocessing approaches. In these methods, establishing a good relationship between the observed and predicted NMME model's data is the most common important task. In this regard, the application of joint distributions and functions (Kelly & Krzysztofowicz 1997; Robertson et al. 2013), neural networks (Pakdaman et al. 2020), hybrid models (Xu et al. 2018; Yazdandoost et al. 2020), and copula-based models (AghaKouchak et al. 2010; Sadegh et al. 2017; Khajehei et al. 2018) is widely used. These methods lead to improved forecast data, but there are some challenges in evaluating extreme events, which are an essential issue.

Most of the postprocessing methods used for precipitation forecasting employ parametric distributions to simulate the behavior of each variable (observation or predicted data) or parametric bivariate functions. However, copula-based models have the unique ability to use parametric, nonparametric, and semiparametric distributions to fit the bivariate distribution that best fits the observed and predicted variables (Chen & Huang 2007). The fully parametric estimation of copulas involves determining parametric models for both marginal distributions and copulas (first suggested by Oakes (1982) in the context of the Clayton copula). Later, semiparametric estimation was suggested, which uses nonparametric marginal distributions to achieve a parametric copula (Oakes 1986; Genest et al. 1995). However, one deficiency of these methods is the elimination of the relation between extreme observed data and forecasts or vice versa. Consequently, they may not accurately predict the extreme precipitation values. Accurate estimation of extreme values is crucial in flood-prone areas (Exum et al. 2018). The use of nonparametric estimations for both the marginal distributions and copula function (such that they are parameter-free) can overcome this weakness (Chen & Huang 2007; Bouri et al. 2019). Therefore, proposing a nonparametric estimator may provide a more effective approach than parametric copula models.

Based on the literature, while preparing for possible future floods has always been emphasized, there has been limited academic attention (from academics) to the assessment and improvement of NMME models' skill in forecasting extreme precipitation, which may lead to flooding (Slater et al. 2019). Following these efforts, this study aims to improve NMME precipitation forecasts through postprocessing using a nonparametric Copula–Bayesian approach, with a specific focus on improving the forecasts of extreme values.

Previously, the Copula–Bayesian approach has been applied in different studies as a method for improving forecasts. This method causes a conditional probability density function (CPDF), which describes the probability of an observation data event given each raw data. Various approaches have been proposed for selecting the improved data based on CPDF (Madadgar & Moradkhani 2012; Khajehei & Moradkhani 2017). One of the most common methods is to select the maximum likelihood of CPDF as the desired improved forecast. In the parametric method, this is an individual point, but in semiparametric and nonparametric methods, there may be multiple relative maximum points in the CPDF. However, when using this method, there is no guarantee that the selected data are the most accurate data. The main contribution of this paper is to introduce a novel method that utilizes CPDF with various relative maximum values to identify the most suitable improvement data, particularly for the extreme precipitation values, by distinguishing between the maximum relative values and selecting the best one. The effectiveness of the proposed method has been evaluated in one of Iran's largest and most important watersheds, the Karoon River basin, which experienced widespread floods and landslides in the spring of 2019 due to unprecedented rainfall within a brief duration, resulting in extensive damage. This event highlighted the importance of timely and accurate precipitation forecasts, especially for extreme precipitation values, as they can contribute to advances in flood early-warning systems. Furthermore, the longer the forecast horizon for precipitation data, the better the preparedness for possible warnings.

The rest of this article is structured as follows. First, the study area and the data used are presented in Section 2. Subsequently, Section 3 presents the proposed research methodology in four subsections: (1) preparing the input data for the following steps, (2) estimating the CPDF based on the Copula–Bayesian method for each raw forecast data, (3) describing the proposed new approach for selecting the most-improved data, and (4) introducing some statistical criteria for evaluating the skill of forecast data. In Section 4, the obtained results are presented, and finally, in Section 5, the concluding remarks are condensed.

In recent years, extreme precipitation events have been observed in this watershed. One of these devastating events occurred in 2019, when several consecutive floods inundated large parts of the region following heavy rainfall. During this incident, many cities in the Karoon basin, such as Khorram Abad, Ahvaz, and Shahrekord, experienced more than 50% of their average annual precipitation. Therefore, accurate precipitation forecasts can be used as effective management tools to avoid or reduce the damage caused by such natural disasters, prevent further degradation of natural resources, and promote sustainable development. Table 1 presents the observed long-term monthly precipitation data.

The Global Precipitation Climatology Center (GPCC) dataset was first created in 1989 by the German Weather Service (DWD) on behalf of the World Meteorological Organization (Yazdandoost et al. 2020). It is a precipitation dataset based on data from around 80,000 observation stations from several different sources, providing gridded data with different spatial resolutions (2.5° × 2.5°, 1.0° × 1.0°, 0.5° × 0.5°, and 0.25° × 0.25°). In this study, monthly records of GPCC with 1° cell spatial resolution (according to the forecast's resolution) were used as reliable surrogate reference gridded data for observed precipitation (Rezayi Banafsheh et al. 2011; Azizi et al. 2015; Darand & Zand Karimi 2016) from 1982 to 2018.

Five NMME models were utilized at a spatial resolution of 1° to provide monthly precipitation forecasts over the study area. Table 2 provides more information about the NMME models used. The ensemble means of each model from 1982 to 2010 (hindcast or reforecast period) were used to form nonparametric Copula and reforecast-based calibration. The validity of the presented postprocessing process was assessed (directly to each model's ensemble means) for the forecast period of 2011–2018.

First, the available data must be classified into the observational and forecast time series for each month of the year. In this regard, GPCC and the ensemble raw NMME data time series are formed for each month separately. Then, the obtained time series containing monthly precipitation data (with the annual time steps for each 1° cell) are used to prepare a marginal distribution. Later, the marginal distribution functions are separately estimated for historical observations and the model in the analyses period based on the normal kernel density distribution, which has been shown to be efficient in previous studies (Yazdandoost et al. 2020; Yazdandoost et al. 2021). The marginal distribution and kernel Copula function are set based on the historical period from 1982 to 2010. Finally, as an input variable, the obtained fitted distributions are evaluated for the forecast period (2011–2018).

See the study by Nelsen (2006) for a comprehensive overview of copulas and their mathematical properties.

In Equation (5), is the CPDF, and are the marginal distributions of the samples from the forecast and observation at time t, respectively. The sample data consist of 500 random data with the same distribution of variable in the hindcast period (Khajehei & Moradkhani 2017). The process mentioned earlier for creating CPDF (as CPDF_f) will be carried out for each specific raw forecast data. We encourage readers to further explore the topic by referring to Sklar (1959) and Yazdandoost et al. (2021).

Next, the relative maximums of ⁠, which are closer to the raw forecast⁠, are chosen. Each of and has a unique conditional likelihood of occurrence based on and ⁠. In this study, the technique for order preference by similarity to ideal solution (TOPSIS) method is used as a decision-making tool to prioritize the ⁠. This method which was developed by Hwang & Yoon (1981) and Yoon & Hwang (1995) is presented in Appendix 1 of the Supplementary Information. In this multicriteria decision-making method, the selection alternatives (maximum relative points) are ranked based on the degree of similarity to the desired values (observation), and the one with the highest rank is introduced as the best potential for selection. See the study by Garg (2019) for the related equations and method properties. The likelihood of an occurrence forecast given the observation and vice versa, as well as the PDF of each sample observation and forecast are the four input criteria of TOPSIS used to introduce the sample observation by taking the relative closeness to the ideal solution.

Validation of forecast data against observations is fundamental to scientific advancements, algorithm/model developments, and integration of data into applications. The Kling–Gupta efficiency (KGE) criterion is used to evaluate the reliability of NMME raw data (Equation (8)). The KGE combines the three components of Nash–Sutcliffe efficiency of model errors (i.e., correlation, bias, ratio of variances, or coefficients of variation) in a more balanced way. It has been widely used for model evaluation in recent years. According to Khajehei & Moradkhani (2017), KGE values of more than 0.6 can be considered acceptable. For the lower values, the importance of the postprocessing increases. Validation and quantification of the raw or improved NMME models' uncertainties are fundamental issues for evaluating their performance.

The validation of forecast NMME models, skill (raw and improved values), particularly in extreme value detection, is investigated using four volumetric indices (Equations (9)–(12) in Table 3) developed by AghaKouchak & Mehran (2013).

Table 3

Equations used for validation and quantification

	(8)
	(9)
	(10)
	(11)
	(12)
Parameters
	Correlation
	The ratio of the variance of the forecast to the variance of the observation
	The ratio bias
	The NMME model value
	The observation
	Total number of observation (or NMME) data in the desired time series
	Threshold

	(8)
	(9)
	(10)
	(11)
	(12)
Parameters
	Correlation
	The ratio of the variance of the forecast to the variance of the observation
	The ratio bias
	The NMME model value
	The observation
	Total number of observation (or NMME) data in the desired time series
	Threshold

View Large

The first index, the volumetric hit index (VHI), calculates the volume of correctly detected improved precipitation volume and missing observation values. The volumetric false alarm ratio (VFAR) calculates the volume of false simulations (inaccurate improved data) to the volume of simulations. The volumetric miss index (VMI) describes the missing observation's volume to the correctly detected simulation volume and missing observations. Finally, the volumetric critical success index (VCSI) indicates an overall measure of volumetric performance, including volumetric hit, false alarm, and misses (AghaKouchak & Mehran 2013). One benefit of volumetric indexes is their ability to decompose biases of improved data by evaluating different thresholds. According to the main purpose of this study, these indexes are a suitable tool to evaluate the efficient performance of NMME data postprocessing and extreme precipitation value detection. In this study, the threshold value applied to volumetric index calculations is selected based on extreme value detection. Extreme precipitation should be rarer than the tenth or 90th percentile of the observed density probable precipitation function (Shaffie et al. 2019). For areas with a high risk of flooding, the threshold value can be considered 0.9, as suggested by Shaffie et al. (2019).

To assess the accuracy of the raw precipitation data of each individual model ensemble mean, the consistency of the data with the GPCC data for the most precipitated months was determined. Table 4 shows the evaluation of the KGE criterion for the entire study area during 1982–2018 and the six rainiest months. As seen, the KGE value of the study area is less than 0.6 in all months; therefore, it is important to perform postprocessing for each cell.

To assess the skill of the postprocessing method, Table 5 presents a comparison of each raw and improved NMME model forecasts against the GPCC calculated over the extent of the study area. In this table, the columns refer to the results of one-month (January) to six-month (January to June) length forecasts starting from January of years 2011 to 2018. As shown in this table, the raw data of CanCM3 and CanCM4 had the lowest estimates, while CCSM4 slightly underestimated and GFDL slightly overestimated. However, applying the double Copula postprocessing method revealed that no single model consistently had the best performance for each forecast length. Therefore, investigating the skill of all models is necessary to address the involved uncertainties. As expected, the model error after postprocessing was significantly reduced in most models and different studied periods. Since accurate estimates of future precipitation are crucial for managing water resources, our results suggest that for short-term planning (1–2 months), CFS2 and CanCM3 models are the most reliable, while for estimating resources in the 3-6-month horizon, CanCM4 outperforms the others. Interestingly, the raw data from the CanCM4 model had the poorest accuracy compared with the other models, but its postprocessing proved highly effective. The findings also indicate that correcting raw data in all models will lead to an overestimation of precipitation compared with actual values (except for the CanCM4 model). Therefore, considering this in future planning is critical.

Furthermore, the intercomparison of models shows that the raw GFDL model often tends to overestimate precipitation. In addition, it has estimated the highest precipitation values compared with the observed values, and the CFSv2 model has the most similarity to the observation data. It is also shown that the raw data of the CanCM4 model were unable to predict periods of high precipitation intensity and the raw data of the CCSM4 model showed the highest pattern similarity of estimating extreme precipitation. Moreover, the results showed that the postprocessed data were more consistent with the observational data, indicating the effectiveness of the double Copula postprocessing method.

Therefore, it can be claimed that the proposed method is truly able to improve the overall results and identify extreme precipitation values. Since the intensity or frequency of heavy rains in the region has not significantly increased, it can be considered that the cause of recent floods in the region may be due to increased urbanization and/or changes in land use/cover, which subsequently cause changes in runoff.

According to the volumetric indices shown in the figure, during the forecast period, the extreme precipitation values are better identified in the raw GFDL model. As seen in the second row of Figure 6, the postprocessed precipitation values in both hindcast and forecast periods have been significantly improved. Among these modified precipitation values, the KGE index has the lowest value, while the CFSv2 has the least value. In other words, modified forecasts can still be improved to gain sufficient certainty. However, in terms of other indices, the postprocessing algorithm has been very successful.

Seasonal precipitation forecasts are one of the main inputs of hydrological forecasting models. If these forecasts are reliable, they can provide useful information for decision-makers in water resources management. In this paper, a Copula-based Bayesian approach is used as the postprocessing method to evaluate the use of nonparametric distribution for both the marginal distributions and copula function to improve NMME precipitation forecasts. Here, the normal kernel density distribution function is employed as the marginal distribution, and the kernel Copula function is used as the bivariate function to create the conditional probability distribution function (CPDF). Introducing several relative maximum likelihoods in the CPDF can make it challenging to choose the most-improved data. To address this issue, here, a novel method is suggested that chooses the relative maximum points and applies the Copula–Bayesian approach for the second time on the selected values diversely to receive the initial forecast value for the given sample observations. Finally, the TOPSIS decision-making method is applied to pick out the most likely sample observation. The proposed postprocessing method is examined on the Karoon basin, one of the regions in Iran that have experienced significant flood damage. Five NMME models for the hindcast (1982–2010) and forecast (2011–2018) periods are used. The GPCC observational data are used to evaluate the accuracy of the ensemble's mean. The KGE values indicate the necessity of improving the NMME model output. By investigating the obtained results for forecasts with one- to six-month lengths, we found the following:

• 1.

  The spatial and temporal distributions of GFDL raw models are more similar to the observational data.

• 2.

  The GFDL raw models' values relatively perform better than those of the other models. Conversely, CanCM3 and CanCM4 models miss the early months of each year.

• 3.

  According to the postprocessed data, the spatial and temporal distributions are highly consistent with the observations. However, in creating time series and subsequent processing methods, the spatial and temporal coherence of data with adjacent cells are ignored.

• 4.

  Based on the used volumetric indexes (VHI, VMI, VFMR, and VCSI), the raw forecast model values have low skill for estimating extreme values of precipitation. After the postprocessing procedure, the strength of the double Copula method in determining the extreme values was demonstrated.

• 5.

  The higher accuracy and correlation of different improved NMME data imply lower uncertainties than raw estimations.

These findings enable us to make a correct decision about the following important questions: (1) which model can more accurately estimate the total amount of precipitation, (2) which models have higher accuracy in months with heavy precipitation, (3) which models perform better in the short-term horizon versus the long-term horizon, and (4) which models tend to underestimate or overestimate the actual values, and what are the corresponding error margins for each. Such information is crucial for effective water resource management planning and the development of reliable warning systems.

Given the methodology utilized in this study and the resultant findings, there are several potential areas for future research that will complete the presented postprocessing method and make critical contributions to the improvement of forecasts. For example, investigating the use of spatial copulas to account for the dependence between neighboring cells, which was not considered in this study, is an interesting topic. Also, the investigated NMME data are monthly, so the time series in each cell are performed for each month's data separately. But now, daily NMME models are presented. Exploring the use of daily NMME models instead of monthly data and developing new strategies to incorporate data dependency and seasonality trends lead to an increase in forecast skill. This could involve the use of temporal copulas to illustrate the temporal dependency of precipitation data and its impact on postprocessing outcomes. Precipitation is a function of various variables, and the authors propose to apply multidimensional copulas to consider other effective parameters such as temperature and topography.

All relevant data are included in the paper or its Supplementary Information.

The authors declare there is no conflict.