Performance comparison of four daily weather generators for historical period and downscaling future GCM scenarios

Assessment of future regional climate change impacts (e.g., hydrological impact) requires fine-scale future climate projections. General circulation models (GCMs) provide future climate projections under future emission scenarios. However, the resolution of GCMs is coarse, and it is required that GCM outputs be downscaled (Zhang et al. 2019; Fatehifar et al. 2021). Weather generators (WGs) are one of the significant downscaling tools for assessing the regional impacts of climate change (Dubrovský et al. 2004; Fowler et al. 2007; Khazaei et al. 2021).

WGs are stochastic models that can produce arbitrarily long synthetic daily time series of a suite of weather variables (e.g., rainfall, maximum temperature (T_max), and minimum temperature (T_min)). Parameters of WGs are obtained from the observed weather series. The synthetic series are statistically similar to the observed series but provide a broader range of feasible situations. The performance of WGs is generally evaluated by comparing the corresponding statistical properties of observed and produced climate variables. Many studies have shown that WGs can reproduce many of the observed weather statistical characteristics in the synthetic series, which indicates the good performance of WGs for a historical period (Semenov et al. 1998; Dubrovský et al. 2004; Khazaei et al. 2013, 2020a, 2021; Chen et al. 2019). Based on these results, WGs have broadly been used for downscaling outputs from climate models to assess the regional impacts of climate change (Forsythe et al. 2014; Baghanam et al. 2020; Kavwenje et al. 2022). For doing so, the statistical characteristics of the observed weather variables at the study site are changed according to the change factors derived from GCM (or regional climate model (RCM)) simulations to obtain future downscaled statistics. Then, the set of future downscaled statistics is used as input to the WG to perturb its parameters and generate the future downscaled weather series (Wilby et al. 2002; Dubrovský et al. 2004; Kilsby et al. 2007; Semenov & Stratonovitch 2010; Khazaei et al. 2020b). As for the historical period, the WG must be able to reproduce the downscaled statistics set in the downscaled future weather series to ensure that the downscaled series is related to the GCM (or RCM) simulation. In other words, it is essential to ensure that the future changes that occur to the downscaled variables are the same as those prescribed by the difference between the two GCM integrations (Jones et al. 2011). However, the capability of WGs for reproducing downscaled statistics in the downscaled future weather series has seldom been evaluated.

Many WGs have been developed (e.g,, WGEN (Richardson & Wright 1984), Met&Roll (Dubrovský et al. 2004), EARWIG (Kilsby et al. 2007), LARS-WG (Semenov & Stratonovitch 2010), MulGETS-MA (Chen et al. 2019), and IWG2 (Khazaei et al. 2021)). In WGs, generally, rainfall is simulated at first. Then, dependent on whether the day is wet or dry, other variables, such as daily T_max and T_min, are produced.

One of the significant advantages of WGs for downscaling is that they can transfer future changes in various statistical characteristics of the weather variables that are predicted by the GCMs (or RCMs) to the downscaled weather series (Fowler et al. 2007; Khazaei et al. 2013; Jones et al. 2016; Keller et al. 2017). However, since WGs model secondary variables (such as T_min and T_max) as dependent on rainfall occurrence, changes in the rainfall occurrence under future climate scenarios may have unintended effects on these variables. So, WGs may not reproduce the statistics of secondary variables well in future scenarios (Wilby et al. 2007; Jones et al. 2011; Chen et al. 2012). Although second-order adjustments can be made to correct this inefficiency (Wilks 2012), it has not been applied to most WGs (Jones et al. 2016). A few approaches have been proposed to correct this inefficiency. Jones et al. (2011) proposed to modify RCM (or GCM) change factors to enter the WG. The modification formulas can be obtained based on rainfall-related changes in secondary variables derived from observations. They used the proposed method to downscale an RCM future scenario by the UKCP09 WG. Evaluation results indicated that the future series statistics downscaled by the WG did not completely match the corresponding statistics prescribed by the difference between the two RCM integrations, which could be due to the complexity of determining the modified RCM (or GCM) change factors. Semenov (2007) proposed a method to modify the temperature change factors dependent on projected GCM change in mean temperature and rainfall occurrence. This method is used in LARS-WG, which improves the reproduction of future T_max and T_min monthly averages, while its performance is not known for other statistics. Khazaei et al. (2013) proposed an approach based on joining a monthly WG to daily WGs. This method is used in IWG (Khazaei et al. 2013), IWG2 (Khazaei et al. 2021), and M-LARS-WG (Khazaei et al. 2020b) models. The effectiveness of the method was evaluated for each of the models just for reproducing the averages of the variables in a weather station under a GCM future scenario (Khazaei et al. 2013, 2020b, 2021). The results showed that the proposed approach resolved the problem of reproducing the averages. However, the method's performance in reproducing other statistics (e.g., standard deviation (SD) and proportion of wet days) was not evaluated.

Although evaluating the performance of WGs to reproduce observed climate statistics over the historical period is a common step in climate change impact assessment using WGs, the performance of WGs to reproduce future climate statistics (which are used as input to WGs to produce future downscaled series) is generally ignored. In this paper, the performances of four WGs with different structures are comprehensively evaluated for downscaling three GCM scenarios at six sites. For this purpose, a wide range of statistics of the series downscaled by each WG for each GCM scenario are compared with the corresponding downscaled statistics based on the GCM scenarios.

After examining data from synoptic weather stations in Iran, six highly accurate synoptic stations were selected for this study based on criteria such as coverage of a common 30-year period, minimal proportion of missing data, strong correlation with nearby stations, data homogeneity, and representation of a wide range of climate conditions. The characteristics of the weather stations are presented in Table 1. The recorded rainfall, T_max, and T_min at all stations were obtained from the Iran Meteorological Organization. Overall, the proportion of missing data for daily temperature and rainfall series was less than 0.015 and 0.0005, respectively. To estimate the missing data, linear regression was employed as a straightforward and efficient technique, interpolating missing values using data from nearby stations (Ng et al. 2018). The weather data series from the study stations showed strong correlations with corresponding series from nearby stations.

To assess the stationarity of the daily rainfall and temperature time series, the Mann–Kendall test for trend detection (Javanshiri et al. 2021) was performed at a 5% significance level. The results showed no significant trends in the rainfall time series at any of the stations. However, nearly all temperature series exhibited slight increasing trends over the study period, which is expected given the changing climate. Javari (2016) analyzed the homogeneity of rainfall data across various weather stations in Iran, concluding that the rainfall time series at the Ramsar, Hamedan, Shahrekord, Mashhad, and Yazd stations were homogeneous. In the present study, the homogeneity of rainfall data at the Zahedan station was also confirmed using the Pettitt and Buishand tests (Javari 2016), with a significance level of 5%. Javanshiri et al. (2021) examined the homogeneity of temperature data and found that the temperature data from the Ramsar, Hamedan, Zahedan, and Yazd stations were homogeneous. They also observed that the temperature data from the Shahrekord station remained homogeneous until 2004. In addition, Rafati & Karimi (2018) confirmed the homogeneity of temperature data at Mashhad (Rafati & Karimi 2018; Javanshiri et al. 2021).

Moreover, control (1976–2005 period) and future (2070–2099 period) daily rainfall, T_max, and T_min data of the CanESM5 model under SSP 119, SSP 370, and SSP 585 emission scenarios (available at https://cds.climate.copernicus.eu/) were used for performance evaluation of the WGs in downscaling of the future GCM scenarios.

IWG2 is composed of three major components, including D-IWG, which is a daily WG as described in the previous section, a trivariate monthly AR(1) model to fit the low-frequency variability of daily T_max and T_min series, and a quantile perturbation model (QPM) for correcting the low-frequency variability of rainfall outputs of the daily rainfall model by perturbing the monthly means of the synthetic daily series.

Consequently, the series of monthly means of T_max and T_min produced by the daily model fits the monthly series produced by the monthly model to correct monthly statistics of the synthetic daily T_max and T_min series. Further details about the IWG2 model can be found in Khazaei et al. (2021).

LARS-WG is a daily WG that has been widely used to downscale outputs from GCMs in climate change impact assessment studies (Mollel et al. 2023; Goodarzi et al. 2024). This model uses semiempirical distributions to approximate probability distributions of weather variables. The parameters of the probability distributions are determined based on observed weather data.

At first, the occurrence of rainfall is modeled by alternating wet and dry series. The lengths of the wet series (WL) and the lengths of the dry series (DL) are approximated by semiempirical probability distributions. A semiempirical distribution is a histogram with a constant number of intervals. A semiempirical distribution is sufficiently flexible and allows for the accurate simulation of various weather statistics (Semenov 2008). This approach reproduces the duration of wet and dry periods well, especially compared to the Markov chain approach used in many WGs (Semenov & Barrow 1997). Then, the rainfall amounts on wet days are modeled using semiempirical distributions. The T_max and T_min for wet and dry days are approximated by semiempirical distributions. Conditioned on whether the day is wet or dry, the cycles of daily means and daily SDs are reduced from the observed series. Autocorrelation and cross-correlation between the variables are modeled by applying the multivariate AR(1) to the normalized residuals. Further details about the LARS-WG can be found in Semenov (2007) and Semenov & Stratonovitch (2010).

For data generation by M-LARS-WG, synthetic daily weather series are first produced by the LARS-WG model. Then, the low-frequency variability of the synthetic series is corrected in the following steps.

The low-frequency variability of the synthetic daily rainfall series in the previous step is corrected by perturbing the monthly means of the series based on a QPM. In this method, for each month of the year, the empirical frequency distribution of the monthly means series of the synthetic daily rainfall series is fitted to the empirical frequency distribution of the observed monthly rainfall series. Unlike the QPM method in the IWG2 model, empirical frequency distributions instead of theoretical frequency distributions are used in the M-LARS-WG model. Thus, this method is coordinated with the semiempirical distributions of rainfall in LARS-WG.

Then, similar to the IWG2 model, the low-frequency variability of the daily T_max and T_min series is corrected using a trivariate AR(1) model, as described for the IWG2 model. This method improves the performance of the basic LARS-WG model regarding the reproduction of low-frequency variability and some of the observed weather characteristics (Khazaei et al. 2020b, 2024). Further details about the M-LARS-WG can be found in Khazaei et al. (2020b).

The steps for downscaling the outputs from GCMs and generating downscaled future climate scenarios using the WGs may be described as follows:

• 1. For each variable statistics used to calibrate the WGs, the change field (CF) of GCM for the future scenario compared to the control period is computed based on GCM outputs. Equation (5) is used to obtain CFs of GCM for T_max and T_min means.

  

  (5)

  where is the GCM CF for the statistic of S and month i, ⁠. is the corresponding statistic for the GCM output for the future period, and ⁠. is the corresponding statistic for the GCM output for the control period.

For each of the WGs, a list of the statistical characteristics modified for downscaling GCM outputs (for rainfall, T_max, and T_min) is provided in Table S1.

The process of evaluating the performance of WGs in the downscaling of GCM scenarios is described in the following steps:

• 1. The performance of each WG for the historical period (1976–2005) was evaluated. For doing so, the parameters of each WG were obtained from the observed weather series. Then, using each of the WGs, 100 daily rainfall, T_max, and T_min series of 30 years were produced for each weather station. Generating long-time series using stochastic WGs is to capture the effects of climate variability (Mukundan et al. 2019; Khazaei et al. 2020b). Then, the statistics of the generated series were compared with the corresponding statistics of the observed series. Since the WGs are calibrated based on the observed series to produce synthetic series that are statistically similar to the observed series, there should not be significant differences between the corresponding statistics of generated and observed series. In other words, reproducing observed statistics by a WG indicates the good performance of the WG (Semenov et al. 1998; Dubrovský et al. 2004; Khazaei et al. 2013, 2020a, 2021; Chen et al. 2019).

• 2. The performance of each WG for downscaling future GCM scenarios (for the 2070–2099 period) was evaluated. For doing so, instead of observed statistics, parameters of the WGs were obtained based on the downscaled statistics. Then, using each of the WGs, 100 daily rainfall, T_max, and T_min series of 30 years for each future scenario at each station were produced as the downscaled series. The statistics of the downscaled series were then compared with the corresponding downscaled statistics used as input to the WGs.

• 3. The performance of each WG for the historical period (in reproducing observed statistics) was compared with its performance for the future period (in reproducing future downscaled statistics) to determine whether the WG's good performance over the historical period guarantee its performance for downscaling future scenarios?

In Figure 2, the performance of the WGs in reproducing daily means of variables in each month of the year (hereafter referred to as monthly means), mean WL (for LARS-WG and M-LARS-WG), and PW (for D-IWG and IWG2) are shown. Because the performance of the models was similar in reproducing T_max and T_min statistics, results for T_max are not shown. All the WGs well reproduced averages of the weather variables in historical and future periods (Figure 2), and their ECs and KGEs were close to 1 (Tables 2–5). However, by the D-IWG, the future T_min averages are not as well reproduced as the other WGs (Figure 2(e)–2(h)), which is due to the unintended effects of the changes in rainfall occurrence under future climate scenarios on temperature means. While the AAE and KGE in reproducing T_min means were 0.17°C, and 0.99 in the historical period, respectively, they were 0.54°C and 0.958 for the future period, respectively (Table 3 and Table S2 in the Supplementary material). This issue is corrected in IWG2 (Figure 2(e) and Tables 3 and S2) because this model reforms the monthly means of the variables using their monthly component. LARS-WG computes and uses corrected CFs for averages of the secondary variables (Semenov 2007), so it quite well reproduced the future daily T_max and T_min averages (although they are not reproduced as well as M-LARS-WG and IWG2) (Tables 3 and S2).

However, regarding the reproduction of PWs (and WLs), the performance of all WGs in the future period was decreased compared to their performance in the historical period. While for reproducing the PWs in the historical period, KGEs of 0.84 were obtained by IWG2 and D-IWG, they turned out to be 0.78 for the future period (Table 5). Also, for reproducing the WLs in the historical period, KGE of 0.89 was obtained by LARS-WG and M-LARS-WG, but they turned out to be 0.73 for the future period (Table 5).

The AAPE in reproducing PW from 11% in the historical period increased to 20 and 19% in the future period by LARS-WG and M-LARS-WG, respectively (Table S3). By IWG2 and D-IWG, AAPEs in reproducing PW were 17 and 22% for the historical and future periods, respectively (Table S3).

In Figure 3, the performance of the WGs in reproducing the monthly SD of variables is shown. Regarding reproducing the monthly SD of rainfall, IWG2 and M-LARS-WG performed well in the historical period. KGE, EC, and AAPE by IWG2 were 0.95, 0.99, and 5%, respectively, and by M-LARS-WG were 0.96, about 1, and 2%, respectively. While by D-IWG, they were 0.84, 0.97, and 14%, and by LARS-WG, they were 0.76, 0.95, and 13%, respectively (Tables 4 and 5 and S3). The results indicate that the monthly component of the IWG2 and M-LARS-WG improves the performance of the models in reproducing monthly SDs of rainfall for the historical period. Regarding reproducing the monthly SD of rainfall in the future period, KGE, EC, and AAPE by IWG2 were 0.97, 0.99, and 3%, respectively. While by M-LARS-WG, they were 0.90, 0.86, and 13%, respectively (Tables 4, 5, and S3). The results indicate that the theoretical frequency distributions in the IWG2 model perform better than the empirical frequency distributions in the M-LARS-WG to reproduce monthly statistics of rainfall in the future period. KGE, EC, and AAPE by D-IWD were 0.94, 0.94, and 23%, respectively, and by LARS-WG were 0.48, 0.81, and 26%, respectively (Tables 4, 5, and S3). Overall, the results indicate that the performance of the D-IWG, LARS-WG, and M-LARS-WG for reproducing the monthly SD of rainfall in the future period is significantly reduced compared to the historical period. However, IWG2 performs well in both the future and historical periods.

Regarding reproducing the monthly SD of T_min and T_max, the performance of neither the D-IWG nor LARS-WG models is good. However, IWG2 and M-LARS-WG perform very well in historical and future periods (Figure 3(e)–3(h) and Tables 2, 3, and S2). Regarding reproducing the monthly SD of T_min for the historical period, KGE, EC, and AAPE by D-IWG were 0.36, 0.30, and 22%, respectively, and they were 0.20, 0.11, and 24%, respectively, for the future period. KGE, EC, and AAPE by LARS-WG were −0.08, −2.39, and 58%, respectively, for the historical period, and they were −0.07, −2.21, and 57%, respectively, for the future period. However, by M-LARS-WG and IWG2, KGEs and ECs are close to 1, and AAPEs are less than 2% in historical and future periods (Tables 2, 3 and S2). The results show that the monthly component of the IWG2 and M-LARS-WG improves the performance of the models in reproducing monthly SDs of T_max and T_min for historical and future periods.

For the reproduction of the daily SD of rainfall, IWG2 and D-IWG performed well in both historical and future periods, and their ECs were higher than 0.99 (Table 4 and Figure 4(a) and 4(b)). However, according to AAPEs, their performance in the future period has slightly decreased compared to the historical period (Table S3). AAPEs for D-IWG and IWG2 were 3 and 7% in the historical period, respectively; however, they were 10% in the future period. AAPEs for M-LARS-WG and LARS-WG in the historical period were 8 and 5%, while in the future period, they were 19 and 20%, respectively (Table S3).

Overall, IWG2 and D-IWG perform better than LARS-WG and M-LARS-WG in reproducing the daily SD of rainfall (Tables 4, 5, and S3 and Figure 4). LARS-WG and M-LARS-WG do not directly consider changes in daily SDs of rainfall in downscaling climate model outputs. For some applications (such as assessing the impact of climate change on floods), changes in rainfall SDs can be even more important than changes in the rainfall averages (Prudhomme et al. 2002).

Regarding the reproduction of the daily SD of T_min, IWG2 and D-IWD performed well in the historical period, with ECs of 0.95 and 0.97 and AAPEs of 6 and 4%, respectively (Figure 4(e) and 4(f) and Tables 2 and S2). For the future, their ECs decreased to 0.72 and 0.81, and AAPEs increased to 11 and 9%, respectively, which are still good. By LARS-WG, the EC for the historical period was 0.62, which decreased to 0.42 for the future period. Also, AAPE was 16% for the historical and future periods. The performance of M-LARS-WG was better than LARS-WG, with ECs of 0.91 and 0.63 and AAPEs of 6 and 12% for historical and future periods, respectively (Tables 2, 3, and S2). The performance of all the WGs in reproducing daily observed SDs of temperature in the future period was reduced compared to its performance in the historical period.

Overall, the results indicated that the performance of WGs in reproducing weather statistics for the future period is diminished compared to the historical period. Concerns about this issue have also been noted in previous studies (Wilby et al. 2007; Jones et al. 2011; Chen et al. 2012). The results indicate that second-order adjustments are both effective and necessary to address this inefficiency (for example, linking a monthly WG to the daily WG in M-LARS-WG and IWG2, and adjusting change factors for secondary variables in LARS-WG). Second-order adjustments have also been suggested by some other studies to correct this inefficiency (Wilks 2012); however, they have not been applied to most daily WGs (Jones et al. 2016). Therefore, in addition to assessing WG performance for historical periods, it is advisable to evaluate their performance in downscaling future scenarios and, if needed, apply the necessary structural corrections.

WGs are one of the main tools for downscaling the output from GCMs to assess the regional impacts of climate change. However, although the performance of WGs in the historical period has been extensively evaluated, the performance of these models for downscaling future scenarios of climate models (GCMs and RCMs) has seldom been evaluated. In this paper, the performance of four WGs, with different structures, for downscaling the future GCM scenarios in six regions with various climate conditions was comprehensively evaluated and compared with their performance in the historical period.

Based on the results, all the WGs performed well in reproducing the monthly means of rainfall and temperature (T_max and T_min) in both historical and future periods, with ECs greater than 0.99. However, by the D-IWG, the average of absolute errors in reproducing monthly mean T_min increased from 0.17°C in the historical period to 0.54°C in the future period. A similar result was obtained for T_max. It is due to the unintended effects of the future changes in rainfall occurrence on the T_max and T_min means, which is not corrected in D-IWG. The other three WGs address these unintended effects on temperature means.

The performance of D-IWG was acceptable in reproducing the daily SDs of rainfall and temperature and PWs (and WLs) in both the historical and future periods. However, its performance in the future period was not as good as in the historical period. For instance, the EC of the daily SDs of T_min decreased from 0.98 in the historical period to 0.81 in the future period. The performance of D-IWG in reproducing the monthly SDs of rainfall was also fairly good in both the historical and future periods. However, it did not reproduce the monthly SDs of temperature well, with ECs of 0.30 and 0.11 for T_min in historical and future periods, respectively. D-IWG utilizes the NSRP method for rainfall simulation, effectively reproducing the monthly SDs of rainfall. However, the monthly SDs of temperature are not accurately reproduced due to the absence of an interannual component in the temperature model.

By LARS-WG, PWs in the historical and future periods were well reproduced; however, ECs from 0.95 in the historical period decreased to 0.80 in the future period. Also, its ECs in reproducing daily and monthly rainfall SDs decreased from 0.99 and 0.95 in the historical period to 0.95 and 0.81 in the future period, respectively. The ECs of the daily SDs of T_min were 0.62 and 0.42 in the historical and future periods, respectively. Also, its performance in reproducing monthly SDs of T_min in both historical and future periods (with ECs of −2.39 and −2.21, respectively) was not acceptable. The deficiency of LARS-WG in reproducing monthly SDs of variables is due to the short memory of the daily stochastic process, which cannot reproduce interannual variability well.

M-LARS-WG performed well in reproducing all observed statistics, including means, daily SDs, monthly SDs of rainfall and temperature, and WLs over the historical period. The performance of the model in reproducing most of the rainfall and temperature downscaled statistics (except the daily SDs of T_min and monthly SDs of rainfall) was also good in the future period. However, the performance of M-LARS-WG in the future period was not as good as its performance in the historical period. In reproducing observed PW and daily and monthly SDs of rainfall, ECs in the historical period were 0.95, 0.99, and around 1, respectively; however, in reproducing future downscaled PW, monthly SDs of rainfall, and daily SDs of T_min, ECs turned to be 0.80, 0.86, and 0.64, respectively. It performs well in reproducing monthly SDs of T_min and T_max in both historical and future periods (with ECs of around 1).

IWG2 performed well in reproducing most of the variable statistics, both in the historical and future periods. The ECs in reproducing rainfall monthly SDs, rainfall daily SDs, and T_min and T_max monthly SDs are higher than 0.99. It should be mentioned that the model's performance in reproducing the PWs and the daily SDs of T_min reduced over the future period compared to the historical period. In reproducing PW and daily SDs of T_min, the ECs were 0.88 and 0.95, respectively, in the historical period. In comparison, they decreased to 0.74 and 0.72, respectively, in the future period.

Overall, IWG2 performed almost the best among the four WGs for reproducing the observed series statistics and downscaling future GCM scenarios. Moreover, the monthly components of IWG2 and M-LARS-WG significantly improve the performance of the models in reproducing monthly statistics and address the unintended effects of changes in rainfall occurrence under future climate scenarios on the secondary variables.

The performance of the WGs in reproducing some statistics in downscaling future GCM scenarios was not as good as their performance in reproducing the observed statistics in the historical period. So, the good performance of a WG over a historical period does not necessarily guarantee its performance for downscaling future climate models' scenarios, and it is suggested to evaluate the performance of the WG for both historical and future periods. However, including a monthly generator in a daily WG can improve the performance of the daily WG both for reproducing observed weather statistics in the historical period and downscaling GCM future scenarios.

The NSRP is taken from the RainSim V3 Model (Burton et al. 2008). I am grateful to A. Burton, C. G. Kilsby, H. J. Fowler, P. S. P. Cowpertwait, and P. E. O'Connell for providing this model.

Research design, data collection and analysis, and manuscript writing were performed by MRK.

All relevant data are available from an online repository or repositories. The recorded data can be obtained from the Iran Meteorological Organization. GCM data are available at https://cera-www.dkrz.de/.

The authors declare there is no conflict.