Abstract

In recent years, the use of climatic databases and satellite products by researchers has become increasingly common in the field of climate modeling and research. These datasets play an important role in developing countries. This study evaluated two reanalyses, CMORPH and SM2RAIN-ASCAT over Maharlu Lake, a semi-arid region in Iran. The results showed that these two near-time datasets do not have accurate data over this basin. However, the Probability of Detection (POD), Critical Success Index (CSI), and False Alarm Ratio (FAR) statistics showed acceptable accuracy in the detection of precipitation. The coefficient of determination and root mean square error statistics have unacceptable accuracy over this area. The monthly changes in each of the indices showed that the CMORPH database had more errors in the spring months, but in other months the error rate was improved. SM2RAIN-ASCAT had better accuracy over this area relative to CMORPH. The estimation of the total accuracy of the data showed that these two satellite databases were not capable of estimating precipitation in the area.

HIGHLIGHTS

  • The performance of the SM2RAIN-ASCAT and CMORPH satellite databases against the observation data were compared.

  • Five statistical metrics have been used.

  • Despite the reanalyses, the satellite products could not provide accurate precipitation estimations.

  • This study is done in a semi-arid basin.

  • SM2RAIN-ASCAT showed better accuracy than CMORPH over the Maharlu Lake basin.

Graphical Abstract

Graphical Abstract
Graphical Abstract

INTRODUCTION

In recent years, the use of reanalyses of climatic databases and satellite products by researchers has become increasingly common in the field of climate modeling and research (Duan & Bastiaanssen 2013). These databases have great potential for hydrological simulations due to their homogeneous equilibrium and long-term statistical period. Moreover, they have provided good results in many hydrological studies (Duan & Bastiaanssen 2013). These databases are divided into two categories, satellite and ground databases, and have yielded different results in every part of the world. These databases can be evaluated to accurately estimate precipitation in different dimensions. Mei et al. (2014) investigated TRMM3B42, CMORPH, and PERSIANN to evaluate a flood event in a basin in Italy; the three products in that area showed similar accuracy rates.

A new satellite database, SM2RAIN-ASCAT, has been able to estimate rainfall using soil moisture (Brocca et al. 2019). This rainfall database has been evaluated in various areas, including Australia, Europe, and North America, and has yielded good results in rainfall estimation (Brocca et al. 2019).

Comparisons of SM2RAIN products with other datasets have shown different outputs. Over India, the latest versions of the four multi-satellite precipitation products (CHIRPS, MSWEP, SM2RAIN-CCI, and TMPA) were compared with observed data. The results showed that the CHIRPS dataset could be used for long-term precipitation analyses with rather higher confidence (Prakash 2019). In addition, Paredes-Trejo et al. (2019) SM2RAIN compared with TMPA in Brazil and results showed that SM2RAIN-ASCAT largely underestimated rainfall across the country. Another study, assessed GPM and SM2RAIN-ASCAT rainfall products over complex terrain in southern Italy; the results indicated that the combination of these two products could have better accuracy (Chiaravalloti et al. 2018). Across Italy, Ciabatta et al. (2016) used SM2RAIN for rainfall-runoff modeling during the 4-year period of 2010–2013. Their results showed that discharge simulations improve when ground rainfall observations and the SM2RAIN product are integrated. The results of a study in Pakistan showed that SM2RAIN-ASCAT dominates SM2RAIN-CCI through all climate regions, with average percentage increases in bias, ubRMSE, Theil's U, MSEs, MSEr, and KGE score (Rahman et al. 2019). Another use of the SM2RAIN algorithm is the quantification of irrigation water (Jalilvand et al. 2019). Brunetti et al. (2018) compared SM2RAIN-ASCAT with TMPA, PERSIANN, and CMORPH. The results showed that satellite products underestimate rainfall with respect to ground observations. Darand & Khandu (2020) tested several rainfall products over Iran, and results show that Asfazari and APHRODITE performed the best followed by CHIRPS and GPCC. Shayeghi et al. (2020) used satellite products and reanalysis in a hydrological model (VIC). Hydrological assessment indicates that PERSIANN is the best rainfall dataset for capturing the streamflow and peak flows for the studied area. Belo-Pereira et al. (2011) tested CRU, GPCC, ERA-40, ECMWF, and ERA-Interim contrary to high-resolution gridded precipitation. They showed that all products sensibly recognized the major precipitation in the mountainous areas in the order stated: GPCC, CRU, ERA-Interim, and ERA-40.

However, other satellite products have been assessed at global and regional scales, and TMPA and CMORPH have been found to perform better in many parts of the world (Ferreira et al. 2014). The evaluation of four global satellite precipitation datasets (TRMM: 3B42V7 and 3B42RT, CMORPH, and PERSIANN) for hydrological simulations in China demonstrated that gauge modification significantly moderated the bias in the research-grade TMPA product 3B42V7, but this dataset is not continually superior to other products, particularly CMORPH in the daily time steps (Li et al. 2015; Jiang et al. 2016). The overall results of a study in eastern parts of Africa showed that CMORPH is able to detect the periodic and spatial patterns of rainfall fairly well in both wet and dry conditions over some parts of the basin, but it significantly overestimated those over a lake and its shoreline (Haile et al. 2015).

Given that Iran is located in an area with very low water, simple and timely access to precipitation information is essential for managerial as well as scientific evaluations (Eini 2019). This study is the first to use the SM2RAIN-ASCAT precipitation database to evaluate its accuracy alongside the CMORPH satellite product in a semi-arid basin in Iran.

METHODOLOGY

Study area

The study area of the current research was Maharlu Lake Basin located in Fars Province, which has an area of 4,270 km2. This area is located in the Central Plateau catchment and in the subdivision basins of Bakhtegan and Maharlu Lakes. The mean rainfall and mean temperature recorded in this area is 368 mm and 17.4 °C, respectively. Based on Amberger's method, the climate of the area is temperate semi-arid. In this study, three climatic observation stations were used for comparison with satellite products (see Tables 1 and 2 for details). Figure 1 shows the study area. In this study, the statistical period used to evaluate the accuracy of satellite products was 2007 to 2017 (Eini et al. 2018).

Table 1

Meteorological station information over study area

Station NameLongitude/◦Latitude/◦Elevation/m
Shiraz Synoptic 52.6 29.53 1,481 
Rain gauge 52.55 29.63 1,553 
Rain gauge 52.35 29.85 1,885 
Station NameLongitude/◦Latitude/◦Elevation/m
Shiraz Synoptic 52.6 29.53 1,481 
Rain gauge 52.55 29.63 1,553 
Rain gauge 52.35 29.85 1,885 
Table 2

Hydrological station information

Station NameAverage of discharge/m3/s
Bagh Safa 1.68 
Chenar Sokhte Azam 1.29 
Chenar Sokhte Khoshk 0.56 
Station NameAverage of discharge/m3/s
Bagh Safa 1.68 
Chenar Sokhte Azam 1.29 
Chenar Sokhte Khoshk 0.56 
Figure 1

The study area, Maharlu Lake, and rivers.

Figure 1

The study area, Maharlu Lake, and rivers.

Satellite products

The SM2RAIN-ASCAT (http://hydrology.irpi.cnr.it/download-area/sm2rain-data-sets/) and CMORPH (https://rda.ucar.edu/datasets/ds502.0/) satellite databases were used in this study. The information from each of the databases is listed in Table 3. As shown, these databases are real-time and work closely for climatic simulations. The SM2RAIN-ASCAT satellite product provides daily rainfall data, and CMORPH provides hourly data on rainfall.

Table 3

Description of SM2RAIN-ASCAT and CMORPH

ProductTime spanSpatial coverageTemporal resolutionSpatial resolution
SM2RAIN-ASCAT 2007–2019 Global scale Daily 0.25° × 0.25° 
CMORPH 2002–2017 60° S–60° N Sub daily 8 × 8 km2 
ProductTime spanSpatial coverageTemporal resolutionSpatial resolution
SM2RAIN-ASCAT 2007–2019 Global scale Daily 0.25° × 0.25° 
CMORPH 2002–2017 60° S–60° N Sub daily 8 × 8 km2 

Statistical indices

To estimate the accuracy of the precipitation satellite databases relative to the observation data, the observation data and the precipitation data of SM2RAIN-ASCAT and CMORPH were evaluated on a point-by-point basis. Then, the accuracy of the databases was estimated using the following seven statistical indices in different months: correlation coefficient (CC), root mean square error (RMSE), mean error (ME), bias index (BIAS), probability of detection (POD), false alarm rate (FAR), and critical success index (CSI). CC and ME represent the linear and mean correlation of the difference between the analyzed database and the observation data, respectively. RMSE calculates the mean error between the estimated rainfall and the observations, taking into account the higher weight for more errors. The relative BIAS calculates the systematic error of the precipitation data. The three (POD, FAR, and CSI) classification indices showed a correlation between occurred and estimated precipitation (Darand et al. 2017; Eini et al. 2019). Table 4 shows the relationships between these indices and the optimal value of each.

Table 4

Statistical indicators, equations, and optimal values

NoIndexEquationRangUnitOptimal value
Coefficient of determination  0 to 1 – 
Root mean square error  0 to  mm 
Probability of detection  0 to 1 – 
False alarm rate  0 to 1 – 
Critical success index  0 to 1 – 
NoIndexEquationRangUnitOptimal value
Coefficient of determination  0 to 1 – 
Root mean square error  0 to  mm 
Probability of detection  0 to 1 – 
False alarm rate  0 to 1 – 
Critical success index  0 to 1 – 

In the above Equation, i indicates the day, N is data frequency, Oi is precipitation data for the observation data, Mi is precipitation value for the satellite database, Oavg is mean precipitation of observation data, Mavg is mean satellite precipitation, H is the frequency of days associated with precipitation recorded by the two databases, M is the frequency of days with precipitation not observed but not recorded in the database, and F is the frequency of days of rainfall that the database has recorded but was not observed at the precipitation station.

RESULTS AND DISCUSSION

To estimate the accuracy of precipitation over Maharlu Lake basin, the statistical indices mentioned in the previous section were calculated for the monthly and seasonal periods, each of which is listed in this section. Estimation of the total accuracy of the data showed that these two satellite databases were not capable of estimating precipitation in the area. The figure below shows the total accuracy of these two satellite databases in the study area.

The results of the coefficient of determination (R2) revealed that the SM2RAIN-ASCAT database showed a better correlation than the CMORPH database. The correlation accuracy of the SM2RAIN-ASCAT data is, on average, higher in the southern areas of the basin and about 0.5. However, the northern areas of the basin have a lower correlation of about 0.3. In general, the CMORPH database showed no acceptable results, and this database had no proper correlation with the observation data. The correlation value of this database is close to zero.

The RMSE index indicates a systematic error in precipitation estimation. The closer the index is to zero, the greater the accuracy of the satellite database. The SM2RAIN-ASCAT satellite database continues to show higher accuracy. In most parts of the basin, the RMSE index was less than one millimeter. According to this index, the SM2RAIN-ASCAT satellite database in the southern areas of the basin was more accurate. There were more CMORPH satellite database estimation errors in this basin, and CMORPH did not perform well in any of the basin points. These errors could be because of the (1) systematic errors in data compiling or (2) this satellite product has not calibrated in this region.

The classification indices showed that the probability of precipitation detection (POD) of the SM2RAIN-ASCAT satellite database is lower than that of CMORPH. Rainfall estimations were more accurate in both databases in the southern areas. The false alert ratio (FAR) showed that the CMORPH database was more accurate in detecting precipitation and, on average, detected and recorded 20% error. According to the FAR index, the SM2RAIN-ASCAT database is less accurate in the western and southern areas. Finally, the Critical Success Index (CSI) values of both satellite databases showed very high accuracy. Table 5 shows the mean and median values as well as the optimal values of each of the statistical indices for both precipitation databases. The following figure also shows the spatial changes of these indices. Figure 2 shows the spatial distribution of statistical indices over the study area.

Table 5

The mean, median, and optimal values of each of the statistical indices for both precipitation databases

IndexSM2RAIN-ASCATCMORPHOptimum Value
R2 Average 0.33 
Median 0.21 
RMSE Average 0.82 6,188 
Median 0.65 6,188 
POD Average 0.6 0.8 
Median 0.59 0.71 
FAR Average 0.39 0.20 
Median 0.33 0.18 
CSI Average 0.51 0.66 
IndexSM2RAIN-ASCATCMORPHOptimum Value
R2 Average 0.33 
Median 0.21 
RMSE Average 0.82 6,188 
Median 0.65 6,188 
POD Average 0.6 0.8 
Median 0.59 0.71 
FAR Average 0.39 0.20 
Median 0.33 0.18 
CSI Average 0.51 0.66 
Figure 2

Spatial distribution of statistical indices over the study area.

Figure 2

Spatial distribution of statistical indices over the study area.

Accuracy of estimation of monthly precipitation

The monthly accuracy of each index is shown in the following table. The monthly changes in each of the indices showed that the CMORPH database had more errors in the spring months; however, in the other months, these errors were improved. From April to June, the highest and lowest correlation coefficient errors were observed for the CMORPH database. According to the values in Table 6, there were generally fewer errors in the rainy months.

Table 6

Monthly values of statistical indices

(a) R2 
Product Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 
SM2RAIN-ASCAT 0.23 0.26 0.21 0.13 0.17 0.19 0.12 0.18 0.09 0.1 0.21 0.23 
CMORPH 0.01 0.09 0.05 0.01 0.11 0.13 0.11 0.19 0.18 
(b) RMSE 
SM2RAIN-ASCAT 3.21 3.5 3.41 2.33 5.5 7.5 7.7 8.5 5.6 6.5 4.5 5.1 
CMORPH 4.59 5.99 9.65 6,477 8,960 7,780 5.9 9.5 8.5 6.5 6.7 0.2 
(C) CSI 
SM2RAIN-ASCAT 0.66 0.71 0.69 0.77 0.74 0.69 0.66 0.78 0.59 0.55 0.79 0.81 
CMORPH 0.77 0.75 0.77 0.68 0.81 0.72 0.73 0.68 0.74 0.81 0.88 0.86 
(d) FAR 
SM2RAIN-ASCAT 0.75 0.35 0.34 0.41 0.49 0.51 0.49 0.44 0.61 0.61 0.45 0.45 
CMORPH 0.44 0.49 0.35 0.22 0.35 0.33 0.49 0.33 0.31 0.34 0.42 0.44 
(e) FAR 
SM2RAIN-ASCAT 0.77 0.79 0.87 0.50 0.59 0.59 0.77 0.55 0.65 0.71 0.75 0.54 
CMORPH 0.47 0.59 0.67 0.54 0.66 0.64 0.62 0.63 0.77 0.78 0.81 0.65 
(a) R2 
Product Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 
SM2RAIN-ASCAT 0.23 0.26 0.21 0.13 0.17 0.19 0.12 0.18 0.09 0.1 0.21 0.23 
CMORPH 0.01 0.09 0.05 0.01 0.11 0.13 0.11 0.19 0.18 
(b) RMSE 
SM2RAIN-ASCAT 3.21 3.5 3.41 2.33 5.5 7.5 7.7 8.5 5.6 6.5 4.5 5.1 
CMORPH 4.59 5.99 9.65 6,477 8,960 7,780 5.9 9.5 8.5 6.5 6.7 0.2 
(C) CSI 
SM2RAIN-ASCAT 0.66 0.71 0.69 0.77 0.74 0.69 0.66 0.78 0.59 0.55 0.79 0.81 
CMORPH 0.77 0.75 0.77 0.68 0.81 0.72 0.73 0.68 0.74 0.81 0.88 0.86 
(d) FAR 
SM2RAIN-ASCAT 0.75 0.35 0.34 0.41 0.49 0.51 0.49 0.44 0.61 0.61 0.45 0.45 
CMORPH 0.44 0.49 0.35 0.22 0.35 0.33 0.49 0.33 0.31 0.34 0.42 0.44 
(e) FAR 
SM2RAIN-ASCAT 0.77 0.79 0.87 0.50 0.59 0.59 0.77 0.55 0.65 0.71 0.75 0.54 
CMORPH 0.47 0.59 0.67 0.54 0.66 0.64 0.62 0.63 0.77 0.78 0.81 0.65 

In the spring and summer (April to September), as there is not much rainfall in the study area, virtually both satellite databases had more errors. The number of errors in these seasons was higher in the CMORPH database than in the SM2RAIN-ASCAT satellite database.

DISCUSSION

In contrast to the results of a study which evaluated the accuracy of Tropical Rainfall Measuring Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA v7) over Iran, neither SM2RAIN-ASCAT nor CMORPH had good quality in this region (Darand et al. 2017). In addition, analysis in the current study revealed that these datasets could not offer accurate precipitation estimations during the dry months. The low accuracy during the dry months may be due to the agreement of satellite precipitation valuation algorithms to detect warm precipitation and screen out no-rain, thin cirrus clouds that are usually very cold (Darand et al. 2017). The results showed that these products had low correlation in the highlands, possibly due to the weak detection of orographic precipitation events (Ebert et al. 2007; Duan & Bastiaanssen 2013; Darand et al. 2017). Some studies showed that SM2RAIN-CCI could be have better performance than SM2RAIN-ASCAT across Brazil (Paredes-Trejo et al. 2019). Brunetti et al. (2018) analysis showed that the satellite rainfall datasets underestimate the actual rainfall against the observed datasets.

In all seasons, the selected satellite products underestimated the precipitation values. This could be attributed to hydrometeors sensed by infrared sensors, and the microwave could partly or entirely evaporate before reaching the surface in (hyper) arid areas (Tesfagiorgis et al. 2011; Chen et al. 2013; Darand et al. 2017). The current assessments showed that the CMORPH dataset significantly underestimated precipitation during the summer, possibly due to errors in exporting data from an online source and/or systematic error; however, in other seasons, this dataset had relatively better data.

A study evaluated the SM2RAIN satellite products in Pakistan in the southeast of Iran (Rahman et al. 2019) and revealed that precipitation was underestimated in mountains in glacial and humid regions, similar to the current research. In contrast, the researchers stated that SM2RAIN-ASCAT offered better accuracy in modest- to low-precipitation seasons (post-monsoon and winter). In addition, the POD, FAR, and CSI were ranged in the same values as the current study. However, Darand et al. (2017) reported POD, FAR, and CSI values better than the current study for TPMA over Iran. In another study, four precipitation reanalyses (NCEP-CFSR, CRU, Asfezari, and APHRODITE) were evaluated over a semi-arid basin in Iran for hydrological applications (Eini et al. 2019).

CONCLUSION

The performance of the SM2RAIN-ASCAT and CMORPH satellite databases against the observation data was evaluated using five statistical metrics. These satellite products were assessed over the semi-arid basin in Iran for the period 2007 to 2016. Analyses results showed that these datasets could not provide reliable precipitation estimations over Maharlu Lake; however, they detected precipitation events well. The results showed that, despite the reanalyses which have high accuracy over Iran, the satellite products could not provide accurate precipitation estimations over this part of Iran. SM2RAIN-ASCAT showed better accuracy than CMORPH over the Maharlu Lake basin in Iran.

REFERENCES

REFERENCES
Belo-Pereira
M.
,
Dutra
E.
&
Viterbo
P.
2011
Evaluation of global precipitation data sets over the Iberian Peninsula
.
Journal of Geophysical Research: Atmospheres
116
,
D20
.
Brocca
L.
,
Filippucci
P.
,
Hahn
S.
,
Ciabatta
L.
,
Massari
C.
,
Camici
S.
&
Wagner
W.
2019
SM2RAIN-ASCAT (2007–2018): global daily satellite rainfall from ASCAT soil moisture
.
Earth System Science Data Discuss
11
(
4
),
1
31
.
Brunetti
M. T.
,
Melillo
M.
,
Peruccacci
S.
,
Ciabatta
L.
&
Brocca
L.
2018
How far are we from the use of satellite rainfall products in landslide forecasting?
Remote Sensing of Environment
210
,
65
75
.
Chen
S.
,
Hong
Y.
,
Cao
Q.
,
Gourley
J. J.
,
Kirstetter
P.E.
,
Yong
B.
,
Tian
Y.
,
Zhang
Z.
,
Shen
Y.
&
Hu
J.
2013
Similarity and difference of the two successive V6 and V7 TRMM multisatellite precipitation analysis performance over China
.
Journal of Geophysical Research: Atmospheres
118
,
13,060
13,074
.
Chiaravalloti
F.
,
Brocca
L.
,
Procopio
A.
,
Massari
C.
&
Gabriele
S.
2018
Assessment of GPM and SM2RAIN-ASCAT rainfall products over complex terrain in southern Italy
.
Atmospheric Research
206
,
64
74
.
Ciabatta
L.
,
Brocca
L.
,
Massari
C.
,
Moramarco
T.
,
Gabriele
S.
,
Puca
S.
&
Wagner
W.
2016
Rainfall-runoff modelling by using SM2RAIN-derived and state-of-the-art satellite rainfall products over Italy
.
International Journal of Applied Earth Observation and Geoinformation
48
,
163
173
.
Ebert
E. E.
,
Janowiak
J. E.
&
Kidd
C.
2007
Comparison of near-real-time precipitation estimates from satellite observations and numerical models
.
Bulletin of the American Meteorological Society
88
,
47
64
.
Eini
M. R.
2019
Discussion of ‘Intra-and interannual streamflow variations of Wardha watershed under changing climate’ (2018) by Naga Sowjanya P., Venkata Reddy K. & Shashi M.(ISH Journal of Hydraulic Engineering, DOI: 10.1080/09715010.2018.1473057). ISH Journal of Hydraulic Engineering, 1-2
.
Eini
M. R.
,
Javadi
S.
&
Delavar
M.
2018
Evaluating the performance of CRU and NCEP CFSR global reanalysis climate datasets in hydrological simulation by SWAT model (Case study: Maharlu Basin)
.
Iran Water Resources Research
14
,
32
44
.
Eini
M. R.
,
Javadi
S.
,
Delavar
M.
,
Monteiro
J. A. F.
&
Darand
M.
2019
High accuracy of precipitation reanalyses resulted in good river discharge simulations in a semi-arid basin
.
Ecological Engineering
131
,
107
119
.
Ferreira
V. G.
,
Andam-Akorful
S. A.
,
He
X.-f.
&
Xiao
R.-y.
2014
Estimating water storage changes and sink terms in Volta Basin from satellite missions
.
Water Science and Engineering
7
,
5
16
.
Jalilvand
E.
,
Tajrishy
M.
,
Hashemi
S. A. G. Z.
&
Brocca
L.
2019
Quantification of irrigation water using remote sensing of soil moisture in a semi-arid region
.
Remote Sensing of Environment
231
,
111226
.
Jiang
S.-h.
,
Zhou
M.
,
Ren
L.-l.
,
Cheng
X.-r.
&
Zhang
P.-j.
2016
Evaluation of latest TMPA and CMORPH satellite precipitation products over Yellow River Basin
.
Water Science and Engineering
9
,
87
96
.
Mei
Y.
,
Nikolopoulos
E. I.
,
Anagnostou
E. N.
&
Borga
M.
2014
Error analysis of satellite precipitation products in mountainous basins
.
Journal of Hydrometeorology
15
,
1778
1793
.
doi:10.1175/ JHM-D-13-0194.1
.
Paredes-Trejo
F.
,
Barbosa
H.
&
dos Santos
C. A.
2019
Evaluation of the performance of SM2RAIN-derived rainfall products over Brazil
.
Remote Sensing
11
(
9
),
1113
.
Tesfagiorgis
K.
,
Mahani
S. E.
,
Krakauer
N. Y.
&
Khanbilvardi
R.
2011
Bias correction of satellite rainfall estimation using a radar-gauge product
.