Statistical assessment of uncertainty and performance of multiple satellite-based precipitation products in capturing extreme precipitation events across Punjab Province, Pakistan

Precipitation holds paramount importance as it contributes to upholding the equilibrium of the atmosphere. With the increasing frequency of hydrological disasters, there is a rising global demand for the monitoring of extreme precipitation events. Precipitation stands as a crucial element within the water cycle. Moreover, precipitation significantly impacts hydrology, environment, agricultural practices, Flora, fauna, and natural ecosystems. (Liu 2015). Hence, to gain a deeper insight into the effects of rainfall on the environment, it is essential to have access to rainfall observations with high spatial and temporal resolution. Accurate precipitation data are crucial for hydro-climatological research, the design of hydrological infrastructure, the scheduling of irrigating crops, and forecasting floods and droughts (Liu 2015).

In conventional approaches, weather radar and on-the-ground measuring stations are regarded as trustworthy sources of precipitation information (Qin et al. 2014). However, the insufficient number of ground-based stations in Punjab Province, along with their uneven distribution, presents a hurdle in obtaining consistent measurements for diverse applications (Nashwan et al. 2019). Furthermore, relying on ground-based measurements is limited by uncertainties stemming from errors introduced by factors like wind, the occurrence of missing values, and heterogeneity (Dunn et al. 2011; Guo et al. 2016; Sharifi et al. 2016). Similar limitations associated with ground-based measurements are also prevalent in the Punjab Province. Hence, it is crucial to explore reliable alternative sources of continuous precipitation data for these areas.

Over the past few years, due to the advancements in remote sensing and geo-information technology, multiple global satellite-based precipitation products (SPPs) have been developed and released for free access and they emerged as a dependable and cost-effective method for retrieving precipitation data (Nawaz et al. 2020). Consequently, it is imperative to explore alternative, trustworthy sources of continuous precipitation data from regional to global scale (Ferraro 1997; Susskind et al. 1997). SPPs can help to address the issue of sparse data in regions where ground-based measurement networks are inadequate. Additionally, they provide continuous data with high spatiotemporal resolution for global precipitation monitoring (Fang et al. 2019). These precipitation products are increasingly used in hydrology, climatology, meteorology, and other scientific studies. This is due to the fact that they have convenient data access, wide spatial coverage, accurate spatiotemporal resolutions, ongoing monitoring, and free access to both real and post-real-time data (Zhao et al. 2017). Fluctuations in rainfall patterns, combined with an increasing population, have become a significant hazard in the Punjab Province, Pakistan (Laraib et al. 2024).

Generally, algorithms utilized by SPPs retrieve precipitation data through the use of either microwave (WM) or infrared (IR) sensors (Susskind et al. 1997; Palomino-Ángel et al. 2019). Recent advancements in SPPs have yielded algorithms with the ability to provide precipitation data by integrating data from both microwave (MW) and infrared (IR) sensors (Anjum et al. 2018). These algorithms were applied in the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) and Climate Hazards Group Infrared Precipitation (CHIRP) (Hsu et al. 1997) and these SPPs provide information about precipitation events with the highest attainable precision in both temporal (3-h intervals) and spatial (0.25°) resolutions (Moazami et al. 2013). The PERSIANN-CDR provides daily global rainfall information with a 0.25 grid resolution. The University of California's Centre for Hydrometeorology and Remote Sensing introduced this dataset in March 2000. The SM2RAIN algorithm employs a ‘bottom-up’ methodology, utilizing information from the ASCAT, SMOS, and AMSRE to calculate worldwide soil moisture levels (Cheema et al. 2011; Hou et al. 2014; Ghazanfari et al. 2016). The SM2RAIN dataset, abbreviated as SM2rain hereon, has been accessible since 2007.

SPPs provide reliable precipitation data, yet their accuracy varies based on the specific geographical region (Dembélé & Zwart 2016; Guilloteau et al. 2016). So, before putting ideas into practice, it is essential to check their accuracy. Over recent years, numerous researchers have evaluated the precision of SPPs in diverse global regions. This evaluation has been conducted in diverse locations including Africa (Guilloteau et al. 2016; Pellarin et al. 2020), Europe (Derin et al. 2016; Duan et al. 2016; Beaufort et al. 2017), America (Mourre et al. 2016; Zubieta et al. 2017; Paredes-Trejo et al. 2019) Austria (Sharifi et al. 2019), Australia (Forootan et al. 2016), China (Ullah et al. 2019; Zhang et al. 2020), Iran (Mosaffa et al. 2020), and Pakistan (Cheema & Bastiaanssen 2012; Anjum et al. 2016, 2018; Ahmed et al. 2020). Mosaffa et al. (2020) suggested utilizing PERSIANN to comprehend the spatial and temporal fluctuations of precipitation in Iran.

It's notable that the Punjab province holds significant importance in agriculture and irrigated farming in the country. Moreover, it exhibits high vulnerability to fluctuations in various meteorological parameters, with frequent occurrences of extreme weather events, making it particularly susceptible to the impacts of climate change (Khattak et al. 2015). While certain studies have evaluated the performance of PERSIANN-CDR and PERSIANN-CCS in Pakistan (Cheema et al. 2011; Anjum et al. 2016; Hussain et al. 2017), but recently developed products like SM2RAIN and CHIRP which have high spatial and temporal resolution still not considered for the Punjab Province region. The review of the literature revealed that the Punjab Province's inability to use the data from on-site meteorological stations for a variety of hydro-meteorological applications (Cheema & Bastiaanssen 2012; Anjum et al. 2018). The current study endeavors to address knowledge gaps by conducting a comprehensive multi-scale assessment of the spatiotemporal uncertainties inherent in various global precipitation products sourced from diverse origins. The primary objectives of this research are: (1) to assess the degree of uncertainty in the selected SPPs against ground-based observations within Punjab Province and (2) to examine the capturing ability of SPPs to detect the variability of extreme precipitation events. The findings of this study will be valuable for both the developers refining the algorithms of the examined satellite products and for end-users of SPPs seeking reliable data.

The Punjab Province is situated geographically at coordinates 27.70° N–34° N and 69.3° E–75.38° E, covering an area of 205,344 km² in the northwestern region of South-central Asia in Pakistan. It constitutes 56% of the total population of the country and encompasses 26% of Pakistan's total land area. The province is subdivided into 36 districts and 9 divisions. Punjab stands as the second-largest province after Baluchistan and is bordered to the south by Sindh, to the southwest by Baluchistan, to the west by Khyber Pakhtunkhwa, and to the north by the Islamabad Capital Territory and Azad Kashmir.

In terms of topography, Punjab is primarily characterized by fertile river valleys. In the southern part, the Cholistan desert comprises barren terrain. The region boasts one of the most extensively irrigated landscapes globally, with canals crisscrossing the province. The topography of Punjab is predominantly characterized by rich alluvial plains, which have been shaped by the Indus River and its four principal tributaries in Pakistan: the Jhelum, Chenab, Ravi, and Sutlej rivers, all of which flow from north to south. Regarding climate, Punjab falls within the tropical continental region. The temperature in this region can vary from 4 to 47 °C, and the average annual precipitation measures approximately 603 mm.

The PERSIANN dataset can give global daily precipitation data with a grid precision of 0.25°, covering a spatial range between 50° South and 50° North (Ur Rahman et al. 2018). For this study, daily precipitation evaluations from PERSIANN at a 0.25 grid scale were obtained through the CHRS website (https://chrsdata.eng.uci.edu/) for a continuous period of 8 years, spanning from 2010 to 2018. The website http://hydrology.irpi.cnr.it/download-area/sm2rain-datasets/ was utilized to acquire the daily precipitation estimates for the SM2RAIN product at a grid scale of 12.5 km. By combining the daily data, time series information for all SPPs covering monthly, seasonal, and annual periods was created. This approach aligns with previous research, in which scientists employed it to gather data at monthly, seasonal, and yearly intervals using daily precipitation estimates from SPPs.

In this research, the accuracy and uncertainty characteristics of four SPPs named as PERSIANN-CDR, PERSIANN-CCS, CHIRP, and SM2RAIN were examined in comparison to data collected at on-site meteorological stations. The study was limited to the SPP grids that had at least one relevant meteorological station for reference. Data obtained from the referenced measuring stations, spanning from January 2010 to December 2018, were employed to assess the effectiveness of SPPs across different timeframes (daily, monthly, seasonal, and annual) and spatial scales.

This study evaluated the ability of all SPPs to accurately capture the spatial variability of precipitation in Punjab Province. Various widely accepted evaluation metrics including correlation coefficients (CC), Bias, relative Bias (rBias), and root mean square error (RMSE) were employed to gauge the performance of the SPPs. These measures have been extensively employed by prior researchers to analyze the error traits of SPPs under various Weather and geographical features (Anjum et al. 2018).

In the provided equations, G_i denotes the gauge-based data obtained from reference stations, while G represents the average of this gauge data. S_i is for the satellite-derived estimations, S is for their average, and n stands for the overall number of observations. A satellite product is considered a perfect match for the gauge-based data when its BIAS and RMSE values are both 0, and its CC value is 1. A noteworthy analysis of SPPs has been conducted by Anjum et al. (2018). To determine the applicability of CC and rBias in hydro-climatological contexts, specific thresholds have been defined. The specified criteria for rBias and CC are ±10 and ≥0.70, respectively. Positive BIAS values indicate an underestimation of precipitation levels, while negative values indicate an overestimation. If BIAS produces negative or positive numbers, it means that the amount of precipitation has either been underestimated or overestimated.

A comparative analysis of probability density functions (PDFs) was conducted between satellite-derived precipitation products and reference data to assess the occurrence and statistical properties of precipitation intensity in Punjab Province. This analysis covered the period from January 2010 to December 2018, both on a daily and seasonal basis in accordance with WMO guidelines. Furthermore, Taylor diagrams, a powerful tool used to evaluate the performance of satellite products or models by comparing them to reference data, are widely used in atmospheric sciences, hydrology, climate modeling, and other geophysical disciplines to evaluate gridded data outputs. The primary purpose of using a Taylor Diagram is to visualize and evaluate the performance of multiple SPPs at a time by comparing its observations with reference data. This diagram combines three complementary statistical measures on a single plot, i.e. CC (R²), standard deviation, and RMSE with respect to the reference data. The RMSE values were depicted as angular distances from the x-axis, while the CC values were represented along the azimuthal angle.

The range of values in the CC chart was more extensive for CHIRP, signifying a wider variation, whereas for SM2RAIN, it exhibited the smallest range. This indicates that SM2RAIN demonstrated a more consistent performance in terms of correlation across the stations. In terms of BIAS, CHIRP exhibited minimal variation in its box chart, while SM2RAIN had the lowest rBIAS values compared to the other SPPs in the Punjab Province. This indicates that SM2RAIN tended to provide estimates that were closer to the observed values with less systematic bias. The CC chart displayed a broader range of values for CHIRP, indicating a wider variation. In contrast, SM2RAIN showed the smallest range, suggesting a more consistent performance in terms of correlation across the stations.

In this study, various statistical indices and methods were chosen to thoroughly evaluate the ability of precipitation products to capture rainfall and their error characteristics across 26 stations in the Punjab Province. These indices and methods fall into three main categories: the first includes CC, which measures the agreement between satellite estimates and gauge observations; the second includes the bias, relative bias (rBias), and RMSE, which assess the error and bias of satellite estimates compared to gauge observations; and the third includes the PDF and Taylor diagram, which describe the event frequency and uncertainty of the SPPs. In situ gauge precipitation stations offer valuable data on rainfall amounts and frequency. However, their uneven spatial distribution and data inconsistencies limit their effectiveness, particularly in regions with challenging geographical environments (Kühnlein et al. 2014). To address this limitation, SPPs provide a viable alternative for precipitation estimation, offering spatially distributed data over extensive areas (Zhu et al. 2011).

Prior research has also examined the performance of these SPPs, including PERSIANN-CCS PERSIANN-CDR, and SM2RAIN in diverse climatic and topographic conditions worldwide (Moazami et al. 2013; Xu et al. 2017; Anjum et al. 2019; Rahman et al. 2019; Sharifi et al. 2019; Zhang et al. 2020). In this research, uncertainty analysis of the selected SPPs revealed that their accuracy is entirely dependent on the local topographical and environmental conditions. It is widely acknowledged that the effectiveness of SPPs is strongly influenced by local topographical and climatic conditions. Nadeem et al. (2022) also reported that the accuracy of SPPs strongly varies with in situ topography and climatic conditions. For instance, a study by Tan et al. (2015) conducted in Malaysia associated six SPPs, including PERSIANN-CDR and TRMM-3B42V7, upon investigation, it was evident that the precision of precipitation products was notably influenced by the regional climate patterns and the algorithms employed in satellite precipitation retrieval. The results of our study reinforce the idea that the performance of these satellite products is markedly influenced by the region's topography and climate conditions, which is consistent with previous research findings (Anjum et al. 2018, 2019; Ahmed et al. 2020).

Notably, our study revealed that SM2RAIN demonstrated an ability to capture spatial variations in precipitation, which stands in contrast to the findings of Asif et al. (2023) concluded that SM2RAIN effectively assessed the spatial variability of precipitation across the Pakistan range, this might be possible due to their superior morphing retrieval algorithm. Among the considered SPPs, CHIRP and SM2RAIN exhibited superior performance in characterizing the spatial variability of observed precipitation. Previously, Chiaravalloti et al. (2018) confirmed the admirable performance of PERSIANN-CDR in delineating spatial variations in precipitation when compared to other satellite products. At a daily scale, the correlations (CC) between all SPPs and the reference data were generally modest, with CC values falling below 0.61. CHIRP, SM2RAIN, PERSIANN-CCS, and PERSIANN-CDR, on the other hand, all had correlations with the reference monthly data that were greater than 0.70. This shows that, compared to daily estimations, the monthly evaluations of SPPs have more reliable agreements with the reference data. Likewise, Hamza et al. (2020) indicated that the correlations of all SPPs with the reference data on a daily scale were unsatisfactory, with values below 0.50. However, at a monthly scale, their values exceeded 0.80, suggesting improved agreements.

Regarding RMSE values, all SPPs exhibited an increase with higher elevations, signifying decreasing accuracy with increasing elevation. As the elevation increased, CC values for PERSIANN-CCS, CHIRP, and SM2RAIN slightly increased, while for PERSIANN-CDR, the increase in CC values was more significant. With increasing elevation, the relative Bias (rBIAS) of PERSIANN-CCS and PERSIANN-CDR significantly decreased compared to CHIRP and SM2RAIN, which showed slight decreases. Similarly, Xu et al. (2017) proposed that various indices (BIAS and RMSE) showed a significant correlation with topographic variations. At a monthly scale, all SPPs showed improved agreement with gauge estimations compared to the daily scale, aligning with findings from the previous study (Hamza et al. 2020). While on a seasonal scale, all SPPs demonstrated proficiency in capturing light precipitation events but during the summer season, they exhibited notable over/underestimation relative to the PDF (%). This result aligns with the previous study (Nadeem et al. 2022).

The performance and uncertainties of four commonly used SPPs were comprehensively assessed across various climatic regions by employing different statistical error metrics, including Taylor diagrams and PDF analysis, alongside ground observation-based precipitation data. This evaluation aimed to gauge the ability of these SPPs to accurately capture precipitation patterns. Our key findings can be summarized as follows:

• After moving average analysis, it was found that the CHIRP and SM2RAIN products successfully tracked the data provided by the reference gauges with a rate of 1.5 and 1.7 mm/day, respectively.

• On a monthly scale, CHIRP and SM2RAIN indicate a particularly high and favorable association because of their high CCs >0.90, lower RMSE value (<0.50), and closely match the SD value with reference data. While, on daily scale performance evaluation, none of these SPPs have the ability to capture accurate daily precipitation patterns when in comparison to the data from gauges.

• It is observed that as elevation increases the accuracy of these SPPs in estimating precipitation tends were decrease. Similarly, in terms of higher precipitation scenarios, capturing accuracy of all SPPs tends to decrease in high-precipitation conditions except PERSIANN-CDR which indicates a better performance with observed precipitation.

• On a seasonal scale, all SPPs performed better in the winter season in terms of CC (>0.5) and in the autumn season as variation in the BIAS and RMSE value is less. Based on PDF analysis, all SPPs showed better performance in detecting the occurrence of events with less than 2 mm of precipitation per day.

CHIRP and SM2RAIN emerged as the preferable solutions for hydro-meteorological investigations in the Punjab Province. In future, researchers utilize a triple collocation (TC) approach which provides reliable and continuous assessment of SPPs data over the ungauged areas. It is also suggested that the large uncertainties in satellite-based products should be paid more attention over mountainous areas where rain gauge data are insufficient.

The research work was made possible through the support of Pakistan Meteorological Department (PMD) for their valuable assistance in conducting this research.

There is no funding for this research work.

U. S. conceptualized the whole article, rendered support in data curation and formal analysis, investigated the article, developed the methodology, arranged the software, and wrote the original draft. M. L. conceptualized the whole article, rendered support in formal analysis, developed the methodology, wrote the review and edited the article. M. A. I. conceptualized the whole article, rendered support in formal analysis, developed the methodology, arranged the resources, supervised the work, wrote the review and edited the article. M. A. rendered support in data curation and formal analysis, wrote the review and edited the article. K. A. rendered support in data curation and formal analysis, supervised the article, wrote the review and edited the article. H. U. K. rendered support in formal analysis, wrote the review and edited the article. A. B. A. rendered support in formal analysis, wrote the review and edited the article. O. K. conceptualized the whole article, and wrote the review and edited the article.

This research article titled ‘ Statistical Assessment of Uncertainty and Performance of Multiple Satellite-Based Precipitation Products in Capturing Extreme Precipitation Events Across Punjab Province, Pakistan ‘ does not require an ethics statement as it pertains to a climate change study and does not involve human subjects, animal subjects, or any sensitive data that would require ethical considerations. The research is focused on performing a comparative assessment of multi-SPPs over Punjab Province, Pakistan. Therefore, no ethical approval was necessary for the conduct of this study.

Data cannot be made publicly available; readers should contact the corresponding author for details.

The authors declare there is no conflict.