An informed decision on building climate resilience and adaptation requires a deep understanding of long-term changes in temperature and precipitation, especially for most vulnerable countries to climate extremes like Vietnam. Hence, this study aimed to comprehend spatiotemporal trend possibilities of temperature and precipitation in Vietnam by employing multiple non-parametric statistical trend tests. This study also suggested a modified procedure, represented as a Hovmöller-like diagram, for enhancing the characterization of long-term spatiotemporal trends by applying multiple monotonic trend tests to all latitude coordinates through all possible 30-year periods. The superiority of multiple trend tests over a single one is that this methodology can assess the sensitivity of trend test results to the beginning years, ending years, and record lengths, thus emphasizing the necessity of performing monotonic trend tests repeatedly. The results show consistent warming trends in all climate sub-regions over the last 4–5 decades, with the estimated trend slopes varying from approximately 0.010–0.042 °C/year. Precipitation anomalies in most climate sub-regions, especially in the southern part, experienced significant increasing trends at the rate of around 0.29–2.76%/year during the last 3–5 decades. These findings are expected to contribute more insights into the spatiotemporal trend patterns of temperature and precipitation in Vietnam.

  • A single trend test is sensitive to the analyzed period in terms of trend direction, magnitude, and statistical significance.

  • Multiple trend tests offer temporal trend patterns concerning different beginning years, ending years, and record lengths.

  • A modified procedure, visualized as a Hovmöller-like diagram, was proposed to explore spatiotemporal trend possibilities.

Graphical Abstract

Graphical Abstract
Graphical Abstract

Vietnam, classified as a low- or lower-middle-income economy during 1987–2020, is among the countries that are most threatened by and vulnerable to climate change. Concerning the national population exposed to current and future flood risks (i.e., 25-, 50-, 100-, and 200-year return period flood), Bangalore et al. (2016) estimated approximately 33–44% of today's population in Vietnam were exposed to flood risks, and these figures were projected to increase by around 4–27% depending on a combination of climate change scenarios and flood risks considered. In addition, Jevrejeva et al. (2018) documented that the annual sea flood costs without additional adaptation, expressed as percentages of country-level GDP, were estimated at approximately 7.2 and 8.9% under warming of 1.5 °C and 2.0 °C by 2100, respectively, thereby ranking Vietnam among the top 10 countries with the highest annual flood cost. Based on an up-to-the-minute report on Global Climate Risk Index 2021 (Eckstein et al. 2021), Vietnam was ranked 13 out of 180 countries most affected by extreme weather events during 2000–2019. Concerning physical exposure to flooding, Inter-Agency Standing Committee & the European Commission (2021) also ranked Vietnam (and Bangladesh) the 1st out of 191 countries.

Regarding the city-level analysis, coastal flooding due to sea-level rise also causes substantial socioeconomic damages and losses in major coastal cities in Vietnam. For example, Ho Chi Minh City and Hai Phong were ranked among the top 20 largest coastal cities, with populations and assets most likely to be severely affected by coastal flooding under different sea-level projections (Hanson et al. 2011; Hallegatte et al. 2013; Abadie et al. 2020). Hanson et al. (2011) documented that the number of current populations exposed to coastal flooding in Ho Chi Minh City (Hai Phong) was around 1.9 million (794 thousand) people, and these figures were also estimated to rise to approximately 9.2 million (4.7 million) people by the 2070s. Additionally, the monetary value of assets exposed to coastal flooding in Ho Chi Minh City was estimated at approximately 26.86 and 652.82 billion USD under the current and future conditions, respectively (Hanson et al. 2011). Abadie et al. (2020) further reported that the estimated values of annual average (expected) damages due to sea-level rise could reach 23.5 billion USD and 32.0 billion USD by 2050 and 161.9 billion USD and 250.1 billion USD by 2100 under high-end scenarios in Ho Chi Minh City and Hai Phong, respectively.

In hydrology and climatology, parametric slope-based (e.g., linear regression) and non-parametric rank-based (e.g., Mann-Kendall and Spearman's rho test) approaches are commonly used to identify temporal trend possibilities in a given time series. In comparison, the non-parametric methods do not need to encounter the fundamental assumption of normal distribution and are not sensitive to the presence of outlier and extreme values (i.e., skewed data). However, these statistically-based approaches still necessitate the independent structure of the considered records. In practical applications, most conventional hydro-meteorological variables and climate extremes indices are serially correlated, which is bound to cause invalid interpretation of trend existence (von Storch 1995; Hamed & Rao 1998; Kundzewicz & Robson 2000; Qian et al. 2019). There have been various additional techniques developed to deal with the effect of significant serial correlation on the performance of the classical Mann-Kendall test, such as pre-whitening (Kulkarni & von Storch 1995), trend-free pre-whitening (Yue et al. 2002; Önöz & Bayazit 2012), variance correction (Hamed & Rao 1998; Yue & Wang 2004), and block bootstrap (Kundzewicz & Robson 2000; Önöz & Bayazit 2012).

Another promising approach to characterize possible non-monotonic trend components was introduced by Şen (2012, 2014, 2017), namely innovative trend analysis methodology. This cutting-edge approach can be applied without considering any restrictive assumptions (i.e., normal distribution, serial independence, and sufficient length of observed records) (Şen 2012; Almazroui & Şen 2020) and offer intuitive interpretation on partial trends based on deliberate categorization according to expert views or critical threshold values concerning hazardous events (Dabanlı et al. 2016; Dabanlı & Şen 2018). Recently, Farrokhi et al. (2020) introduced an integrated framework combining the innovative trend analysis methodology with other intelligent algorithms to compare temporal trend possibilities found by simulated and observed datasets. Almazroui & Şen (2020) presented a comprehensive review and discussion on the use of these statistically-based and graphically-based approaches.

It is acknowledged that climate change and variability pose a significant threat to Vietnam, especially in low-lying coastal regions, thus requiring a sound understanding of spatiotemporal trends and variations in climate means and extremes over the country. Over the last few decades, there have been various salient case studies of long-term trends in temperature, precipitation, and extreme events. For example, Ngo-Duc (2014) quantified annual and seasonal trends in several climate elements and phenomena (i.e., 2 m temperature, rainfall, daily maximum wind speed, and several climate extremes indices) by applying the classical Mann-Kendall test and Sen's slope estimator to 23 coastal meteorological stations in Vietnam over 1960–2011. Nguyen et al. (2014) also employed these non-parametric statistical trend tests to estimate historical trends in surface temperature and rainfall in all climate sub-regions of Vietnam. Kien et al. (2019) further integrated a spatial interpolation technique (i.e., co-kriging) with the outputs found by the classical Mann-Kendall test and Sen's slope estimator to construct spatial distribution maps of temperature and rainfall trends in Vietnam from 1975–2014. Pham-Thanh et al. (2020) also presented long-term trends in total annual rainfall and several rainfall extremes indices from 1979–2019 and the possible difference between 1979–1998 and 1999–2019 by employing the Mann-Kendall test and Sen's slope method. Unlike previous case studies conducted based on observed records, Le et al. (2020) applied the original Mann-Kendall test and Sen's slope estimator to a satellite-based dataset to explore spatial patterns of air temperature, precipitation, and drought trends in Vietnam over the period 1989–2018.

However, most previous investigations addressed long-term trends mainly based on a single monotonic trend test for specified beginning and ending times. This traditional approach cannot clarify the nature of temporal changes (e.g., gradual or abrupt change) in a given hydro-meteorological record. Moreover, Zhang et al. (2010a) substantiated that adding (or removing) only one more observation to the analyzed records or changing the beginning (or ending) years of the considered period can affect the outputs of temporal trend tests in terms of trend direction, magnitude, and statistical significance. To tackle these difficulties, McCabe & Wolock (2002) and Zhang et al. (2010a) proposed an alternative approach, namely the multiple non-parametric statistical trend tests, which perform monotonic trend tests repeatedly for all possible periods of at least ten years in length during the whole analyzed period. This alternative approach can assess the sensitivity of hydro-meteorological trend test results to the beginning years, ending years, and record lengths, as demonstrated by Zhang et al. (2010b); Seo et al. (2012); Zhang et al. (2012); Obregón et al. (2014); Maleski & Martinez (2017).

In an attempt to offer a solid grounding in long-term trends in precipitation and temperature over Vietnam, the present study aimed to explore temporal trend patterns by applying the multiple non-parametric statistical trend tests (i.e., Sen's slope estimator and block bootstrapping Mann-Kendall tests) to the well-known high-resolution (0.5°×0.5°) gridded dataset, namely UDel_AirT_Precip Version 5.01. Moreover, this study proposed a modified procedure to comprehend spatiotemporal trend patterns by performing multiple trend tests to all latitude coordinates. This modified procedure is conducted by varying beginning and ending years through all successive 30-year periods during 1900–2017.

Vietnam, geographically extending between the latitudes 8°27′N – 23°23′N, is located in the tropical zone of the Northern Hemisphere. Vietnam's climate is mainly tropical and highly controlled by the South West and North East Monsoons (Suppiah & Thang 2014). According to the well-known world map of the Köppen-Geiger climate classification proposed by Peel et al. (2007), the general climate of Vietnam is categorized as tropical monsoon (Am), tropical savannah (Aw), and humid subtropical (Cwa) climate. Due to the combined effects of monsoon systems, complex topography, and latitudinal extent, Vietnam is subdivided commonly into seven climate sub-regions, such as North West (N1), North East (N2), North Plain (N3), North Central (N4), South Central (S1), Central Highland (S2), and South Plain (S3) (Ho et al. 2011; Nguyen et al. 2014; Nguyen-Xuan et al. 2016; Kien et al. 2019; Le et al. 2020). In the northern parts of Vietnam, there are four distinct seasons, including spring (February to early April), summer (late April to early September), autumn (late September to October), and winter (November to early February) (Ho et al. 2011). Meanwhile, the southern parts have only two seasons, i.e., wet season lasting typically from May to November and dry season.

The present study utilized the well-known high-resolution dataset, i.e., UDel_AirT_Precip Version 5.01, to explore the long-term spatiotemporal trend patterns of air temperature and precipitation anomalies relative to a standard reference period (1961–1990) in Vietnam. The 1961–1990 period was adopted according to a commonly used guideline for long-term climate change assessments (World Meteorological Organization 2017). The UDel_AirT_Precip Version 5.01 dataset is a high-resolution (0.5°×0.5°), monthly climatology, and land-only dataset spanning from 1900 to 2017 (Willmott & Matsuura 2018). Table 1 shows some statistical characteristics of annual air temperature and precipitation at each climate sub-region and the whole of Vietnam over the period 1961–1990. It is worth noting that these basic statistics were computed directly from the UDel_AirT_Precip Version 5.01 dataset by aggregating all grid cell values within each climate sub-region, which means that there may be several discrepancies compared to those calculated from observations.

Table 1

Descriptive statistics of annual air temperature and precipitation in seven climate sub-regions of Vietnam

Climate sub-regionsTemperature
Precipitation
Mean (°C)SD (°C)CV (%)Mean (mm)SD (mm)CV (%)
North West (N1) 19.7 1.6 8.02 1,749.8 343.9 19.7 
North East (N2) 21.0 2.3 10.94 1,705.2 337.6 19.8 
North Plain (N3) 23.1 1.3 5.46 1,681.2 310.0 18.4 
North Central (N4) 22.5 2.3 10.22 1,833.1 581.7 31.7 
South Central (S1) 25.0 2.0 8.08 1,676.3 562.0 33.5 
Central Highland (S2) 22.8 2.2 9.72 1,752.3 518.1 29.6 
South Plain (S3) 27.3 0.6 2.19 1,627.6 615.2 37.8 
Vietnam 23.1 3.0 13.0 1,722.8 498.0 28.9 
Climate sub-regionsTemperature
Precipitation
Mean (°C)SD (°C)CV (%)Mean (mm)SD (mm)CV (%)
North West (N1) 19.7 1.6 8.02 1,749.8 343.9 19.7 
North East (N2) 21.0 2.3 10.94 1,705.2 337.6 19.8 
North Plain (N3) 23.1 1.3 5.46 1,681.2 310.0 18.4 
North Central (N4) 22.5 2.3 10.22 1,833.1 581.7 31.7 
South Central (S1) 25.0 2.0 8.08 1,676.3 562.0 33.5 
Central Highland (S2) 22.8 2.2 9.72 1,752.3 518.1 29.6 
South Plain (S3) 27.3 0.6 2.19 1,627.6 615.2 37.8 
Vietnam 23.1 3.0 13.0 1,722.8 498.0 28.9 

SD is the standard deviation. CV is the coefficient of variation. These basic statistics were computed from the UDel_AirT_Precip Version 5.01 dataset for the period 1961–1990.

Methodological framework

Figure 1 depicts a brief description of the methodological framework employed in this study. Having selected the appropriate dataset (i.e., UDel_AirT_Precip Version 5.01 covering the whole extent of Vietnam), the process starts by computing annual anomalies relative to the standard reference period (1961–1990) for individual grid cells before aggregating by each climate sub-region as well as latitude coordinate. Then these annual time series are subjected to trend identification by employing the multiple non-parametric statistical trend tests with various beginning years, ending years, and record lengths. The following sub-sections describe more details about the multiple trend tests. In essence, the block bootstrapping Mann-Kendall tests and Sen's slope estimator, which indicate the statistical significance and magnitude of trends, respectively, are performed in a running manner for various periods. It is worth noting that the block bootstrapping Mann-Kendall tests are employed to eliminate the effect of significant serial correlation (Kundzewicz & Robson 2000; Önöz & Bayazit 2012). Finally, spatiotemporal trend patterns of air temperature and precipitation anomalies can be clarified by examining the outputs of multiple trend tests represented as an upper triangle graph or a Hovmöller-like diagram in case of grouping by climate sub-regions or latitude coordinates.

Figure 1

Schematic framework of research methodology.

Figure 1

Schematic framework of research methodology.

Close modal

Multiple non-parametric statistical trend tests

Traditionally, a period with a fixed beginning and ending year is predetermined, and (non-)parametric trend tests are applied to detect trend existence in a given hydro-meteorological record. This traditional application is pretty sensitive to the analyzed period. Therefore, this study employed the multiple non-parametric statistical trend tests proposed by McCabe & Wolock (2002) and Zhang et al. (2010a). A step-by-step procedure to implement this methodology was described in detail by Zhang et al. (2012). In essence, the fundamental basis of the multiple trend tests is iterating the non-parametric trend tests (e.g., Mann-Kendall test and Sen's slope estimator) with various beginning years, ending years, and record lengths accordingly.

Theoretically, the Mann-Kendall test statistic is a standard normal variate when the sample size is at least ten observations. Zhang et al. (2010a) suggested that the total number of repetitions, N, is expressed as follows: N=[(n−8)×(n−9)]/2, where n is the record length of the whole analyzed period. However, the World Meteorological Organization (2017) generally recommended utilizing the 30-year periods in climatological studies. Almazroui & Şen (2020) also stated that the statistically-based methods (e.g., Mann-Kendall test) necessitate at least 30 observations. Therefore, the sample sizes in this study are in the range of 30 to the maximum length of analyzed records when implementing the multiple trend tests. The present study identified temporal trends over 118 years (1900–2017), so the total number of repetitions, N, is 4,005 trend test calculations according to the slightly modified equation as follows: N=[(n−28)×(n−29)]/2.

After executing the total number of repetitions, the results of multiple trend tests are visualized graphically in the form of an upper triangle on a Cartesian coordinate system to identify temporal trend patterns. In particular, the sequence of beginning years is plotted on the abscissa axis against ending years on the ordinate axis. Then the outputs of multiple trend tests (i.e., p-values, Mann-Kendall test statistics, or trend slopes) can be displayed by a color gradient as the third dimension. McCabe & Wolock (2002) showed the number of sites with significant trends, while Zhang et al. (2010a), Zhang et al. (2010b) and Zhang et al. (2012) displayed the p-values of trend tests by plotting with graduated colors in combination with upward (downward) triangle symbols indicating significant trends. Similarly, the present study represented the outputs of Sen's slope estimator by a color gradient in combination with stippling, which implies significant trends at the significance level of 0.05 by applying the block bootstrapping Mann-Kendall test (Kundzewicz & Robson 2000; Önöz & Bayazit 2012). These upper triangle graphs are partly analogous to those illustrated by Obregón et al. (2014) and Maleski & Martinez (2017).

In addition, the 1:1 line and its parallel ones indicate estimated trends of 30 years and longer periods with different beginning and ending times. The most bottom-left and upper-right grid cells stand for the earliest and latest 30 years, respectively, while the most top-left one denotes the result for the entire analyzed period (i.e., 1900–2017). Based on these graphs, temporal trend patterns can be characterized in terms of beginning years, ending years, and record lengths (McCabe & Wolock 2002; Zhang et al. 2010a). In particular, inspecting values of different beginning (ending) years while fixing a specific ending (beginning) year reveals the sensitivity of trend tests to beginning (ending) years, and examining a given set of 45° lines yields potential impacts of different record lengths on the trend test results.

A modified procedure for clarifying spatiotemporal trend patterns

Concerning spatiotemporal trend analysis of hydrology and climatology, the common practice is to conduct (non-)parametric trend tests using all available data within the selected period at individual sites, and the trend test results (e.g., Mann-Kendall test statistics or trend slopes) are then displayed in maps over the study area. It is also possible to integrate the trend test results with a given spatial interpolation technique to generate continuous representations of spatiotemporal fields. However, such a traditional approach offers spatiotemporal trend patterns within only one specified period.

The present study suggested a modified procedure to enhance the characterization of long-term spatiotemporal trends in precipitation and temperature over Vietnam. In fact, this proposed procedure is inspired by the aforementioned multiple trend tests and the well-known Hovmöller diagram (Hovmöller 1949). The implementation of this proposed procedure starts by aggregating all gridded values grouped by individual latitudes (or longitudes in other cases). Then the multiple trend tests described above are applied to aggregated time series at each latitude coordinate. It is noted that the multiple trend tests are performed by varying beginning and ending years through all successive 30-year periods during 1900–2017. The sample size required for each calculation is set to 30 according to the general recommendation by the World Meteorological Organization (2017). Finally, the outputs of multiple trend tests are displayed by a color gradient as the third dimension (as mentioned above) in the form of a Hovmöller-like diagram, where the abscissa and ordinate axis represent latitude coordinates and all successive 30-year periods, respectively.

Trend patterns of annual air temperature anomalies in seven climate sub-regions of Vietnam

Figure 2 displays the results of multiple trend tests for annual air temperature anomalies during 1900–2017 over seven climate sub-regions in Vietnam. There is an apparent discrepancy in temporal trend patterns between the northern and southern parts of Vietnam. In particular, annual air temperature anomalies in the North West (N1) exhibited significant increasing trends commonly for the period with a beginning year after 1930 and an ending year after 2000, and the estimated trend slopes were in the range of approximately 0.006–0.042 °C/year.

Figure 2

Temporal trend patterns of air temperature anomalies relative to 1961–1990 in seven climate sub-regions in Vietnam during 1900–2017. The middle- and center-aligned subplot is the long-term means of annual air temperature during 1961–1990. Seven surrounding upper triangle graphs indicate the results of multiple trend tests for air temperature anomalies in corresponding climate sub-regions. The blue-red color palette represents estimated trend slopes by employing Sen's slope estimator. Stippling indicates significant trends at the significance level of 0.05 by applying the block bootstrapping Mann-Kendall test. The 1:1 line and its 45° parallel lines stand for trend test results with the record lengths of 30, 50, and 70 years.

Figure 2

Temporal trend patterns of air temperature anomalies relative to 1961–1990 in seven climate sub-regions in Vietnam during 1900–2017. The middle- and center-aligned subplot is the long-term means of annual air temperature during 1961–1990. Seven surrounding upper triangle graphs indicate the results of multiple trend tests for air temperature anomalies in corresponding climate sub-regions. The blue-red color palette represents estimated trend slopes by employing Sen's slope estimator. Stippling indicates significant trends at the significance level of 0.05 by applying the block bootstrapping Mann-Kendall test. The 1:1 line and its 45° parallel lines stand for trend test results with the record lengths of 30, 50, and 70 years.

Close modal

In the North East (N2) and North Plain (N3), the outputs of iterating multiple monotonic trends revealed a reversal of the cooling trends with warming ones from the 1950s-1960s to the present. Similar to the North West (N1), annual air temperature anomalies in the North East (N2) and North Plain (N3) also exposed a gradient of increasing trends throughout the last five decades, with the estimated trend slopes varying from approximately 0.011–0.028 °C/year. Meanwhile, temporal trend patterns of annual air temperature anomalies in the North Central (N4) were different from those found in other northern regions (i.e., N1, N2, and N3). In particular, the North Central (N4) exhibited few scattered significant increasing trends at the rate of around 0.005–0.032 °C/year, mainly found for the period with the beginning year of 1930–1950 and the last four decades as well.

In the southern part of Vietnam, the South Central (S1), Central Highland (S2), and South Plain (S3) exposed the same temporal trend patterns of annual air temperature anomalies. In particular, significant declining trends were found commonly for the period with a beginning year before 1930 and an ending year before 1965. Then the later periods experienced high domination of significant warming trends with the estimated trend slopes ranging from approximately 0.007–0.044 °C/year. It is worth noting that the South Central (S1) and Central Highland (S2) also exhibited a progressive warming trend over the last 4–5 decades analogous to most northern sub-regions, while the South Plain (N3) exhibited few scattered significant increasing trends during the same consider periods.

Considering the long-term trends in the annual air temperature anomalies for the whole analyzed period 1900–2017 (i.e., grid cell in the most top-left corner), only two sub-regions, i.e., South Central (S1) and Central Highland (S2), were characterized by significant increasing trends. Additionally, temporal trend patterns in the North Central (N4) were more sensitive to the beginning and ending years compared to those found in the remaining sub-regions. Moreover, taking a careful look at several grid cells in the most top-right corner of the North Plain (N3) and South Plain (S3), it is apparent that annual air temperature anomalies during the most recent 30-year period (1988–2017) were characterized by significant increasing trends as shown by the grid cell in the most top-right corner. However, its surrounding cells were not stippled, implying non-significant trends even though adding very few observations to the considered records. Generally, these findings highlight the superiority of repeatedly multiple non-parametric trend tests over a single one for clarifying temporal trend patterns of annual air temperature anomalies.

Trend patterns of annual air temperature anomalies at all latitude coordinates from the south to north of Vietnam

Figure 3 further delineates spatiotemporal trend patterns of annual air temperature anomalies from the south to north of Vietnam through all successive 30-year periods during 1900–2017. The outcomes pointed out opposite trends between the south and north of Vietnam found commonly during the periods with ending years before the 1940s. In particular, the south (extending between the latitudes 9.25–15.75 decimal degrees) experienced significant declining trends at the rate of around −0.016 °C/year to −0.037 °C/year, while the north (extending between the latitudes 17.25–23.25 decimal degrees) exhibited non-significant increases with the estimated trend slopes varying approximately from 0.007–0.020 °C/year.

Figure 3

Results of multiple trend tests for air temperature anomalies relative to 1961–1990 in all latitude coordinates for all successive 30-year periods during 1900–2017. The color gradient represents estimated trend slopes by applying Sen's slope estimator. Stippling indicates significant trends at the significance level of 0.05 by applying the block bootstrapping Mann-Kendall test.

Figure 3

Results of multiple trend tests for air temperature anomalies relative to 1961–1990 in all latitude coordinates for all successive 30-year periods during 1900–2017. The color gradient represents estimated trend slopes by applying Sen's slope estimator. Stippling indicates significant trends at the significance level of 0.05 by applying the block bootstrapping Mann-Kendall test.

Close modal

During the periods with ending years around the 1940s-1960s, most latitude coordinates showed cooling trends, and the highest declining ones were found in the regions extending between the latitudes 18.25–20.25 decimal degrees, with the estimated trend slopes ranging approximately from −0.011 °C/year to −0.044 °C/year. During the following periods with ending years around the 1965s-1980s, annual air temperature anomalies relative to 1961–1990 were characterized mainly by significant warming trends at the rate of around 0.017–0.043 °C/year over the regions extending between the latitudes 9.25–18.75 decimal degrees, while the farthest north of Vietnam experienced significant declining trends at the rate of around −0.012 °C/year to −0.025 °C/year. Finally, most latitude coordinates exhibited warming trends at the rate of approximately 0.008–0.036 °C/year during the last 4–5 decades with ending years after the 2000s till the present.

Trend patterns of annual precipitation anomalies in seven climate sub-regions of Vietnam

Concerning temporal trend patterns of precipitation anomalies expressed as percentage difference relative to 1961–1990, Figure 4 depicts the results of iterating multiple monotonic trend tests over seven climate sub-regions in Vietnam during 1900–2017. It is discernible that the vast majority of trend test calculations were dominated mainly by significant declining trends, with the estimated trend slopes ranging approximately from −0.11%/year to −2.88%/year. Meanwhile, the last 4–5 decades experienced significant increasing trends, notably found in the southern part of Vietnam.

Figure 4

Temporal trend patterns of precipitation anomalies relative to 1961–1990 in seven climate sub-regions in Vietnam during 1900–2017. The middle- and center-aligned subplot is the long-term means of annual precipitation totals during 1961–1990. Seven surrounding upper triangle graphs indicate the results of multiple trend tests for precipitation anomalies in corresponding climate sub-regions. The brown-blue color palette represents estimated trend slopes by employing Sen's slope estimator. Stippling indicates significant trends at the significance level of 0.05 by applying the block bootstrapping Mann-Kendall test. The 1:1 line and its 45° parallel lines stand for trend test results with the record lengths of 30, 50, and 70 years.

Figure 4

Temporal trend patterns of precipitation anomalies relative to 1961–1990 in seven climate sub-regions in Vietnam during 1900–2017. The middle- and center-aligned subplot is the long-term means of annual precipitation totals during 1961–1990. Seven surrounding upper triangle graphs indicate the results of multiple trend tests for precipitation anomalies in corresponding climate sub-regions. The brown-blue color palette represents estimated trend slopes by employing Sen's slope estimator. Stippling indicates significant trends at the significance level of 0.05 by applying the block bootstrapping Mann-Kendall test. The 1:1 line and its 45° parallel lines stand for trend test results with the record lengths of 30, 50, and 70 years.

Close modal

In the North West (N1), significant decreasing trends were found commonly for the period with the record lengths of 50 years or longer and a beginning year before 1945 and an ending year after 1995. In the North East (N2), North Plain (N3), and North Central (N4), precipitation anomalies exposed the same temporal trend patterns. In particular, significant positive trends were obtained mainly for the period with a beginning year before 1930 and an ending year before 1960. Then the later periods were dominated commonly by significant decreasing trends until the last 3–4 decades, when precipitation anomalies in the North East (N2), North Plain (N3), and North Central (N4) exhibited significant increasing trends at the rate of around 0.17–1.51%/year.

In the southern part of Vietnam, temporal trend patterns of precipitation anomalies in the South Central (S1), Central Highland (S2), and South Plain (S3) were highly similar to each other. These southern sub-regions mainly experienced significant negative trends throughout the analyzed periods until the recent 4–5 decades. In particular, precipitation anomalies in these southern sub-regions exhibited progressive increases at the rate of around 0.52–2.76%/year found commonly for the period with a beginning year after 1970 and an ending year after 2000.

As far as the long-term trends in precipitation anomalies for the whole analyzed period 1900–2017 are concerned (as shown in the grid cells in the most top-left corner), only two sub-regions, i.e., Central Highland (S2) and South Plain (S3) were characterized by significant decreasing trends. On the other hand, although North West (N1) and North East (N2) did not undergo significant trends for 1900–2017, removing very few observations from the considered records could result in significant trends, thereby implying that considering a single long-term time series is not sufficient for understanding trend possibilities. Moreover, comparing the most recent 30-year period (1988–2017) and its slightly different periods in terms of beginning years, ending years, and record lengths, the statistical significance of detected trends is not always similar to each other, as shown in several grid cells in the most top-right corner, notably found in North East (N2), North Plain (N3), South Central (S1), and Central Highland (S2). These findings indicate that the individual trend tests are sensitive to the analyzed records, thus making the application of multiple monotonic trend tests more reliable.

Trend patterns of annual precipitation anomalies at all latitude coordinates from the south to north of Vietnam

Figure 5 further presents the outputs of iterating multiple monotonic trend tests conducted at individual latitude coordinates from the south to north of Vietnam for all successive 30-year periods during 1900–2017. During the periods with ending years before the 1960s, the regions extending between the latitudes 9.25–16.75 decimal degrees experienced mixed patterns of slight increases and decreases in precipitation anomalies, while the areas extending between the latitudes 17.75–20.25 decimal degrees exhibited significant increasing trends with the estimated trend slopes varying approximately from 0.20–1.11%/year. Then there was high domination of significant decreasing trends in precipitation anomalies during the periods with ending years around the 1960s-1980s, found mainly in the regions extending between the latitudes 9.25–14.25 and 19.75–22.75 decimal degrees, with the estimated trend slopes ranging approximately from −0.37%/year to −1.15%/year.

Figure 5

Results of multiple trend tests for precipitation anomalies relative to 1961–1990 in all latitude coordinates for all successive 30-year periods during 1900–2017. The color gradient represents estimated trend slopes by applying Sen's slope estimator. Stippling indicates significant trends at the significance level of 0.05 by applying the block bootstrapping Mann-Kendall test.

Figure 5

Results of multiple trend tests for precipitation anomalies relative to 1961–1990 in all latitude coordinates for all successive 30-year periods during 1900–2017. The color gradient represents estimated trend slopes by applying Sen's slope estimator. Stippling indicates significant trends at the significance level of 0.05 by applying the block bootstrapping Mann-Kendall test.

Close modal

It is apparent that most latitude coordinates exposed significant decreasing trends at the rate of around −0.36%/year to −3.10%/year during the periods with ending years around the 1990s-2005s. Meanwhile, precipitation anomalies at most latitude coordinates were characterized mainly by significant positive trends at the rate of around 0.70–3.42%/year during the last 3–4 decades with ending years after the 2010s to the present. It is worth noting that the southern part of Vietnam, extending between the latitudes 9.25–13.25 decimal degrees, experienced the highest changes (both increases and decreases) in precipitation anomalies compared to other latitude coordinates.

On the evidence of spatiotemporal trend patterns discussed above, it can be concluded that air temperature and precipitation in Vietnam have changed substantially during 1900–2017. Moreover, there were progressive increases in air temperature and precipitation anomalies relative to 1961–1990 throughout the last 4–5 decades. To a large extent, these findings are consistent with previous in-depth investigations based on observed and satellite-based datasets (Nguyen et al. 2014; Suppiah & Thang 2014; Kien et al. 2019; Le et al. 2020; Pham-Thanh et al. 2020).

Concerning historical trends in temperature, Nguyen et al. (2014) pointed out significant warming trends at the rate of around 0.22±0.11 °C/decade to 0.36±0.13 °C/decade over the 40-year period (1971–2010). Suppiah & Thang (2014) also documented evidence of warming trends of 0.15–0.35 °C/decade during the period 1961–2011. Kien et al. (2019) indicated significant positive trends in air temperature found in over half of considered sites located proportionally to the whole extent of Vietnam over 1975–2014. Similarly, Le et al. (2020) showed approximately 40% of the total land area of Vietnam experiencing significant warming trends over 1989–2018 based on analyzing the MERRA-2 dataset.

Regarding trend possibilities in annual rainfall during recent decades in Vietnam, Nguyen et al. (2014) revealed that negative trends were found in five out of seven climate sub-regions (i.e., N1, N2, N3, N4, and S2) for the period 1971–2010, but most of these detected trends were statistically insignificant, except the N3. Suppiah & Thang (2014) found high increases in rainfall over the South Central, Central Highland, and some southernmost stations in the North Central during 1961–2011, while the remaining sub-regions were characterized mainly by declining trends. Likewise, Kien et al. (2019) and Pham-Thanh et al. (2020) indicated mixed patterns of both decreases and increases at analyzed meteorological sites over the last four decades, but most of these estimated trends were detected insignificantly. Le et al. (2020) pointed out clear positive trends in annual rainfall over the northern portions of Vietnam (i.e., N1, N2, and N3) while declining trends were found mainly in the central and southern parts during 1989–2018. Moreover, the proportion of significant increases and decreases was around 25.7–26.5% and 11.4–11.8% of the total land area of Vietnam, respectively (Le et al. 2020).

In general, all findings found in this study and previous ones indicated positive trends in annual air temperature over seven climate sub-regions in Vietnam, and the magnitudes of warming trends were highly similar to each other. Unlike temperature, there are several discrepancies in temporal rainfall trends, which can be attributed partly to different considered periods and data sources (e.g., observed, satellite-based, and high-resolution gridded datasets). It is worth noting that most previous investigations utilized the classical Mann-Kendall test, which may lead to misinterpretation of trend existence when there is temporal dependency in considered records. Meanwhile, the present study employed the block bootstrapping Mann-Kendall test, which takes the effect of significant serial correlation into consideration.

In comparison, previous studies were carried out for a fixed period with a predetermined beginning and ending time. Meanwhile, the present study performed monotonic trend tests repeatedly with various beginning and ending years for all possible periods of at least 30 years in length during 1900–2017, which yields more details about temporal trend possibilities. It is also evident that a single trend test is quite sensitive to the considered period in terms of beginning years, ending years, and record lengths. Thus, it is highly advisable to employ multiple non-parametric statistical trend tests in further temporal trend analysis.

The present study clarified spatiotemporal trend patterns of climate means in Vietnam. Further investigations should delve deeper into climate extremes based on an internationally coordinated core set of climate extremes indices. Additionally, spatiotemporal changes in drought characteristics (i.e., intensity, frequency, event, duration, and magnitude) at different time scales should be further examined based on a set of (multi-)scalar drought indices (e.g., Standardized Precipitation Index, Standardized Precipitation Evapotranspiration Index, Palmer Drought Severity Index), since several regions have undergone rainfall reduction within a certain considered period. To do this, one can utilize observed daily records or the next version of the Vietnam Gridded Precipitation (VnGP) Dataset (Nguyen-Xuan et al. 2016). Another essential concern is to examine potential links between large-scale drivers and inter-annual variability of climate means and extremes. One promising approach is to conduct correlation analysis of climate means/extremes with a given index characterizing large-scale ocean-atmospheric circulation (e.g., Oceanic Niño Index or Dipole Mode Index) in a running manner for various periods.

In Vietnam, the northern regions have four distinct seasons (i.e., spring, summer, autumn, and winter), while rainy and dry seasons are dominant features in the southern ones. This study was carried out on an annual basis, but the same methodology can be further applied to seasonal time series to assess seasonal trends. Apart from understanding historical variations and trends in hydro-meteorological variables, it is essential to explore potential future changes based on a wide range of the state-of-the-art Climate Model Intercomparison Project Phase 6 projections. To investigate possible impacts of climate change and variability on local livelihood and biophysical systems, one can employ a hybrid approach introduced by Anaraki et al. (2021), which integrates many machine learning methods to assess the uncertainty of flood risk (or other climate-induced risks) concerning emission scenarios, downscaling methods, hydrological models, and combined effects of all these elements.

The modified procedure is most suitable for analyzing spatiotemporal trend patterns in regions or nations that cover a narrow longitudinal extent but a broad latitudinal one and vice versa. The modified procedure is also conducive to countries covering several types of the Köppen-Geiger climate classification. This proposed Hovmöller-like diagram is expected to be a fruitful complement to spatiotemporal trend analysis. In addition, the present study employed multiple monotonic trend tests by iterating Sen's slope estimator and block bootstrapping Mann-Kendall tests. Alternatively, one can perform trend slope computation by applying the Innovative Trend Analysis methodology proposed by Şen (2017).

The global threat of climate change is bound to cause a wide range of far-reaching implications for all walks of life, thus posing various serious challenges to low- and middle-income economies. Vietnam is among the top countries characterized by a high level of exposure and vulnerability to climate-induced risks, therefore requiring in-depth comprehension of long-term trends in temperature and precipitation over the nation. Hence, the present study was carried out to explore the nature of temporal changes in air temperature and precipitation anomalies relative to 1961–1990 by applying the multiple non-parametric statistical trend tests to the UDel_AirT_Precip Version 5.01 dataset spanning 1900–2017. Moreover, a modified procedure was suggested to clarify spatiotemporal trend patterns of air temperature and precipitation anomalies at all latitude coordinates from the south to north of Vietnam for all possible successive 30-year periods.

The outcomes indicate a marked difference in temporal trend patterns of annual air temperature anomalies between the northern and southern parts of Vietnam, but all climate sub-regions exhibited significant warming trends at the rate of around 0.010–0.042 °C/year over the last 4–5 decades. Concerning temporal trend patterns of precipitation anomalies, there was high domination of declining trends throughout the whole analyzed period until the last 3–5 decades, when Vietnam has become wetter, especially in the southern part, with the estimated trend slopes ranging from approximately 0.29–2.76%/year.

It is evident to conclude that the individual trend tests are sensitive to the considered period in terms of beginning years, ending years, and record lengths, including the most recent 30-year period (i.e., 1988–2017) or the longer and longest ones (i.e., 1900–2017) as discussed above. On this basis, multiple trend tests with various beginning years, ending years, and record lengths are preferable to identify spatiotemporal trend possibilities. Additionally, the proposed procedure, represented as a Hovmöller-like diagram, can be applied to a larger region (e.g., Southeast Asia or other IPCC climate reference regions) to quantify regional climate variability and its potential effects on local biophysical systems.

The authors thank the editor and reviewers for their constructive comments and supportive suggestions. The authors would like to express our heartfelt thanks to Prof. Cort J. Willmott and Dr Kenji Matsuura for developing, maintaining, and making the UDel_AirT_Precip Version 5.01 dataset available publicly.

Dang Nguyen Dong Phuong was involved in the conceptualization, methodology, data collection, formal analysis, original draft preparation, review, and editing. Nguyen Le Tan Dat was involved in the methodology, data collection, formal analysis, original draft preparation. Nguyen Kim Loi was involved in the conceptualization, methodology, review, editing, resources, and supervision.

The authors declare no competing interests.

All relevant data are included in the paper or its Supplementary Information.

Abadie
L. M.
,
Jackson
L. P.
,
Sainz de Murieta
E.
,
Jevrejeva
S.
&
Galarraga
I.
2020
Comparing urban coastal flood risk in 136 cities under two alternative sea-level projections: RCP 8.5 and an expert opinion-based high-end scenario
.
Ocean & Coastal Management
193
,
105249
.
https://doi.org/10.1016/j.ocecoaman.2020.105249
.
Almazroui
M.
&
Şen
Z.
2020
Trend analyses methodologies in hydro-meteorological records
.
Earth Systems and Environment
4
(
4
),
713
738
.
https://doi.org/10.1007/s41748-020-00190-6
.
Anaraki
M. V.
,
Farzin
S.
,
Mousavi
S.-F.
&
Karami
H.
2021
Uncertainty analysis of climate change impacts on flood frequency by using hybrid machine learning methods
.
Water Resources Management
35
(
1
),
199
223
.
https://doi.org/10.1007/s11269-020-02719-w
.
Bangalore
M.
,
Smith
A.
&
Veldkamp
T.
2016
Exposure to Floods, Climate Change, and Poverty in Vietnam
.
Policy Research Working Paper 7765. The World Bank
.
Dabanlı
İ.
&
Şen
Z.
2018
Classical and innovative-Şen trend assessment under climate change perspective
.
International Journal of Global Warming
15
(
1
),
19
37
.
https://doi.org/10.1504/IJGW.2018.091951
.
Dabanlı
İ.
,
Şen
Z.
,
Yeleğen
M. Ö.
,
Şişman
E.
,
Selek
B.
&
Güçlü
Y. S.
2016
Trend assessment by the Innovative-Şen method
.
Water Resources Management
30
(
14
),
5193
5203
.
https://doi.org/10.1007/s11269-016-1478-4
.
Eckstein
D.
,
Künzel
V.
&
Schäfer
L.
2021
Global Climate Risk Index 2021, Who Suffers Most From Extreme Weather Events? Weather-Related Loss Events in 2019 and 2000–2019
.
Germanwatch e.V.
,
Germany
.
Available from: www.germanwatch.org/en/cri (accessed 19 October 2021).
Hallegatte
S.
,
Green
C.
,
Nicholls
R. J.
&
Corfee-Morlot
J.
2013
Future flood losses in major coastal cities
.
Nature Climate Change
3
(
9
),
802
806
.
https://doi.org/10.1038/nclimate1979
.
Hamed
K. H.
&
Rao
A. R.
1998
A modified Mann-Kendall trend test for autocorrelated data
.
Journal of Hydrology
204
(
1–4
),
182
196
.
https://doi.org/10.1016/S0022-1694(97)00125-X
.
Hanson
S.
,
Nicholls
R.
,
Ranger
N.
,
Hallegatte
S.
,
Corfee-Morlot
J.
,
Herweijer
C.
&
Chateau
J.
2011
A global ranking of port cities with high exposure to climate extremes
.
Climatic Change
104
(
1
),
89
111
.
https://doi.org/10.1007/s10584-010-9977-4
.
Ho
T. M. H.
,
Phan
V. T.
,
Le
N. Q.
&
Nguyen
Q. T.
2011
Extreme climatic events over Vietnam from ­observational data and RegCM3 projections
.
Climate Research
49
(
2
),
87
100
.
https://doi.org/10.3354/cr01021
.
Hovmöller
E.
1949
The trough-and-ridge diagram
.
Tellus
1
(
2
),
62
66
.
https://doi.org/10.1111/j.2153-3490.1949.tb01260.x
.
Inter-Agency Standing Committee and the European Commission
2021
INFORM REPORT 2021; Shared Evidence for Managing Crises and Disasters, EUR 30754 EN
.
Publications Office of the European Union
,
Luxembourg
.
Jevrejeva
S.
,
Jackson
L. P.
,
Grinsted
A.
,
Lincke
D.
&
Marzeion
B.
2018
Flood damage costs under the sea level rise with warming of 1.5°C and 2°C
.
Environmental Research Letters
13
(
7
),
074014
.
https://doi.org/10.1088/1748-9326/aacc76
.
Kien
N. D.
,
Ancev
T.
&
Randall
A.
2019
Evidence of climatic change in Vietnam: some implications for agricultural production
.
Journal of Environmental Management
231
,
524
545
.
https://doi.org/10.1016/j.jenvman.2018.10.011
.
Kulkarni
A.
&
von Storch
H.
1995
Monte Carlo experiments on the effect of serial correlation on the Mann-Kendall test of trend
.
Meteorologische Zeitschrift
4
(
2
),
82
85
.
10.1127/metz/4/1992/82
.
Kundzewicz
Z. W.
&
Robson
A. J.
2000
Detecting Trend and Other Changes in Hydrological Data
.
World Climate Programme – Water, World Climate Programme Data and Monitoring, WCDMP-45, WMO/TD no. 1013, World Meteorological Organization
,
Geneva, Switzerland
.
Le
M.-H.
,
Kim
H.
,
Moon
H.
,
Zhang
R.
,
Lakshmi
V.
&
Nguyen
L.-B.
2020
Assessment of drought conditions over Vietnam using standardized precipitation evapotranspiration index, MERRA-2 re-analysis, and dynamic land cover
.
Journal of Hydrology: Regional Studies
32
,
100767
.
https://doi.org/10.1016/j.ejrh.2020.100767
.
Maleski
J. J.
&
Martinez
C. J.
2017
Historical trends in precipitation, temperature and drought in the Alabama–Coosa–Tallapoosa and Apalachicola–Chattahoochee–Flint river basins
.
International Journal of Climatology
37
(
2
),
583
595
.
https://doi.org/10.1002/joc.4723
.
McCabe
G. J.
&
Wolock
D. M.
2002
A step increase in streamflow in the conterminous United States
.
Geophysical Research Letters
29
(
24
),
38–31
38–34
.
https://doi.org/10.1029/2002GL015999
.
Ngo-Duc
T.
2014
Climate Change in the Coastal Regions of Vietnam
. In:
Coastal Disasters and Climate Change in Vietnam
(
Thao
N. D.
,
Takagi
H.
&
Esteban
M.
, eds.).
Elsevier
,
Oxford
, pp.
175
198
.
https://doi.org/10.1016/B978-0-12-800007-6.00008-3
.
Nguyen
D. Q.
,
Renwick
J.
&
McGregor
J.
2014
Variations of surface temperature and rainfall in Vietnam from 1971 to 2010
.
International Journal of Climatology
34
(
1
),
249
264
.
https://doi.org/10.1002/joc.3684
.
Nguyen-Xuan
T.
,
Ngo-Duc
T.
,
Kamimera
H.
,
Trinh-Tuan
L.
,
Matsumoto
J.
,
Inoue
T.
&
Phan-Van
T.
2016
The Vietnam gridded precipitation (VnGP) dataset: construction and validation
.
Sola
12
,
291
296
.
https://doi.org/10.2151/sola.2016-057
.
Obregón
G. O.
,
Marengo
J. A.
&
Nobre
C. A.
2014
Rainfall and climate variability: long-term trends in the Metropolitan Area of São Paulo in the 20th century
.
Climate Research
61
(
2
),
93
107
.
https://doi.org/10.3354/cr01241
.
Önöz
B.
&
Bayazit
M.
2012
Block bootstrap for Mann–Kendall trend test of serially dependent data
.
Hydrological Processes
26
(
23
),
3552
3560
.
https://doi.org/10.1002/hyp.8438
.
Peel
M. C.
,
Finlayson
B. L.
&
McMahon
T. A.
2007
Updated world map of the Köppen-Geiger climate classification
.
Hydrology and Earth System Sciences
4
(
2
),
439
473
.
https://doi.org/10.5194/hess-11-1633-2007
.
Pham-Thanh
H.
,
Ngo-Duc
T.
,
Matsumoto
J.
,
Phan-Van
T.
&
Vo-Van
H.
2020
Rainfall trends in Vietnam and their associations with tropical cyclones during 1979–2019
.
Sola
16
,
169
174
.
https://doi.org/10.2151/sola.2020-029
.
Qian
C.
,
Zhang
X.
&
Li
Z.
2019
Linear trends in temperature extremes in China, with an emphasis on non-Gaussian and serially dependent characteristics
.
Climate Dynamics
53
(
1–2
),
533
550
.
https://doi.org/10.1007/s00382-018-4600-x
.
Şen
Z.
2012
Innovative trend analysis methodology
.
Journal of Hydrologic Engineering
17
(
9
),
1042
1046
.
https://doi.org/10.1061/(ASCE)HE.1943-5584.0000556
.
Şen
Z.
2014
Trend identification simulation and application
.
Journal of Hydrologic Engineering
19
(
3
),
635
642
.
https://doi.org/10.1061/(ASCE)HE.1943-5584.0000811
.
Şen
Z.
2017
Innovative trend significance test and applications
.
Theoretical and Applied Climatology
127
(
3–4
),
939
947
.
https://doi.org/10.1007/s00704-015-1681-x
.
Seo
L.
,
Kim
T.-W.
&
Kwon
H.-H.
2012
Investigation of trend variations in annual maximum rainfalls in South Korea
.
KSCE Journal of Civil Engineering
16
(
2
),
215
221
.
https://doi.org/10.1007/s12205-012-0004-3
.
Suppiah
R.
,
Thang
N.
2014
Observed climate variability of Vietnam
. In:
High-resolution Climate Projections for Vietnam: Technical Report
(
Katzfey
J.
,
McGregor
J.
&
Suppiah
R.
, eds.).
CSIRO
,
Australia
, pp.
18
29
.
von Storch
H.
1995
Misuses of statistical analysis in climate research
. In:
Analysis of Climate Variability: Applications of Statistical Techniques
(
von Storch
H.
&
Navarra
A.
, eds.).
Springer
,
New York
, pp.
11
26
.
https://doi.org/10.1007/978-3-662-03744-7_2
.
Willmott
C. J.
&
Matsuura
K.
2018
Terrestrial Air Temperature and Precipitation: Monthly and Annual Time Series (1900–2017)
. .
World Meteorological Organization
2017
WMO Guidelines on the Calculation of Climate Normals (WMO-No. 1203)
.
World Meteorological Organization
,
Geneva
.
Yue
S.
&
Wang
C. Y.
2004
The Mann-Kendall test modified by effective sample size to detect trend in serially correlated hydrological series
.
Water Resources Management
18
(
3
),
201
218
.
https://doi.org/10.1023/B:WARM.0000043140.61082.60
.
Yue
S.
,
Pilon
P.
,
Phinney
B.
&
Cavadias
G.
2002
The influence of autocorrelation on the ability to detect trend in hydrological series
.
Hydrological Processes
16
(
9
),
1807
1829
.
https://doi.org/10.1002/hyp.1095
.
Zhang
Z.
,
Dehoff
A. D.
&
Pody
R. D.
2010a
New approach to identify trend pattern of streamflows
.
Journal of Hydrologic Engineering
15
(
3
),
244
248
.
https://doi.org/10.1061/(ASCE)HE.1943-5584.0000179
.
Zhang
Z.
,
Dehoff
A. D.
,
Pody
R. D.
&
Balay
J. W.
2010b
Detection of streamflow change in the susquehanna river basin
.
Water Resources Management
24
(
10
),
1947
1964
.
https://doi.org/10.1007/s11269-009-9532-0
.
Zhang
Z.
,
Pody
R. D.
,
Dehoff
A. D.
,
Balay
J. W.
2012
Analysis of Streamflow Trend in the Susquehanna River Basin, USA
. In:
Hydrologic Time Series Analysis: Theory and Practice
(
Machiwal
D.
&
Jha
M. K.
, eds.).
Springer
,
Netherlands, Dordrecht
, pp.
181
200
.
https://doi.org/10.1007/978-94-007-1861-6_9
This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying, adaptation and redistribution, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/).