Comparative evaluation of 24 reference evapotranspiration equations applied on an evergreen-broadleaved forest

Evapotranspiration (ET) is an important component of both the water and energy cycles. ET is used in agricultural and forest hydrometeorology and in urban planning. It is also used in many rainfall–runoff and ecosystem models (Vörösmarty et al. 1998; Hay & McCabe 2002; Oudin et al. 2005a, 2005b), even for the estimation of ecosystem productivity (Currie 1991). ET also affects significantly regional water availability and use (Zhang et al. 2001; Sun et al. 2006). In order to carry out a long-term study of hydrological, environmental, and ecological processes, reliable estimates and/or accurate measurements of ET are required (Rosenberry et al. 2007; Tabari et al. 2011).

The concept of ‘reference evapotranspiration’ (ET₀) was defined by Doorenbos & Pruitt (1977) as ‘the rate of evapotranspiration from an extensive surface of 8 to 15 cm tall, green grass cover of uniform height, actively growing, completely shading the ground and not short of water’. Allen et al. (1998) evolved an ET₀ definition based on a reference surface (a hypothetical grass and/or alfalfa reference crop) to define unique evaporation parameters for each crop and growth stage. Hence, the FAO Expert Consultation on Revision of FAO Methodologies for Crop Water Requirements accepted the following unambiguous definition for the reference surface: ‘A hypothetical crop with an assumed height of 0.12 m having a surface resistance of 70 s m⁻¹ and an albedo of 0.23, closely resembling the evaporation of an extension surface of green grass of uniform height, actively growing and adequately watered’ (Allen et al. 1998). The method proposed by Allen et al. (1998) was the FAO56 Penman–Monteith (ET_P-M) equation, which was accepted worldwide as the best estimator of ET₀ in many regions and in different weather conditions (Droogers & Allen 2002; Xu & Singh 2002; Oudin et al. 2005a, 2005b; Alexandris et al. 2006; Gavilán et al. 2006; Tabari et al. 2011; Rahimikhoob et al. 2012; Khoshravesh et al. 2015; Valipour 2015a, 2015b).

As ET₀ is considered to be the most difficult component to estimate, the wealth of ET₀ methods and empirical equations proposed by many researchers have certain strong points and limitations depending on the methods' applications and assumptions (Rana & Katerji 2000; Grismer et al. 2002; Valipour 2014a, 2014b). Additionally, past studies at various scales proved that different ET₀ methods gave widely different values at particular locations (Federer et al. 1996; Vörösmarty et al. 1998). This means that all these methods cannot be used globally as they need calibration for regional application (Kolka & Wolf 1998; Grismer et al. 2002; Xu & Singh 2002; Rosenberry et al. 2004; Lu et al. 2005; Tabari et al. 2011; Rahimikhoob et al. 2012; Xu et al. 2013; Bogawski & Bednorz 2014; Samaras et al. 2014; Valipour & Eslamian 2014; Valipour 2015c). The selection of the appropriate method based on the availability of data, its cost, estimation accuracy, operational time and space scales, is challenging.

There are many methods in the literature used for the estimation of ET₀. These methods can be grouped into categories depending on the variables needed for input. The main categories reported in the literature are: mass-transfer, temperature-based, radiation-based, pan-evaporation, and combination. Many researchers reported overviews by using these methods and categories (Jensen et al. 1990; Xu & Singh 2002; Rosenberry et al. 2004, 2007; Oudin et al. 2005a; Alexandris et al. 2008; Trajkovic & Kolakovic 2009; Tabari et al. 2011; Xystrakis & Matzarakis 2011; Xu et al. 2013; Valipour 2014c) in different areas and environments.

It would be really interesting to investigate the performance of the ET₀ methods in a forested area because: (1) forests affect the climatic status variability since they influence the hydrological and carbon cycles at regional and global scale (Houghton 1991; Musselman & Fox 1991; Nepstad et al. 1994) and (2) it is well established that forested sites and catchments have higher ET rates than grassed catchments (Zhang et al. 2001). In the literature there is a lack of studies dealing with ET₀ in forests. This is due to the costly equipment requirements (lysimeters, eddy covariance towers, etc.) and to the fact that the main interest of the scientific community focuses on agriculture (Fisher et al. 2005; Alexandris et al. 2008).

Such studies have been performed mainly in coniferous species. Indicatively, McNaughton & Black (1973) measured the ET in a Douglas fir forest for 18 days and came up with a proposed estimation method. Scholl (1976) determined the ET in a Chaparral stand. Spittlehouse & Black (1980) used the Bowen ratio/energy balance method to measure the ET of a thinned Douglas fir forest. Riekerk (1985) measured the ET of a young splash pine stand (Pinus elliottii) with lysimeters for 2 years. Stannard (1993) evaluated three ET models in a sparsely vegetated, semi-arid rangeland. Federer et al. (1996) used specific coefficients to estimate the potential evapotranspiration (PET) in different forested areas. Farahani & Ahuja (1996) worked in partial canopy/residue-covered fields. Kolka & Wolf (1998) modified the Thornthwaite model in order to estimate the actual ET in 29 forested sites. Fisher et al. (2005) compared five models of PET in a mixed (dominant species Pinus ponderosa) coniferous forest. Ha et al. (2015) worked in semi-arid high-elevation disturbed ponderosa pine forests and compared ET between eddy covariance measurements and meteorological and remote sensing-based models.

To our knowledge, except for Gebhart et al. (2012) who studied some temperature-based and radiation-based methods in northern Greece, there have been no reports for comparative evaluation of the behavior of ET₀ methods in forested areas of Greece. Other studies conducted in Greece used meteorological data from the Greek National Meteorological Service (Xystrakis & Matzarakis 2011; Samaras et al. 2014) exclusively for urban and agricultural areas. The meteorological stations that provided the data did not always follow the protocols imposed by the FAO (Alexandris et al. 2013). Moreover, there has not been such a study in the Mediterranean forests containing evergreen sclerophyllous broadleaved species.

For the above reasons, the main objective of this study was to test and evaluate the accuracy of different ET₀ estimation equations, taking into account the data requirements for each model and making the assumptions that: (1) the ET_P-M model is the best estimator for the ET₀ and (2) the environmental conditions of the site approximate the conditions for the application of the ET_P-M model. The 24 selected equations are very common, extensively used in other studies, and represent the four main categories (mass-transfer, temperature-based, radiation-based, and combination).

The models tested in this work produce ET₀ or PET estimates above ground covered with grass in a Mediterranean forest environment in Greece. The results will be useful to other researchers for incorporating them as input into hydrological, environmental, and soil models applied on similar Mediterranean vegetative and climatic conditions.

The study was carried out in a small experimental forest watershed (1.23 km²) covered by evergreen sclerophyllous broadleaved vegetation (maquis vegetation) in Western Greece close to Varetada village (Figure 1). This is a multi-layer, dense coppice forest with canopy closure 1.2–1.3 (tree canopies overlap). The understorey is dominated by Phillyrea latifolia, Arbutus unedo, and Erica arborea. Sporadic stems of Cercis siliquastrum and Erica verticilata are also present. In the upper storey there is a number of Quercus ilex stems distributed almost uniformly (Baloutsos et al. 2009). The height of all species varies from 4 to 15 m. The terrain is hilly and the soil is a Haplic Luvisol one (FAO 1988) and its parent material is flysch.

The site receives a mean annual amount of precipitation of 1,174 mm in the form of rain which ranges from 696 to 2,230 mm, as calculated from climatic data of the period 1996–2012. The wettest months are October (161 mm) and November (130.8 mm) and the driest ones July (46 mm) and June (46.5 mm). The mean annual air temperature is 15.6 °C. The coldest month is January with a mean monthly value of 7.1 °C with August being the hottest (25.5 °C) one. The mean annual relative humidity is 67.5% with an average of 80% during December and 59% during June. The average annual wind speed is 1.9 m/s with a monthly average value of 2.4 m/s during July and 1.5 m/s during December.

The wider area is classified in the Csa climatic type according to the Köppen–Geiger updated world map (Kottek et al. 2006), which shows seasonal variability (warm temperate rainy climates with mild winters and very hot dry summers).

The particular site was selected for forest research purposes for two reasons: (1) the maquis vegetation is one of the most representative vegetations in Mediterranean forest ecosystems and (2) the watershed is a long way from urban and industrial areas, so it is not likely that it receives any kind of pollution. Additionally, the entire existing forest has not been managed by the local Forest Service for over 40 years; therefore, there has not been any land use change for the same period of time.

The meteorological data were collected from an automatic meteorological station (latitude 38 ° 50′ 35″, longitude 21 ° 18′ 25″, elevation 332 m a.s.l.) installed in a natural forest opening inside the watershed. The main advantage of the location and the ground vegetation of the meteorological station is that they meet the specifications imposed by the FAO to avoid (as much as possible) significant and systematic cumulative errors in determining ET₀ (from 27% up to 47% during the warm season – Alexandris et al. 2013). Meteorological variables, such as air temperature (T_aver, T_max, T_min), relative humidity (RH_aver, RH_max, RH_min), solar radiation (R_s), wind speed (u₂), and precipitation (PR) were continuously recorded for a time period of 17 years (1996–2012). All sensors were set at 2 m above the ground level except for the rain gauge which was at a height of 1.3 m. The sampling period for all the monitored variables was set up to 15 min and the collected data were stored in a digital datalogger connected to the sensors. The data were periodically downloaded to be summarized and provide hourly, daily, monthly, and annual averages. These values constituted the input data used for the estimation of the daily values of ET₀ in all of the equations.

For the present study, daily data covering the 87.5% of the total length of the 17-year period (5,433 days) were used. The existing gaps (12.5% or 777 daily values: 356 in winters, 114 in springs, 100 in summers, and 207 in autumns) were randomly distributed and were excluded from the statistical analysis, since any gap filling could possibly affect the reliability of the results. The gaps were due to the lack of measurements of some parameters that made the application of the ET_P-M equation prohibitory. In some cases, some of the methods included in this study estimated a negative ET₀ daily value. These values were also excluded from the statistical analysis. A summary of the notations, definitions, and the units of the symbols used are shown in Table 1.

The 24 different equations used in this study were categorized in the following groups: five mass-transfer (Albrecht, Mahringer, Penman, Romanenko, and WMO), four combination (Copais, Solar Thermal Unit, Valiantzas (1) and (2)), ten radiation-based (Abtew, Caprio, De Bruin–Keijman, FAO24 Radiation, Jensen–Haise, Hansen, Makkink, McGuiness–Bordne, Priestley–Taylor and Turc), and five temperature-based (Hargreaves original, Hargreaves–Samani, two modified Hargreaves–Samani, and modified Thornthwaite) methods. The formulas of the equations are presented in Table 2 along with their references. The conversion of the units is in agreement with the units shown in Table 1.

There are many widely used statistical indices and coefficients to evaluate the systematic quantification of the accuracy of compared models (Willmott 1982; Berengena & Gavilán 2005; Alexandris et al. 2008; Valiantzas 2013). A great number of them was selected in this study aiming to facilitate further comparison of the results with those of other studies.

where E_i and E_aver are the predicted daily and the average of the ET₀ method values, respectively, O_i and O_aver are the calculated daily and the average of the ET_P-M values, respectively, and n is the total number of data.

Specifically, Correl, R², a and b indices, of the least squared regression analysis, are commonly used correlation measures. The MV and SD indices provide a general view of the models' performance. For more efficient model assessment, Krause et al. (2005) suggest the use of the combined index wR². For the mean error evaluation, MBE, S_d², and rMSE indices were used (Fox 1981; Berengena & Gavilán 2005). For the absolute and/or relative errors' estimation, MAE, AAE, RMSE, and RMAE indices were also calculated, so as to facilitate the discussion in this work, since they are widely reported in the literature (Xystrakis & Matzarakis 2011; Gebhart et al. 2012; Kisi 2014; Samaras et al. 2014). The descriptive d index was used for the cross-comparison between the models, expressing the degree to which a model's predictions are error free (Willmott 1982). Finally, the EF index specifies the relationship between calculated and predicted mean deviations (Greenwood et al. 1985), while rt returns a long-term value of the predicted against the calculated MVs.

On an annual basis, the average observed ET₀ rate calculated from the ET_P-M for the period 1996–2012, was found to be 1,190 mm. This value is a little different from the mean annual precipitation (1,174 mm) and specifically indicates that the water requirements of a reference crop are totally sufficed. Additionally, the mean seasonal values of ET₀ showed seasonal variation, as expected for the Mediterranean climate (Csa), varying from 99 mm (winter) to 558 mm (summer) and moderate values during the transitional periods of spring and autumn (307 mm and 226 mm, respectively).

From the analysis of the annual values, the best approaches seem to give the ET_HS2, ET_COP, and ET_STU methods, in which the percentages of ET₀ average annual values diverge by +0.47%, −0.69%, and −2.27%, respectively. In contrast, the largest annual deviations appeared in the ET_ROM (+60.85%), ET_MAK (−26.47%), and ET_PEN (−26.23%) methods. These results were in line with Federer et al. (1996), who reported that different methods gave widely differing estimates of annual ET₀ at particular locations which sometimes were up to several hundreds of millimeters.

From the analysis of 5,433 daily values with 18 different statistical parameters and indices, the 24 tested methods were evaluated comparatively. Their performance against the ET_P-M method is shown in Figure 2 and Table 3 and the obtained results are the following.

Among the five examined methods, the ET_PEN showed average daily ET₀ equal to 2.44 mm, a value significantly lower (−27.8%, rt = 0.722) compared to the corresponding ET_P-M. Simultaneously, it displayed the smallest correlation coefficient (R² = 0.826). The approaches of the average daily ET₀ of the ET_ALB, ET_WMO, and ET_MAH were very similar, with deviations of +9.1% (rt = 1.091), −9.9% (rt = 0.901), and +13.0% (rt = 1.131), respectively. However, their R²s are not considered satisfactory (0.830, 0.855, and 0.860 respectively). Finally, the ET_ROM method displayed the worst statistics for almost all of the evaluation indices.

It is worth noting that this category of methods resulted in the smallest EF index ranging from −0.437 (ET_ROM) to 0.759 (ET_WMO). Overall, the statistical indices of the mass-transfer equations were not satisfactory (Table 3). Hence, these methods cannot be recommended for use without calibration, in models that need the input of ET₀ daily values. Similar results were reported by Valipour (2014b, 2015c), who examined R² and MBE and suggested new calibrated mass-transfer equations for the provinces of Iran which relatively improved the performance of the original models.

In our study (forest environment with warm humid Mediterranean climate), except for ETP_PEN, the high summer daily values (>5 mm) are overestimated with the mass-transfer equations. Although Tabari et al. (2011) reported that the majority of the mass-transfer empirical equations they tested, had also the worst performances but at the same time underestimated ET_P-M in humid environments. This could be attributed to the fact that VPD (e_s – e_a) presents significant variations among locations, as stated by Irmak et al. (2006), who performed a sensitivity analysis of the Penman–Monteith method for several regions with different climate types, in the USA.

Combination methods, in general, showed much better statistical indices compared to the methods of all other categories (Table 3). The four methods examined here presented sufficiently strong correlations (R² is ranging from 0.937 in ET_COP to 0.958 in ET_VA1), compared to ET_P-M. However, the rt was significantly lower in ET_COP deviating only by −0.6% (rt = 0.994), compared to the other three methods, while ET_VA1 presented the greatest divergence (+11.3%, rt = 1.113). Impressive was the finding that despite the relatively small R² of ET_COP, most of the other statistical indices outweighed the respective indices which derived from the analysis of all 24 tested methods (Table 3). Similar results for the ET_COP, ET_VA1, and ET_VA2 equations were presented by Kisi (2014) in a Mediterranean environment in Turkey. The findings of Valipour (2015b) in Iran for the Valiantzas' equations are also in line with the results presented here.

The ET_STU method also gave satisfactory results (rt = 0.978, R² = 0.955) despite the small requirements in input data. These methods are strongly recommended for use in models which need the input of ET₀ daily values because they have a very satisfactory EF index ranging from 0.894 (ET_STU) to 0.934 (ET_COP).

In this category ten methods were tested. The radiation-based equations overall performed better than the mass-transfer equations, since a more important role is expected for R_s when estimating ET₀ in humid climates (Irmak et al. 2006) and in forest environments (Gebhart et al. 2012). ET_J-H and ET_ABT (R² = 0.956 and 0.953, respectively) methods presented satisfactory correlations. Additionally, the ET_J-H method showed the lowest deviation from the daily MV (+3.2%, rt = 1.032), while ET_ABT had the best indices, wR² and S_d², over all of the 24 tested methods. These findings for ET_J-H are in contrast to Tabari et al. (2011), who worked in humid environments.

ET_TUR can be considered as the best performing equation in this category, in terms of its EF (0.906), rMAE (0.716), RMSE (0.674), and MAE (0.513). The best performance of the equation was also found by Lu et al. (2005) in forest watersheds with warm and humid climates in the southeastern USA, and by Trajkovic & Kolakovic (2009) who recommended ET_TUR for use under humid conditions, and by Gebhart et al. (2012) who suggested the use of ET_TUR for the southern regions of Central Macedonia in Greece.

In contrast, the ET_MGB and ET_DBK methods gave the worst correlations among all the radiation-based equations (R² = 0.865 and R² = 0.901, respectively). Under humid conditions, similar results were reported for ET_MGB by Tabari et al. (2011). The largest deviations from the daily MV were displayed by ET_MAK (−26.4%, rt = 0.736) and ET_ABT (−17.6%, rt = 0.824). The EF index ranged from 0.707 (ET_MAK) to 0.886 (ET_J-H). The poor performance of ET_MAK was also reported by Lu et al. (2005).

From the above, it can be concluded that some methods (ET_TUR, ET_J-H, and ET_ABT) of this category can be satisfactorily accepted for use in models needing the input of ET₀ daily values.

In this category five methods were tested. The best correlation was exhibited by ET_HAR with R² = 0.945 and the worst by ET_THO (R² = 0.837). Concerning the deviation from the daily MV, the ET_HS1 gave the best of all 24 tested methods (along with ET_COP), with rt = 1.006. These findings were similar to those of Valipour & Eslamian (2014) and Valipour (2015a) who gave specific ranges of the meteorological parameters used in 11 temperature-based equations for Iran's provinces and found a better performance of ET_H-S, ET_HS1, and ET_HS2 against ET_THO. The results in our study indicated a slight underestimation in ET₀ daily values for all of the tested temperature-based equations (except of ET_THO which underestimated by 17.8%). Valipour (2015a) also found a slight underestimation in ET₀ daily values with ET_H-S and ET_HS1 equations, while the ET_HS2 equation showed overestimation but not a significant one.

Contrary to these results, Trajkovic & Kolakovic (2009) and Tabari et al. (2011) found overestimation when using ET_HS1, ET_HS2, and ET_H-S equations, on a monthly time-step analysis and under humid conditions. They also found very poor performance of the ET_THO equation. Additionally, Lu et al. (2005) suggested careful calibration and verification when applying the ET_THO equation.

The ET_HAR method, despite the small data requirements, exhibited in general impressive statistical indices (e.g., best MAE, rMSE, rMAE, d, S_d², and EF values) compared to all tested methods of all categories (Table 3). Gebhart et al. (2012) also proposed the ET_HAR equation as a good alternative for ET₀ estimations in northern regions of Central Macedonia in Greece. From the above mentioned, it can be concluded that this category of equations seems to have similar performance (with the exception of ET_THO) and they are recommended for use in forest environments.

This study attempted to investigate and evaluate the best-fit methods for the estimation of daily ET₀ in a humid Mediterranean evergreen broadleaved forest environment. Twenty-four different equations classified in four categories were tested and seventeen different statistical indices were used for the evaluation.

At the category level, the combination equations seem to have the best performance followed by temperature-based and radiation-based methods. The mass-transfer methods have the worst coefficients and overestimate ET₀, especially for the high summer daily values (>5 mm).

At the method level, the most accurate and consistent estimates of ET₀ derived from ET_COP and ET_HAR, followed by ET_VA1 and ET_VA2. The methods ET_TUR, ET_HS2, ET_STU, and ET_J-H are also proposed for use because their ET₀ estimations compared quite well with those of the ET_P-M method. The latter ones can also be used for the annual estimation of the ET₀. Concerning the rest of the tested methods, it is suggested that calibration should be made for local conditions, mainly at a seasonal time-step to obtain more reliable daily estimates. Especially, calibration is necessary for all the mass-transfer equations and the ET_MAK and ET_THO methods. There is an ongoing work by the authors, in which new coefficients will be proposed for some of these methods. The results of this study will be useful to a multidisciplinary community working on similar climates. More specifically, the best performing equations in Figure 2 could be tested further in order to optimize the ET₀ obtaining values of the empirical models in the Mediterranean. Further research is needed in order to evaluate the performance of the proposed modified equations in other areas with different climates. Finally, evaluation is needed for the performance of the models on a different time scale (monthly and seasonal).

The authors would like to express their sincere thanks to the Greek ‘Ministry of Agriculture and Rural Development’ and the European Union that funded various research projects from which the data used in this work were derived. The contribution of the anonymous reviewers is also greatly acknowledged.