Assessment of maize water status using a consumer-grade camera and thermal imagery Open Access

As the world population grows, the world population's living standards increase, diets change and the effects of climate change intensify. So, the development of agriculture is critical for food security. Agriculture is both a major cause and casualty of water scarcity, especially in arid and semi-arid areas worldwide (FAO 2019). Consequently, a deeper understanding is necessary of abiotic stress such as drought, salinity and low or high temperatures on productivity demands and plant growth and development, through precision irrigation (Ihuoma & Madramootoo 2017). Drought or crop water stress induces various biochemical and physiological responses in plants. Extensive studies on plant water status based on detecting the crop parameters and soil water status have been reported (Osakabe et al. 2014). However, laboratory methods of determining plant water status such as the measurement of the stomatal conductance, leaf water potential and sap flow are costly and time-consuming (Li et al. 2010; Pinheiro & Chaves 2011; Gerhards et al. 2018). For decades, sensing of crops from platforms such as satellites or unmanned aerial vehicles (UAVs) has presented a more appropriate method to use for extensively planted crops such as maize (Toureiro et al. 2017; Zhang et al. 2019). All of them are equipped with hyperspectral, multispectral or digital imaging techniques to calculate indices from their spectral bands and also provide crop details such as plant water status (Sugiura et al. 2007; Santesteban et al. 2017; Zhang et al. 2018a, 2018b). However, satellite platforms have limitations such as high cost for high special resolution images, short availability of usable data (because clouds and cloud shadows may hide ground features), low temporal resolution and fixed schedule. On the other hand, most aerial imagery utility and availability are limited by the cost and complexity of the imaging system. So, consumer-grade cameras that have been designed to acquire imagery, reduce costs and increase the spatial resolution of images. Modified consumer-grade cameras, as another type of consumer-grade camera, replace the infrared (IR)-blocking filter in front of the complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) sensors with a long-pass IR filter to capture IR light spectral information details (Yang et al. 2014; Zhang et al. 2016). A low-cost modified camera that use external filters does not require special skill for opening the sensors and removing or changing them (Putra & Soni 2017).

Under water-stress, plant leaves display higher leaf temperature. Therefore, an infrared thermographic camera can measure canopy temperature. Hence, vegetation temperature can be used as an indicator of vegetation health. One issue that needs to be eliminated is the soil background from the thermal image. There are two commonly used methods to eliminate background pixels: (i) a threshold-based approach that uses thermal imagery only, and canopy temperature (Tc) extraction using algorithms such as Otsu and edge detection (Rud et al. 2014; Ludovisi et al. 2017; Park et al. 2017); (ii) a co-registration approach that uses both thermal and visible (red–green–blue (RGB)) images. These visible images were used to make a mask image to eliminate background pixels (Baluja et al. 2012; Han et al. 2018; Poblete et al. 2018).

The first indicator of crop stress conditions was proposed by Idso et al. (1981). They established a relationship between leaf-to-air temperature difference and vapor pressure deficit as the crop water stress index (CWSI). This model has been widely used to monitor plants' water status. However, this approach requires more meteorological data (Taghvaeian et al. 2014). Some researchers have attempted to reduce the number of parameters required to calculate crop water stress indicators. Han et al. (2016) developed the standard deviation of canopy temperature (CTSD) as a new approach to monitoring the water stress of maize crops, within a thermal image. Also, Zhang et al. (2018b) monitored water stress of cotton crops using Tc characteristics obtained from thermal images, including CTSD and coefficient of variation of Tc (CTCV).

In our study, a low-cost modified camera and external filters that included RGB and IR spectrum wavebands and also a thermal camera were used to collect images of maize plants cultivated in a greenhouse under different irrigation and nitrogen treatments. The objectives were to: (i) explore the performance of Tc-based indicators for monitoring maize water status; (ii) explore the use of low-cost modified camera and extracted vegetation indices in response to irrigation treatments and nitrogen variations; and (iii) clarify their correlations with relative water content (RWC). The results would provide guidance for developing non-destructive and rapid monitoring of water status in maize crops.

The field experiment was planned with a randomized complete block design and three replicates during 2018. An early maturing variety of maize in a greenhouse located in the western region of Iran (34°47′N, and 48°28′E) was used. Four irrigation treatments (I₁: 100% (control), I₂: 80%, I₃: 60%, and I₄: 40% of field capacity) and also four nitrogen treatments (N₁: without nitrogen, N₂: 100 kg N/ha, N₃: 200 kg N/ha and N₄: 300 kg N/ha) were applied as urea (46% N). Maize seeds were sown, in pot containers of 18.38 L capacity. The pot substrate was made by loam soil and plant nutrient content. The chemical and physical properties were assessed and are presented in Table 1. Soil testing procedures were utilized to determine the level of plant-available nutrients. Also, a soil moisture meter (LUTRON PMS-714) was used for soil moisture analysis and irrigation volume calculated based on field capacity. The experiment was conducted in late June/early July 2018, and the harvesting date was 81 days after sowing (DAS). Plants emerged approximately two weeks prior to initiation of the experiments. We took visible, infrared and thermal images in the cases of days 23, 27, 36, 43, 48, 55 and 61 after sowing dates. Then one leaf of each plant was cut using scissors; afterwards leaf nitrogen content and RWC were calculated for each plant sample by Kjeldahl and relative turgidity methods, respectively (Kjeldahl 1883; Barrs & Weatherley 1962).

Figure 1 shows some of the equipment and processing steps of extracting plant indices from RGB, IR and thermal images. Accordingly, for taking RGB and IR images, a Canon^® PowerShot A720 IS digital camera (8 Megapixel CCD sensor) was modified to detect radiation in IR spectral bands and placed in front of each pot. Thereafter, the combination of the IR blocking filter and the IR filter was used. An IR filter (GREEN.L 67 mm IR 720) prevents visible light from passing through while only allowing IR light to strike the camera's sensor (IR720 passes only infrared rays above 720 nm), and a UV/IR cuts filter (Night Sky UV/IR Cut 2’ Filter) cuts out IR rays above 700 nm and retains the visible light spectrum. These two circular filters attached to the front of the camera lens. In this way, after taking the image using the IR filter, the IR filter was separated from the lens and UV/IR cut filter placed on the front of the lens, and then the image was taken.

Then the Otsu algorithm was applied to the RGRI to separate green plants from the background (Verrelst et al. 2008). Otsu's algorithm is undoubtedly one of the most suitable thresholding techniques. It uses the image histogram data as input and finds a pixel value (so-called threshold level) that is able to separate pixels into two classes, foreground and background (Bangare et al. 2015). Therefore, the maize fractional vegetation cover (FVC) on RGB images was extracted. As shown in Figure 1, the FVC was used as a binary mask to segment the green plants against the background by multiplying FVC in the vegetation indices.

In the next step, to take thermal images, an FLIR ONE thermal imager for Android (resolution 0.1 °C) was used. Afterward, the spatial resolution of the RGB and FVC images was resampled to match the scale of the thermal images by using the nearest-neighbour interpolation algorithm. So, resampled visible and thermal images were registered, using MATLAB R2017a (Version 9.2). At each image, ground-truth temperature measurements were taken. The ground-truth temperature was measured by a gun-style design of the TESTO 830-T1 infrared thermometer from water, maize leaves and white–black boards. To calibrate the thermal imagery, a linear regression model was established between ground-truth temperature and the obtained digital number from the thermal images (Harvey et al. 2016; Bian et al. 2019; Sagan et al. 2019). Then by multiplying the FVC images in registered and calibrated images, crop temperature was extracted. In the following, we chose and calculated eight indices. The formulas for calculating the indices are given in Table 2. We obtained these indices in each plant and compared with RWC by statistical methods.

Figure 3(a) shows the specific dates and cumulative irrigation depth of irrigation events. The average total amount of water applied was 457 mm for the control treatment, 416 mm for 80% of field capacity (I₂), 360 mm for 60% of field capacity (I₃), and 349 mm for 40% of field capacity (I₄). Figure 3(b) presents the box plot of leaf N concentration during the study. Throughout the growing season, younger leaves have lower leaf N concentrations, but with plant growth, leaf N concentrations increased. Also, because of the rapid growth before silking, for all fertilizer N rates, a gradual decline in leaf N concentration was observed. After silking, N content slightly increased then continued to decline (Schepers et al. 1992).

In Figure 4, the box plot shows the RWC rate data over the study. Regarding the irrigation scheduling (Figure 3(a)), RWC values were varied in each day of the growth stage and their values were also lower when water stress increased. Figure 5 shows the distributions of ground-truth Tc and RWC for I₁, I₂, I₃ and I₄ based on data acquired on DAS 36 and 48, respectively. As shown in Figure 5, there was a difference between the distributions of ground-truth Tc and RWC among treatment groups. The results, which are not presented in this paper, showed that there were also not significant interaction effects between nitrogen and irrigation on ground-truth Tc and RWC. In I₁ treatment, soil moisture was always maintained at field capacity level, so RWC was higher than 80% and the temperature was lowest. However, with prolonged water deficit among other treatments, the Tc increased with decreasing severity of RWC, and generally, as the RWC increases temperature usually decreases or vice versa, and generally, drought-stressed plants display higher Tc than well-watered plants. The results obtained in this experiment indicated that drought stress significantly decreased the relative water content of maize and increased leaf temperature due to drought stress that might have occurred due to increased respiration and decreased transpiration resulting from stomatal closure (Siddique et al. 2000).

Figure 6 shows the relationships between temperatures derived from the thermal images and the temperature of water and white and black boards as ground-truth measurements. There are good linear correlations with a slope of about 0.8. However, the intercept of the linear correlation for water and the two diffuse boards is 5.25, 6.29, and 8.51, respectively. Also Figure 7 shows the relationship between ground-truth Tc and Tc derived from thermal images using linear regression analysis. There is a high correlation with an R² value of 0.87 and an RMSE value of 1.41 °C. Meanwhile, the slope and intercept of the linear correlation are 0.91 and 2.16, respectively. In this study, we acquired data at about 2 m height and in the RGRI-Otsu method, the nearest-neighbour algorithm was applied to resample the FVC image resolution to thermal image resolution (from 3,264 × 2,448 to 640 × 480 pixels). Similar to the results of Yang et al. (2014), our findings showed that the linear regression model (Figures 6 and 7) had a slope of approximately 1.0. Consequently, the temperatures derived from thermal images were accurate (Zhang et al. 2019).

Figure 8 and Table 3 illustrate the relationships between crop water stress indicators and RWC and the detail of the linear regression models. In particular, degrees above non-stress (DANS), CTSD and CTCV provided the best coefficient of determination values with R² = 0.88, 0.76 and 0.67, respectively. All three methods showed a low RMSE and MBE for RWC estimation compared with the other indices studied. Other used indices like difference vegetation index (DVI), transformed vegetation index (TRVI), normalized difference vegetation index (NDVI), enhanced vegetation index (EVI) and simple ratio (SR) showed coefficients of determination less than 0.50. When comparing RWC estimation through these indices by means of RMSE and MBE, these indices performed worse than the three top indices. Also, there were negative correlations between RWC and the three top indices, that is, the mean DANS, CTSD and CTCV increased with the decrease in relative water content (Taghvaeian et al. 2014), while for other indices, there were positive correlations. In general, compared with the studied indices, DANS was the more effective indicator of RWC in maize and was more sensitive to irrigation treatments, so that we were able to determine when and how much water to apply. Zhang et al. (2019) presented the same result in their paper. However, some researchers have investigated that CWSI has great potential to be used in the assessment of crop water stress (Lee & Park 2019; Costa-Filho et al. 2020). But the more important point is that DANS greatly simplifies the calculation and does not require any meteorological factors compared with CWSI, and so could be used in the diagnosis of crop water stress (Zhang et al. 2018b). After reviewing these studies and studies on CWSI, we strongly recommend additional research to compare the performance of the studied water stress indicators in different locations and under different meteorological conditions.

The use of both modified consumer-grade cameras and thermal cameras allowed the estimating of RWC, to assess maize water status. In fact, high correlations were found between indices based on thermal images with maize water status. To improve the accuracy of Tc extraction and background elimination, we used the RGRI-Otsu method based on the combination of RGB and thermal images. The Tc extracted was highly correlated with the ground-truth measurements with R² of 0.87 and RMSE of 1.41 °C. This finding showed that DANS, CTSD and CTCV can estimate maize water stress and were also simpler than the CWSI index. This method is low-cost and easy to use; therefore, there is a huge potential for implementing it on UAVs that allow farmers to observe their fields from the sky. However, further studies are required to test the performance of it for other crops and climatic and farming-practice conditions.

All relevant data are included in the paper or its Supplementary Information.