The effects of conservation messaging on retail water deliveries

In January 2014, California's Governor Jerry Brown declared a drought state of emergency and called on all Californians to reduce water use by 20%. The governor's action was prompted by the fact that the state was entering its third consecutive year of record drought with no relief in sight. As part of its response, Metropolitan's Board of Directors allocated $5.5 million to create a large scale public outreach campaign designed to foster public awareness about the drought and to promote water conservation.

The Metropolitan outreach effort, initiated during April of 2014, was multifaceted. Roadside billboards and freeway signs were used to increase the public's drought awareness. Between April and September 4,507 radio spots were broadcast throughout the six Southern California counties comprising Metropolitan's service territory. The message was reinforced by 754 TV spots airing between July and October. Against this background a host of interviews, op-ed pieces, news stories and public service messages contributed to the overall effort.

Metropolitan did not act alone. Many local wholesalers and retail water agencies conducted their own public information campaigns. Meanwhile, the scope and diversity of conservation programs increased as agencies sought to translate the water use efficiency message into practice.

There were early indications that the media campaign was effective in increasing public awareness. Hits on the conservation-oriented Bewaterwise.com website increased by more than 150% during July 2014 compared to July 2013 according to a survey conducted by Metropolitan. Additionally, 8 of 10 Southern Californians reported having recently heard a conservation-related message with many adding that they had recently engaged in some type of conservation-oriented behavior.

While increased drought awareness is one indicator of success, it is important that more tangible outcomes be identified and measured. In particular, did the campaign have any effect on the retail demand for water? This is the central question addressed by this paper.

Ideally, one would like to establish a causal link between the demand for water and the media outreach efforts designed to reduce these demands. But this is not possible in the present situation, largely because of the absence of a suitable control group for gauging what would have happened if the media campaign had not taken place. Still, the circumstances of the media effort do suggest a type of natural experiment that can be probed for possible cause-like patterns. It is possible, for example, to advance certain expectations about what should happen if media effects are indeed present. First, one would expect a reduction in demand coincident with the media campaign. Second, because of the way the campaign evolved – radio followed by TV ads – a certain pattern of effects should occur. In particular, an initial reduction in demands should accompany the onset of the radio ads. This should be followed by an even greater reduction when the TV ads begin, assuming the relative potency of TV as a communication medium. Finally, achieved savings might be expected to decay over time without active reinforcement.

Efforts to infer a relationship between water demands and Metropolitan's outreach campaign are complicated by the fact that this effort was only one piece in a combination of influences occurring during the period. Outreach programs and practices by Metropolitan's member agencies and their retail customers were undoubtedly an important factor in this respect. So was the increased vigilance of the State Water Resources Control Board. Thus, while it may be possible to establish an association between Metropolitan's outreach efforts and subsequent patterns in water use, the outcome cannot be attributed to Metropolitan's efforts alone.

Another important factor to consider when searching for distinct media effects is that agency sponsored conservation activity increased dramatically during the spring and summer of 2014. Because of this, it becomes difficult to know what portion of any observed reduction in demand to attribute to the media campaign and what to assign to conservation. Indeed, it seems likely that the increase in conservation program activity and the media events enjoyed a symbiotic relationship in the sense that media ads promoted the availability of conservation programs while participation in these programs heightened awareness about the drought.

Despite the temporal overlap between conservation activity and Metropolitan's media campaign, it is possible to logically distinguish between the two influences. In particular, increased conservation activity should be associated with a sustained reduction in demand. In contrast, the occurrence of media effects is expected to be accompanied by an initial reduction in demand followed by a gradual restoration as the initial effect of the media ‘treatment’ decays.

As shown, the demand for water closely mimics the path of the regional economy between 2004 and 2014. (Values are expressed relative to a 2010 base year.) In particular, the years from 2004 to 2008 reflect rather normal economic fluctuations. This is followed by a dramatic drop in demand during 2008–2010, a period that marks the onset of the Great Recession. This condition continues through 2011 and is then followed by a period of recovery. By 2014 demands began to approach pre-recessionary levels. The general pattern shown here will serve as a useful point of contrast during later analysis.

Generally considered, Figure 1 suggests that the demand for water should have been increasing during the first part of 2014. Contrary evidence would be consistent with the notion of conservation effects.

Because the intensity of the media campaign varied over time, it is useful to partition these effects in terms of their constituent parts. While not a perfect fit, partitioning by calendar month seems a helpful analytic strategy. The resulting classification is as follows:

• April. Targeted radio and digital.

• May. Targeted radio and digital with cessation of radio during the third and fourth weeks of the month.

• June. Targeted radio and digital with general market radio beginning in the middle of the month.

• July. General TV, targeted and general market radio and expanded digital with TV ads ceasing in the middle of the month.

• August. General and cable TV, targeted and general market radio, billboards/cinema ads, and expanded digital with TV ads ceasing in the middle of the month.

• September. General and cable TV, targeted and general market radio, billboards/cinema ads, and expanded digital.

• October. Spanish language TV, targeted and general market radio, billboards/cinema ads and expanded digital.

Intuition suggests an ordinal ranking of the impact of these different formats. TV ads should have the greatest impact on behavior followed by radio and then digital or billboards/cinema. This ranking creates another expectation about the correspondence between media effects and water demand. Specifically, demand reductions should be most prominent during July, August and September, the months during which the TV ads were broadcast.

The data used in the analysis come from nine retail water agencies in Southern California. Combined, these agencies represent about 30% of the population and 28% of Metropolitan's wholesale water sales. These agencies do not constitute a random sample of agencies or of customers served by Metropolitan. Appropriate caveats and cautions must therefore be applied when attempting to extend this study's findings to other populations.

Each of the nine study agencies provided information about daily potable water deliveries. The amount of historical data available varied among agencies. Most were able to provide data extending back at least seven years. Several had shorter histories, some were longer. In all cases at least three years’ worth of daily water delivery information was available for estimating the regression models discussed later.

There are at least two important elements to consider when specifying a regression model (Wooldridge 2006; Kennedy 2008; Greene 2012). The first involves the selection of explanatory variables. Failure to include important variables in the model can result in biased and inconsistent estimates. Second, the model must be expressed in the correct functional form. Treating a model as linear when a different functional form is appropriate will also result in biased and inconsistent estimates.

The dependent variable in the analysis measures retail agency daily potable water deliveries in acre-feet. Ideally, customer billed water use would be used for this purpose, since this represents the quantity of water actually consumed by retail customers. The two measures differ in that deliveries do not account for water losses and metering errors. This difference, however, is likely to be small. For purposes of the following discussion retail deliveries and demand will be treated as synonymous.

The merits of a log-transformed dependent variable were considered during early stages of the analysis. But the transformation did not seem warranted in this particular application. While demand measured in levels was generally symmetric in shape, the log transformation often produced skewed distributions. Demand was therefore left in untransformed acre-feet for purposes of the analysis.

Any analysis of the demand for water must reasonably account for both climate and weather. As used here, ‘climate’ refers to the periodic changes in temperature and precipitation that come to be regarded as typical during the course of the year at a particular location. ‘Weather’ refers to the more transitory effects of temperature and rainfall at a point in time.

Temperature was measured using each day's maximum hourly temperature. Precipitation was measured as total daily rainfall. Both variables have frequently been used in prior studies. Measurements for these variables were obtained from the NOAA (National Oceanic and Atmospheric Administration) weather station closest to the center of each study agency's service area.

Following Chesnutt & Paramduas (2013), a Fourier series was used to model the cyclical pattern of climate. The interest here was not so much in interpreting the associated coefficients as it was in statistically controlling for the effects of climate on demand over time. Weather variables were represented using both contemporaneous and lagged terms. All other things being equal, demand should tend to increase during hotter weather as the use of water for both air-conditioning and outdoor irrigation increases. Conversely, demand should fall during periods of rainfall as the need for outdoor irrigation is reduced, although the observed tendency of consumers not to adjust their timers during these periods may act to mute the strength of this relationship.

Because today's water use may depend on yesterday's temperature, or because the amount of rain falling during the preceding week may effect today's use, a set of lags were used to represent weather effects. The appropriate number of lags, like the number and frequency of Fourier terms, was empirically determined.

The contemporary analysis of water demand has increasingly included variables like price and measures of economic vitality. Since good historical data about the retail price of water at the individual agencies was not readily available, Metropolitan's marginal price of water was used as a proxy. While this is an imperfect gauge of the prices charged by the different agencies, it was the best information available.

Total employment was used to represent general economic conditions. Generally speaking, higher levels of employment should index a healthier economy, one characterized by a higher demand for water.

Separate regression models were developed for each of the nine participating agencies. Since the results of each search process were generally similar, the following narrative outlines a common set of outcomes rather than repetitiously describing each search. Any departures from the general pattern are noted.

Factor variables were used to estimate monthly media effects for the period April through October. In this sense, the monthly factor variables can be thought of as representing a particular level of media potency while also accounting for any other systematic monthly influences.

The initial model specification also included terms representing price, employment and time. While the marginal price of water charged by each retail agency would have been the preferred way of expressing price, this information was not readily available. Metropolitan's marginal wholesale rate was used instead. Employment, measured in terms of the total number of people employed, was used as a general index of economic health. Climate was represented using a Fourier series. Precipitation and temperature were measured using daily rainfall and maximum daily temperature. Finally, a general trend term – measured in years – completed the preliminary specification.

The variable section process began with a simple model comprised solely of climate and weather terms. The cyclical behavior of climate was initially represented using a twelfth-order Fourier series although the higher frequency terms (sin4-cos12) were usually not statistically significant at the 5% level. Additionally, contemporaneous temperature and rainfall often tested statistically insignificant in the presence of the higher frequency terms. This led to the decision to keep only the first three terms of the series.

The next step was to estimate a basic model containing climate, weather, price, employment and a time trend. This revealed that price and the time trend could not comfortably occupy the same regression equation because of a high degree of collinearity (r = 0.98). Since dropping either variable would cause standard errors to increase, the choice narrowed to the question of which one was likely to minimize this effect while maintaining theoretical fidelity.

The fact that the time trend could potentially capture multiple influences, including employment, argued in its favor. This led to questions about the appropriate form of the time variable. Treated as continuous, the variable would have limited ability to capture within-year affects like price changes, ordinances, annual conservation and the erratic path of the economy during the early 21st century. Some tinkering with the model suggested a possible solution – using a set of dummy variables to more effectively capture inter-year affects. But this solution produced another complication. Because employment is measured annually, it had a limited number of values. So did the set of indicator variables used to represent discrete time. In fact, the two variables occupied the same vector space, meaning that they could not both be estimated in the same model. So once again, a decision had to be made about which variable to delete from the regression model. Because they so effectively captured economic patterns, the set of factor variables representing time was retained and employment was dropped from the equation. Parameter estimates were largely unaffected by this exclusion.

Given the initial set of variables discussed above, a second round of screening was conducted. This was designed to identify outliers and influential observations. Using Stata's robust covariance estimator a generalized measure of distance was iteratively calculated for each observation. These values, or weights, were then used to flag highly deviant observations. Subsequently, all observations with small weights (large influence) were excluded from the analysis.

Autocorrelation and heteroskedasticity are common issues in the analysis of time series data. Whenever one or both are present, the ordinary least-squares estimator (OLS) remains consistent but is no longer efficient, meaning that the OLS standard errors will be too large. Of even greater concern where time series data are involved is the possibility that the series is not stationary over time.

Separate regression models were estimated for each agency. Based on the F-statistic, the fit of each model was highly significant. Standard diagnostics did not reveal any problems of concern. Finally, the model R² s indicate that each model accounts for a large portion of the total variation in demand.

The regression results give a mixed impression of the effectiveness of the radio and digital ads used during April-June. Demands actually increased during this period for two utilities. Since it is unlikely that the radio and digital media ads would have increased demand, these positive estimates probably reflect the continuing recover of the economy during early 2014.

Reductions in demand become apparent beginning in July. Demands during this month declined significantly among all agencies. These reductions coincide with the start of the TV spots and suggest the relative impact of TV as a communications medium. It is important to note here, however, that the intensity of the campaign also picked up in July, making it difficult to separate the effects of media intensity from the impact of the TV medium.

Significant demand reductions continued during August at all agencies. Since the potency of the message increased only slightly during this month, these decreases may indicate that the campaign was beginning to have a type of cumulative impact on customers, or it may again indicate the relative impact of TV.

Perhaps the most noticeable thing about the individual graphs in Figure 3 is the degree of uniformity across most agencies. Demand reductions began in April-May for most agencies and were well underway by July. These reductions hit their lowest point in August and then began to recede during September in most cases.

Seven of the nine agencies evidence the U-shaped pattern discussed at the outset of the study: following a slow start, delivery reductions bottom out during August and then decay during the months of September and October. In contrast, Glendale and Las Virgenes show patterns suggesting that the media messages had a relatively sustained effect in these two areas. Additional data is needed to determine if this effect will subsequently decay and the reason for persistence if it does not.

On balance, total demands among the nine agencies were 6.8% lower than they otherwise would have been. This translates into savings of 41,505 acre-feet between April and October. With a program cost of $5.5 million, this pencils out to a cost of $121.94 per acre-foot saved.

It is important to note here that the $122/AF estimate may be only a temporary outcome. Despite encouraging findings, the sustainability of the estimated savings remains an open question. Additionally, the ability to evoke conserving behavior during follow-up campaigns, or at other times of the year, remains untested. This type of uncertainty contrasts with the more reliable water savings associated with the installation of water-efficient device hardware and related programs.

Regression analysis was used to identify and measure changes in retail delivery patterns associated with Metropolitan's water use efficiency media campaign during April-October of 2014. Demand reductions are apparent following the radio spots broadcast during April and are especially prominent during the TV spots run from July to September. While the initial media effects decay at most agencies, this is not universally true. This outcome merits future attention.

The most important qualification involves the fact that the data generated from a true experimental design. As noted at the outset, this precludes making any causal statements about the relationship between the media event and associated reductions in demand. While the associations found are intriguing, replication and further investigation are clearly warranted.

Statistically significant reductions are apparent at all agencies by June, bottom out during August and begin to decay during September. This pattern coincides with the occurrence of TV advertising and supports the intensity-impact hypothesis. The pattern of sustained savings for two agencies following the end of the TV spots is not consistent with the hypothesis. It may be that the overall campaign had a type of cumulative effect that manifest itself beyond the campaign. The reason this would occur at some agencies and not others is not apparent. A more thorough investigation is needed to explore this outcome.

Because of other conservation activities implemented during the same time period, it is not possible to definitively distinguish between the separate effects of the media events and other activities. Indeed, the two may be strongly related – media publicity about drought increases demand for conservation programs and participation in these programs increases sensitivity to drought-related information. However, the U-shaped pattern of demand observed for most agencies is highly consistent with the hypothesis of media effects tending to increase with media potency and decaying once the effect is removed. The U-shaped effects also contrast with the more monotonically decreasing pattern likely to be associated with the effect of conservation hardware and similar programs.

An important limitation of the current study was noted at the outset. This is the inability to draw causal inferences about the observed media effects. These types of inference require a stringent experimental design under which a treatment can be methodically applied and withheld. Whether it is possible to implement this type of research design where the general population is involved is debatable. But it is important to recognize that true experimental design sets the standard where causal inference is concerned.

A second caution comes from the fact that the agencies used in the analysis do not constitute a random sample of retail water agencies in Southern California. Strictly speaking, this means that the results cannot be generalized to other areas in the region. At the same time, the patterns uncovered are sufficiently consistent across agencies that it seems plausible that the media effects are more likely than not to extend beyond the limited sample used in the analysis.

Despite these qualifications the analysis shows that demands at the nine study agencies were 6.8%, or 41,005 acre-feet, lower that they otherwise would have been between April and October of 2014.

This paper has benefited greatly from the suggestions and council of Dr. Anil Bamezai of Western Policy Research and Dr. Thomas Chesnutt of A&N Technical Services (Chesnutt et al. 1992). We also wish to express our sincere appreciation to the nine agencies that provided data for the analysis.