Repeated games for eco-friendly flushing in reservoirs

Sedimentation in reservoirs causes loss of capacity, increased flood risk, degradation of water quality, and increased difficulty in reservoir operation and maintenance, with consequent increases in their costs. Sediment storage can also have a significant impact on downstream ecosystems in large river basins. Given the scarcity of undeveloped dam sites in watersheds that are already extensively exploited, it will be necessary in future to focus increasingly on storage preservation. The effects of reservoir sedimentation can be mitigated by implementing suitable management techniques (Nikolaos et al. 2017).

Flushing is one of the most economical methods for recovering lost storage without incurring the costs of dredging. It involves remobilizing deposited sediments by increasing the flow velocity in the reservoir. The entrained deposits are discharged downstream through low-level outlets. The flow velocity is increased by drawing down the reservoir water level through a suitably designed outlet. Flushing also releases large volumes of sediment downstream, however, creating potentially serious problems. Scouring of polluted sediments from the reservoir threatens downstream water quality and ecology (Sloff 1991), while the high sediment concentrations released may have a significant impact on downstream biota (Morris & Fan 1997). This has led to the development of an ‘environmentally friendly flushing’ technique (Fruchart 2008).

In this method the hypolimnic water from the bottom outlet, which has a high suspended sediment load, is mixed with water from the mid-depth outlet, which is low in suspended sediment. Thus, the flushing flow created has much lower suspended sediment concentrations than would otherwise be the case. The technique also improves downstream oxygen concentration, and minimizes temperature changes and pollution: water from near the bottom of a stratified reservoir is usually cold, oxygen-depleted, and high in hydrogen sulphide and other pollutants, whereas water from the mid-depths is more oxygenated and warmer, and contains lower concentrations of heavy metals and pollutants (McCartney et al. 2001).

Since about 2,000, many researchers, including those working in environmental resources management, have implemented a range of game theory tools and concepts. Carraro et al. (2005), Parrachino et al. (2006), and Zara et al. (2006) review game theory water conflict resolution studies. Some hydrological issues like water resources allocation (Wang et al. 2003; Kucukmehmetoglu 2012), watershed eco-compensation (Cao et al. 2011), bi-objective optimization of reservoir watershed management (Üçler et al. 2015) and water pollution control (Shi et al. 2016) have also been resolved by applying game theory. Nevertheless, the full extent of game theory has not been used in systems analysis in water and soil resource management, particularly at watershed scale (Adhami & Sadeghi 2016). Predicted stable game outcomes are not necessarily Pareto-optimal. The main concern of players is to maximize their own benefit, knowing that the final outcome is the product of all decisions made (Madani 2010). Game theory provides more realistic simulation of stakeholders' interest-based behavior. The self-optimizing attitude of players and stakeholders, represented in game theory, often results in non-cooperative stakeholder behaviors even when cooperative behavior is more beneficial to all parties. Game theory can help provide some planning, policy, and design insights that would not be available from other traditional systems engineering methods.

Another advantage of game theory over traditional quantitative simulation and optimization methods is its ability to simulate different aspects of the conflict, incorporate various characteristics of the problem, and predict possible resolutions in the absence of quantitative payoff information. Often non-cooperative game theory methods can help resolve conflicts based on qualitative knowledge about players' payoffs (i.e. how the players order (rank) different outcomes (ordinal payoffs)). This enables it to handle the socio-economic aspects of conflicts and planning, design, and policy problems when quantitative information is not readily available.

This study presents the principles of repeated and dynamic games, one of the game theory methods that has shown great potential for conflict problem solution. After developing the necessary elements, an adaptation of this algorithm is provided for conflict resolution for eco-friendly flushing in reservoirs. The results of applying the methodology are shown in a case study.

The study was carried out at Dez Reservoir, in southern Iran, a large hydroelectric dam built in 1963 by an Italian consortium. At the time it was Iran's largest development project. The dam is a 203 m high, double curvature, arch dam. Its crest is 352 m above sea level. The original reservoir volume was 3,315 × 10⁶ m³, and the estimated volume of sediment arriving over 50 years was 840 × 10⁶ m³. The reservoir's minimum and maximum operating water levels are 300 and 352 m above sea level, respectively.

The Dez project is now more than 40 years old and reaching mid-life. Its useful life is threatened by a sediment delta, which is approaching the dam's intake tunnels. A hydrographic study in 2002 showed that sedimentation had reduced the reservoir's useful storage from 3,315.6 to 2,700 × 10⁶ m³ (19%). The difference between the turbine inlet and bed surface levels of the sedimentary deposit is 14 m, and the sedimentation rate near the turbine inlet is 2 m/year, so sediment management in the reservoir is of considerable importance (Khakzad & Elfimov 2014a).

A field measurement program for the turbidity currents in the reservoir ran from December 2002 to June 2003 (Dezab Consulting Engineers 2004). The program consisted of a series of daily measurements at various depths and locations along seven cross-sections. The measurement station locations are shown in Figure 1. Valeport 108 MK II (Valeport Ltd, Devon, UK), for the first four months, and, subsequently, RCM9 instruments (Aanderaa Data Instruments, Bergen, Norway) were used to measure the current velocity and direction, electrical conductivity, temperature, and pressure.

The Valeport 108 MK II was developed to meet the needs of oceanographers, hydrographers and surveyors. Speed and direction sensors are fitted as standard, with optional conductivity, temperature and depth sensors. The RCM9, used from the fourth month, can also measure turbidity and is self-recording. Figures 2–5 show measurement record samples on turbidity currents at stations A2, B3, C3, E, and F, collected on April 24, 2003. The water level then was 351 m, maximum depth 94 m, and the reservoir inflow and outflow 1,210.6 and 590.8 m³/s, respectively (KWEO 2003).

The data collected from December 2002 to April 2003 showed that there were only two significant turbidity currents. The velocity and suspended sediment concentrations measured at section A indicated that the first turbidity current occurred on January 28 and 29, 2003. The turbid layer was about 15 m thick, the near-surface water flow direction was upstream, and the velocity was low. As would be expected, large slow-moving eddies were present above the turbidity current, whose velocity and direction changed with time. Some 44,000 m³ of fine sediment moved into the reservoir on January 29, 2003.

The second turbidity current occurred on April 23 and 24, 2003. It is difficult to quantify the turbidity load using the measurements available. However, on the basis of the average reservoir inflow values, the total volume of sediment for the rainstorm events rose to around 1.2 Mm³ over the two days. Table 1 shows the range and values of the different parameters used in this study. The median particle size of all samples was less than 0.01 mm, with that of the sediments collected upstream in the reservoir slightly larger than those closer to the dam.

Dynamic games – e.g., repeated, Markov chain and stochastic games – extend stage-games (Figure 6). A repeated game is a dynamic game in which the same stage-game is played in periods (or stages) t = 0, 1, 2, etc. When the number of game periods is not known in advance and can be infinite, the repeated game is called infinite (Burkov & Chaib-draa 2015). The simplest case occurs when, at t = 0, 1, 2, etc, a game is played (whether identical or not), and the strategy selections and payoffs at time t are independent of those in the previous time periods. In other words, games played at different time periods are completely independent. The payoff of this repeated game for each player is the (possibly discounted) sum of payoffs at the different time periods (Matsumoto & Szidarovszky 2016).

Repeating a game raises two further issues about the players: 1. How do they remember the past? 2. How do they appraise the future? The set of the repeated game histories up to period t is given by ⁠. The set of all possible histories is given by⁠. Player i's (mixed) strategy is a mapping of ⁠. A strategy profile is a vector ⁠. denotes player i's strategy set, and the set of strategy profiles.

A subgame is a dynamic game that continues after a certain history. For a pair ⁠, the subgame strategy profile induced by h is denoted as ⁠.

A strategy profile is a subgame-perfect Nash equilibrium (SPNE) in the repeated game if, for all histories, ⁠, the subgame strategy profile is a Nash equilibrium in the subgame.

Finally, a pair of strategies (⁠⁠) satisfies the one-stage deviation condition if neither player can increase their payoff by deviating (unilaterally) from such strategy in any single stage and returning to the specified strategy thereafter. In these conditions: a pair of strategies is an SPNE (a strategy profile that induces an NE in every subgame) for a discounted game if and only if it satisfies the one-stage deviation condition.

Finally, it is noted that there are a cluster of results, commonly known as folk theorems, characterizing the set of payoff profiles of Nash and subgame-perfect equilibria in a repeated game. To illustrate the common principle behind these theorems, the complete proofs are given below for some of them.

Criteria for determining whether flushing at a particular reservoir will be successful are required. There are two key requirements; first, the sediment quantities transported through the low level outlets during flushing must be sufficient to enable a long-term balance between the sediment inflow and the sediment flushed, and second, the volume of deposits remaining in the reservoir after sediment balance has been achieved is sufficiently small to enable a specified storage requirement to be met. These criteria depend on the hydraulic efficiency of flushing and by applying them, reservoirs where flushing might be viable can be identified. The hydraulic efficiency of flushing can be defined in several ways and some definitions are shown in Table 2 (Khakzad & Elfimov 2014b).

The Dez River provides fish habitat, and supports commercial (market), subsistence and recreational fisheries. A list of economically important fish species found along the river downstream of the dam is given in Table 3. The categories most representative of the river's fish community are adult freshwater non-salmonids, and the eggs and larvae of salmonids and adult non-salmonids. Local fishermen indicate that high suspended sediment loads during flood seasons reduce catches substantially, because fish abandon their usual locations and cannot be found. The likelihood of damaging nets is also higher during high turbidity events. This is consistent with the responses from fishermen the day after flushing, i.e., that the fish were not biting or that only small fish were being caught in the nets.

Dez Hydropower Dam is provided with three low-level outlets within the dam body. Their original purpose was to provide irrigation releases downstream during periods of low flow through the turbines. They were also intended to provide a means of emergency release from the reservoir, if necessary. The historical outflows from the powerhouse, spillway facilities and low-level outlets are 200, 80 and 150 m³/s, respectively.

There are two ways to interpret the discount factor, d. The first can, for convenience, be called ‘economic’. Economists have observed that individuals (or groups such as private companies) value their current or near-term well-being substantially more than that in the long-term; the power of the discount factor reflects this. Another interpretation, which is mathematically equivalent, can be called ‘natural’. In this, the discount factor is viewed as the probability that the repeated game will continue at the next iteration. This explanation is more convenient for artificial agents because it is generally questionable whether they value the future in a similar way to humans. The probability of continuation, in turn, seems to be more ‘natural’ because the machine always has a non-zero probability of fault. For very shortsighted players with a low discount factor in this study, the (common) discount factor was set to for every transition and the maximum number of iterations at 200. These parameter settings are regarded as optimal for the standard repeated games algorithms. Figures 7 and 8, and Table 4 present the results of optimization of eco-friendly flushing operations in Dez Hydropower Reservoir. They describe the payoffs – maximum sediment evacuation with minimum water outflow, acceptable average water volume per flush (Mm³) and acceptable average sediment volume per flush (Mm³) for the two scenarios.

Three distinct kinds of information are displayed in the solution, in addition to the standard game display: Above each move is the probability that it is chosen in this equilibrium. Below each node is the expected payoff for each player when reaching that node. Above the probability of each move is the expected payoff for its owner of making that choice.

As most fish in the Dez River spawn in March or April, the most conservative of the eggs and larvae, or adult, graphs should be employed from March through June (scenario 1), and the adult graphs throughout the rest of the year (scenario 2). The results of the proposed flushing optimization framework showed that this method provides a powerful tool for selecting the optimum response scenario against the identified risks, and can be an important instrument for change in the quality of sediment management.

There is increasing attention to river ecosystem conservation in Iran due to massive dam construction and reservoir regulations. The need to release ecological flows downstream has gradually reached a consensus in reservoir operations. In this study, repeated games were used successfully to optimize flushing of Dez Hydropower Reservoir and evaluate the derived predictive model. The study was based on maximizing sediment evacuation with minimum water outflow, as objectives, and existing flushing criteria and ecological flow as constraints. All scenarios proposed in the study enhance sediment evacuation and thus the reservoir's sustainability. The results show that the repeated games algorithm can obtain high-quality solutions quickly for establishing an objective quantitative relationship between technical and executive requirements, and environmental impacts. Although the model was only applied to Dez Hydropower Reservoir in this study, its generic form enables it to be applied to a broad range of reservoirs to extend their useful lives.