In this paper, a comprehensive modified stochastic dynamic programing with artificial neural network (MSDP-ANN) model is developed and applied to derive optimal operational strategies for a reservoir. Most water resource problems involve uncertainty. To show that the MSDP-ANN model addresses uncertainty in the input variable, the result of the MSDP-ANN model is compared with the performance of a detailed conventional stochastic dynamic programing with regression analysis (CSDP-RA) model. The computational time of the CSDP-ANN model is modified with concave objective functions by deriving a monotonic relationship between the reservoir storage and optimal release decision, and an algorithm is proposed to improve the computational efficiency of reservoir operation. Various indices (i.e. reliability, vulnerability, and resiliency) were calculated to assess the model performance. After comparing the performance of the CSDP-RA model with that of the MSDP-ANN model, it was observed that the MSDP-ANN model produces a more reliable and resilient model and a smaller supply deficit. Thus, it can be concluded that the MSDP-ANN model performs better than the CSDP-RA model in deriving the optimal operating policy for the reservoir.

INTRODUCTION

Development of a decision-making model for reservoir operation that depicts field-scale conditions is a challenging task, and a variety of different modeling approaches exist for this purpose. Many researchers have published comprehensive reviews of optimization and simulation techniques related to reservoir operations and modeling (Yeh 1985; Labadie 2004; Fayaed et al. 2013a, b). Chandramouli & Deka (2005) and Chandramouli & Raman (2001) developed new approaches with the decision support model (DSM) based on trained neural network models that use real-time data from previous time periods to decide on operating policies. The DSM that was developed based on an artificial neural network (ANN) outperforms the regression-based approach. Chaves & Kojiri (2007a, b) introduced a new approach for system optimization and operation known as the stochastic fuzzy neural network, which can be defined as a neuro-fuzzy system that is stochastically trained (optimized) by a genetic algorithm (GA) model to represent the system operational strategy. Moreover, to address the imprecision originated by the discretization of inflow intervals (events) in calculating the transition probabilities, the researchers applied a method based on the conditional probability of a fuzzy event.

Uncertainties associated with random properties commonly exist in a reservoir operation system (e.g. precipitation, stream inflow, and water demands); moreover, these uncertainties are further compounded by ambiguity and vagueness from the relevant hydrologic information obtained (Jing & Chen 2011). Previously, a number of stochastic dynamic programing (SDP) methods were explored to reflect the dynamic complexities and random uncertainties of the reservoir operation system (He et al. 2010; Housh et al. 2013). The current work considers the ability of ANN to mimic the nonlinearity features of the reservoir system and shows that the ANN model's capabilities are suitable for hydrological processes. The ANN is a useful tool for resolving complex problems than can be addressed using other traditional models (regression analysis (RA)). In this work, the results obtained showed that the ANN provides more accurate and reliable results than other methods, especially in the simulation result obtained on the relationships between the elevation–storage and storage–surface area capacities of the reservoir. Chaves & Chang (2008) proposed a novel intelligent reservoir operation system based on an evolved ANN. The reference to evolution indicates that the parameters of the ANN model are identified using the GA evolutionary optimization technique. Accordingly, the ANN model should represent the operational strategies of reservoir operation. This system is capable of successfully and simultaneously handling various decision variables and provides reasonable and suitable decisions.

Adeloye & Munari (2006) developed an optimization model integrated with ANN and regression models. The results showed that the latter are marginally better; however, given that the regression models require the over-year capacity to be known a priori, the ANN models are more generic and should be preferred.

An important controversy in the literature of stochastic reservoir operation and stream flow modeling concerns the appropriate statistical assumptions for the inflow sequence (Philbrick & Kitanidis 2001; Ponnambalam et al. 2003; Liu et al. 2006; Chaves & Chang 2008). The appropriateness of a stochastic assumption for this inflow sequence assumption will depend on the interval between time steps.

These remarks are cogent because many authors have adopted the independent assumption. For computational reasons (compared with the independence assumption, the first-order Markov process assumption does not add more complexity and is consistent with the solution process of the SDP formulation), the first-order Markov process description of inflows has been adopted in nearly all of the SDP formulations in statistical studies that tested the Markov assumption on rivers (Mousavi et al. 2005; Panagoulia 2006; Mousavi et al. 2007; Rani & Moreira 2010; Safavi et al. 2013).

In this situation, a first-order Markov process is not appropriate. If the correlations among inflows beyond lag one are strong, the occurrence of current inflows can be more precisely conditioned on previous inflows. In other words, the current inflow can be more precisely modeled using not only the inflow in the previous time step but also using the inflows in the preceding time steps (higher lags). The transition probability of the current inflow should be conditioned on additional previous inflows, which is a more precise description of reality. However, this approach yields a mathematical formulation that is impractical to solve, because these previous inflows must be included in the state variable set. These difficulties stem from the structural limitations of the SDP in which the addition of each new state variable forces a significant increase in the number of system states to be evaluated. Therefore, in SDP reservoir operation modeling, the number of state variables rarely exceeds two (one storage variable and one hydrological variable) (Raman & Chandramouli 1996; Jain et al. 1999; Moeini et al. 2011; Kumar et al. 2013).

An innovative approach to tackling the dimensionality problem treats the inflow sequences as inflow patterns, which are represented in the SDP formulation as a single state variable. The resultant model may be referred to as an inflow pattern stochastic dynamic programing model.

However, the efficiency and successful operation of a reservoir depends largely on the reservoir simulation system. The determination of the reservoir simulation approach must therefore be given special attention. Different methods can be used to determine the simulation, and most of them make use of statistical analysis. However, the problem with most of these methods is that they cannot easily handle the phenomenon of nonlinearity processes that occur in the reservoir (Celeste & Billib 2009; Fayaed et al. 2011). Simulation models associated with reservoir operation are usually based on a mass balance equation, which represents the hydrological behavior of reservoir systems using inflows and other operating conditions (Wang et al. 2010; Wu & Chen 2013).

This study examines the integrated ANN as a reservoir simulation component and the modified stochastic dynamic programing model (MSDP) as an optimization method in determining optimal reservoir operation. This knowledge will assist in obtaining more accurate values for optimal reservoir policy, which will act as a foundation for the case study. In addition, this work also reviews the application of the MSDP model together with other simulation models and compares the effectiveness of the different results. This work provides an understanding of better performance that will be reflected in enhanced reservoir release and an effective distribution system, which are the results of a more accurate reservoir simulation.

Sg LANGAT DAM – CASE STUDY

This study considers the Sg Langat dam as a case study. This Malaysian dam is located geographically at latitude 3° 12′ 43.07″ north of the equator and longitude 101° 53′ 39.28″ east of the prime meridian in Kuala Lumpur. The dam was built and completed in 1979 for the purpose of water supply. The Sg Langat dam has an earth embankment made of 2.5 million cubic meters (MCM) of earth, and the crest elevation is 223.72 m with a maximum height of 61 m. The primary spillway is gated, and the crest elevation is 220.96 m. The reservoir catchment area is 41 km2, as shown in Figure 1. The dam controls and regulates the volume of water released into the Langat river during the dry season to ensure that adequate water is available at the Sg Langat water treatment plant, located approximately 14 miles downstream of the dam. This dam plays a highly significant role in the neighborhood of Kajang, Bangi, because it is a major supplier of drinking water to the residents. The dam's (reservoir) capacity is 37,480.0 million L. The relationships among water surface elevation, storage volume, and water surface area constitute fundamental information that characterize the reservoir and are required in modeling studies. The elevation versus area relationship is used in evaporation computations, because evaporation volumes are a function of evaporation rates and water surface area. The elevation versus storage relationship is required to determine the head pressure in hydroelectric power computations. Storage/elevation relationships are required to relate storage capacities to elevation-based operation rule curves. The results from statistical analysis of 15 years of historical data pertaining to the Sg Langat reservoir are presented in Table 1. Table 1 shows that the average annual inflow is 46.78 × 106 m3 with a deviation of 19.77 × 106 m3. The average maximum inflow occurs during the month of September, and the average minimum occurs during August. The maximum and minimum inflows realized in each month are also depicted in Table 1.

Table 1

Statistical analysis of historical inflow

Month Maximum (MCM) Minimum (MCM) Average (MCM) Std dev. (MCM) Skewness Demand (MCM) 
Jan. 6.61 0.32 2.75 2.00 0.34 4.00 
Feb. 3.85 0.46 2.23 1.17 −0.44 4.00 
Mar. 8.14 1.58 4.05 1.57 1.26 2.50 
Apr. 5.70 2.50 4.12 1.17 −0.07 2.00 
May 11.24 0.39 3.73 2.82 1.37 3.00 
Jun. 8.70 0.13 3.29 2.15 1.02 3.00 
Jul. 4.97 1.36 2.75 1.07 0.42 3.00 
Aug. 5.51 2.54 3.90 0.98 0.36 3.00 
Sep. 10.08 1.84 6.21 2.34 −0.39 2.00 
Oct. 6.04 0.23 3.92 1.55 −1.22 2.50 
Nov. 8.72 3.45 5.67 1.36 0.57 1.80 
Dec. 6.76 1.88 4.16 1.59 0.29 3.00 
Annual   46.78 19.77   
Month Maximum (MCM) Minimum (MCM) Average (MCM) Std dev. (MCM) Skewness Demand (MCM) 
Jan. 6.61 0.32 2.75 2.00 0.34 4.00 
Feb. 3.85 0.46 2.23 1.17 −0.44 4.00 
Mar. 8.14 1.58 4.05 1.57 1.26 2.50 
Apr. 5.70 2.50 4.12 1.17 −0.07 2.00 
May 11.24 0.39 3.73 2.82 1.37 3.00 
Jun. 8.70 0.13 3.29 2.15 1.02 3.00 
Jul. 4.97 1.36 2.75 1.07 0.42 3.00 
Aug. 5.51 2.54 3.90 0.98 0.36 3.00 
Sep. 10.08 1.84 6.21 2.34 −0.39 2.00 
Oct. 6.04 0.23 3.92 1.55 −1.22 2.50 
Nov. 8.72 3.45 5.67 1.36 0.57 1.80 
Dec. 6.76 1.88 4.16 1.59 0.29 3.00 
Annual   46.78 19.77   
Figure 1

Catchment area of the Sg Langat reservoir.

Figure 1

Catchment area of the Sg Langat reservoir.

MODEL FORMULATIONS AND DEVELOPMENT

Stochastic dynamic programing model

In SDP, the randomness of reservoir operation systems can be expressed in terms of probabilities, and these probabilities of various states allow for incorporation of occurrences of different inflow levels into the optimization framework at each stage. The recursive function f of the employed SDP model is as follows: 
formula
1
where f*n(.) = objective function; C(.) = immediate return function; St = storage at the beginning of time period t; Qt = inflow during month t; Rt = release during time period t; α = discount rate in the SDP algorithm; Pr = transition probability matrix.
In the current study, a monthly time-stepped SDP model has been developed with an objective function that minimizes the squared deviation of the monthly release deficit. Mathematically, the objective function is given by 
formula
2
where t is the month index, N is the operating horizon in months, and R(t) and D(t) are the respective release and demand in month t. The objective function in Equation (2) is subjected to mass balance, storage, evaporation, and surplus constraints. Storage and inflow are discretized into 60 and 100 classes, respectively. The first class of the inflow state variable is represented by a value of zero. The last inflow class has no upper bound and is represented by the maximum historical inflow of the corresponding month.
The mass balance in the reservoir from one time step to another time step (e.g. month-to-month relationship of the mass) is given by the continuity equation. The continuity equation is stated as follows: 
formula
3
where St+1 = final storage in month t; Qt = inflow during month t; Et = evaporation loss in the reservoir during time t; Ot = surplus from the reservoir during time t.
The reservoir storage in any month should not exceed the capacity of the reservoir and should not be less than the desired target storage and the statistical analysis of the historical storage given in Table 2. Mathematically, this constraint is given as follows: 
formula
4
where Smax = capacity of the reservoir in millions of cubic meters.
Table 2

Statistical analysis of historical storage

Month Maximum (MCM) Minimum (MCM) Average (MCM) Std dev. (MCM) 
Jan. 38.64 26.6 37.19 3.2 
Feb. 38.64 28.9 35.93 3.5 
Mar. 38.64 19.8 34.06 5.73 
Apr. 38.68 14.6 33.7 7.22 
May 38.7 13 34.17 7.76 
Jun. 38.7 12.3 34.19 7.69 
Jul. 38.7 12 33.58 8.13 
Aug. 38.8 13.7 33.1 7.99 
Sep. 38.9 17.4 33.71 7.84 
Oct. 38.9 18.4 33.28 7.94 
Nov. 38.9 19.8 35.68 5.53 
Dec. 38.9 23.4 37.08 4.01 
Month Maximum (MCM) Minimum (MCM) Average (MCM) Std dev. (MCM) 
Jan. 38.64 26.6 37.19 3.2 
Feb. 38.64 28.9 35.93 3.5 
Mar. 38.64 19.8 34.06 5.73 
Apr. 38.68 14.6 33.7 7.22 
May 38.7 13 34.17 7.76 
Jun. 38.7 12.3 34.19 7.69 
Jul. 38.7 12 33.58 8.13 
Aug. 38.8 13.7 33.1 7.99 
Sep. 38.9 17.4 33.71 7.84 
Oct. 38.9 18.4 33.28 7.94 
Nov. 38.9 19.8 35.68 5.53 
Dec. 38.9 23.4 37.08 4.01 
The reservoir evaporation loss during any time period t is given as the product of the evaporation rate and the average water spread area at the beginning and end of the time period (Vedula & Mujumdar 2005). It is found that this simple equation is well suited for this particular case study. Therefore, the evaporation loss during any time period t is given by 
formula
5
where Et = evaporation loss during time period t, which is computed as the function of water spread area corresponding to the active initial and final storage and evaporation rate; et = evaporation rate in millimeters per month during time period t; Ad = water surface area at the top of the minimum storage level (m2); m = slope of the straight line of water surface area plotted against reservoir active storage; SAt = active initial storage at the beginning of time period t; SAt+1 = active final storage at the end of time period t.
This constraint addresses the situation in which the final storage exceeds the capacity of the reservoir. Mathematically, this constraint is given by 
formula
6

Regression analysis

The hypothesis of regression and correlation is a classical framework used to describe relationships between two or more variables. Of course, the relationship may not necessarily be physical in nature and might also involve methods for modeling and analyzing several variables if the focus is on the relationship between a dependent variable and one or more independent variables. This analysis further explains how the typical value of the dependent variable changes when any one of the independent variables is varied and the other independent variables are held fixed. Determination of the relationships among elevation, storage, and surface area is one example of the application of RA in the science of hydrology. In investigating the relationship between two variables x and y, the realizations of the random pairs (xi, yi) are plotted as a scatter plot, which illustrates the relationship between the two variables.

Artificial neural network

The ANN is a new technique with a flexible mathematical structure that is more capable of identifying the complex nonlinear relationships between input and output data compared with other classical model techniques. The ANN model contains an input layer, one or two hidden layers, and an output layer in a forward multilayer neural network. The input layer contains i nodes, the hidden layer contains j nodes, and the output layer contains k nodes. Therefore, the output zk can be expressed as follows: 
formula
7
where function f is the transfer function or activation function, is the input quantity, and are the weighted values, and and are the deviations. In Equation (7), function f is a type of mapping rule used to convert a neuron from a weighted input to an output and is also a type of design used to introduce nonlinear influences into the ANN network. This study chooses the most general binary logistic sigmoid function defined as follows: 
formula
8
where the range of the value is (0, 1). If a linear function is selected for this transfer function, i.e. f(x) = x, the entire ANN structure will become a linear influence from the input layer to output layer. The structure of the storage/surface area model is shown in Figure 2. The storage/surface area model contains 10 neurons in the input layer and five neurons in the hidden layer. Because the ANN is a supervisory learning algorithm, the optimum parameter is obtained by adjusting the weighted values in the network, where optimum means that the squared deviations between the network outputs zk and the actual values or target values tk achieve a minimum, i.e. the set of all available data is separated into two disjoint sets, the training set and testing set. The test set is not involved in the learning or training phase of the networks and is used to evaluate the performances of the models. 
formula
9
Figure 2

Feed-forward back-propagation neural network.

Figure 2

Feed-forward back-propagation neural network.

The set of all available data is separated in two disjoint sets: the training set and test set. The test set is not involved in the learning or training phase of the networks and is used to evaluate the performances of the models. In this study, 100 inputs of reservoir elevations (first 50 and last 50 inputs) were selected for calibration of the ANN model, whereas the remaining 19 inputs (in between) were used to verify or test the performance of the model. In this particular study, the structure of the ANN model is designed based on Figure 2. The structure of the elevation-area model is the same as that of the elevation-storage model, i.e. both models have the same structures. For additional details on the structure of the ANN simulation model for this reservoir, see Fayaed et al. (2013b).

MODIFIED STOCHASTIC DYNAMIC PROGRAMING

During integration of the conventional stochastic dynamic programing with artificial neural network (CSDP-ANN) model, the results of the simulation and optimization model were more reliable and robust than those of the CSDP-RA, but the disadvantage of the first model is that it is computationally time consuming, which leads us to look for a modification for the CSDP-ANN model to tackle this problem. The MSDP-ANN procedure is shown and presented in Figure 3.

Figure 3

Flowchart for solution of the CSDP-ANN model (Huang et al. 1991).

Figure 3

Flowchart for solution of the CSDP-ANN model (Huang et al. 1991).

Improved dynamic programing using the monotonic relationship

In conventional dynamic programing (DP), the optimal storage carry-over corresponding to a current storage st is determined via an exhaustive search approach, and to find for each st, every st+1 value should be searched (Figure 4, left), i.e. if st is discretized into nt intervals and st+1 into nt+1 intervals (equal interval length), as shown in Figure 4, then to determine the optimal for p, the nt+1 discretized st+1 intervals should all be tested.

Figure 4

Schematic diagram of the conventional dynamic programing computation (left) and the modified dynamic programing computation (right).

Figure 4

Schematic diagram of the conventional dynamic programing computation (left) and the modified dynamic programing computation (right).

If the single-period utility function is concave, according to the monotonic relationship between st and , if we know the optimal state q (at stage t + 1) corresponding to p (at stage t), then to search for the optimal for p + 1, we need only to test two st+1 values at the states of q and q + 1 (Figure 4, right). Thus, by applying the monotonic relationship, the computation for determining for st can be potentially reduced from nt+1 to 2. An improved DP algorithm is proposed (as shown in Figure 5), which includes the following steps. (1) Start: discretize st and st+1 into intervals of equal length and number them in ascending order, i.e. 1 to nt and 1 to nt+1, respectively. (2) Initialization: setting p = 1 (the minimum discretized st), search from 1 to nt+1 to find its corresponding optimal , and initialize q = st+1*. (3) Computation: setting p = p + 1, search between q and q + 1, and use the better of the two to update corresponding to p. (4) Repeat step 3 until p = nt. Steps 1–4 are designed for DP computation at stage t.

Figure 5

Flowchart of the improved dynamic programing algorithm.

Figure 5

Flowchart of the improved dynamic programing algorithm.

Using recursive computation from T to 1, the T-stage reservoir optimization operation problem can be solved using Equations (1) and (2). As illustrated in Figure 4, when n = nt = nt+1, the computational complexity of the improved algorithm is n + 2*(n–1) = 3*n–2, whereas that of the conventional algorithm is n2, i.e. each of the st+1 states (n) should be tested with each of the st states (n). Thus, for a large number of state discretizations (n), 3*n–2 < <n2, and the computational order of the improved algorithm can be considerably reduced.

Monotonicity with stochastic parameters and improved SDP

In SDP, hydrological uncertainty can be described by the state transition probability P(Qt+1/Qt) of the reservoir inflow, which represents the conditional probability of Qt+1 in period t + 1 on Qt in period t (Faber & Stedinger 2001; Zhao et al. 2011). Incorporating Pr(Qt+1/Qt) into Equation (1), we obtain the recursive function of SDP as follows: 
formula
10
Defining 
formula
11
Equation (9) is rewritten as follows: 
formula
12

If assuming a partially concave dependence relationship between and St+1 (i.e. ), given that the state transition probability P(Qt+1/Qt) is positive, the partial dependence relationship between FFt+1(St+1) (the weighted sum of , Equation (10)) and st+1 is concave (i.e. ). Thus, with a fixed Qt (Equation (11)), Ft and FFt+1(St+1)│Qt are concave functions, and an assumption of partially concave dependence relationship between and st+1 leads to the same relationship between and St. Hence, if assuming that Ft (rt) is concave, following the procedures described in the ‘monotonicity in reservoir operation analysis’ section, we can conclude that: (1) FT–1(ST–1, QT–1), … , F1(S1,Q1) are partially concave functions of ST–1, … , S1, respectively; and (2) with a fixed Qt, a monotonic relationship exists between St and (and also ). If applying the monotonic relationship to improving the SDP, the procedures are shown in Figure 6 with two steps. Step 1: for a fixed value of Qt, run the modified DP (Figures 4 and 5) to search for corresponding to each of the possible values of St. Step 2: update Qt and repeat step 1 until all possible Qt values are tested and corresponding to all (St, Qt) combinations is identified. Using the modified SDP in the backward recursive formulation, the multistage stochastic reservoir operation problem can be solved.

Figure 6

Flowchart of the modified stochastic dynamic programing algorithm (l and n represent inflow and storage discretization, respectively).

Figure 6

Flowchart of the modified stochastic dynamic programing algorithm (l and n represent inflow and storage discretization, respectively).

Integration of neural network stochastic dynamic programing

In this work, integration between ANN and CSDP is applied to derive the reservoir operation policy for the Sg Langat dam. The nonlinearity of natural physical processes causes a major problem in determining the simulation of reservoir systems, especially if using the conventional linear methods, e.g. RA method. To tackle this problem, a nonlinear computational method, i.e. the ANN, is introduced as a simulation system.

The model implementation procedure is applied first, and the operating storage volume (level) and stream inflows are discretized. Next, the SDP solution approach begins by initializing the values of the objective function at the last stage (a period in the future) to zero for each combination of state variables. The reservoir system is simulated using the ANN. Afterwards, the process continues backwards along the temporal stages. Each iteration consists of T stages that complete one annual cycle. The algorithm finds the end-of-period storage levels for each combination of the discretized beginning-of-period storage level and the average inflow of the current period. The behavior of the policy convergence after several iterations is due to the characteristics of the Markov transition probability matrix incorporated into the recursive equation.

Model verifications

Failure

The following description of reliability, resilience, and vulnerability is based on the assumption that the system under consideration at a given time t can be in either a satisfactory (i.e. nonfailure) state NF or an unsatisfactory (i.e. failure) state F. In this study, the focus is on water supply systems, and therefore, the NF state occurs when the water supply is able to meet the water demand; hence, the F state occurs when supply cannot meet demand. Moving from time step t to t + 1, the system can either remain in the same state or migrate to the other state. The duration of the jth excursion into a failure period is denoted as d(j), and the corresponding deficit volume is denoted as v(j), j = 1, … , M, where M is the total number of failure events. The deficit volume of the failure event is calculated as the cumulative difference between demand and availability as follows: 
formula
13
where d(j) is the duration of the failure, and and are the water demand and the water actually supplied, respectively. The following sections describe how to estimate R–R–V from the extracted series of failure duration and deficit volume.

It should be noted that the reliability, resilience, and vulnerability value ranges between 0 and 1. For the reliability index, a value is close to one means that the model is effective, whereas the model is effective in the case in which the value of resilience and vulnerability is close to zero.

Reliability

The oldest and the most widely used performance criterion for water resource systems is reliability, which is defined by Hashimoto et al. (1982) and Park et al. (2009) as follows: 
formula
14
where S is the system state variable under consideration. The most widely accepted and applied definition is occurrence reliability, which can be estimated as follows: 
formula
15
where d(j) is the duration of the jth failure event, M is the number of failure events, and T is the total number of time intervals.

Resilience

Resilience is a measure of how rapidly a system is likely to return to a satisfactory state once the system has entered an unsatisfactory state. Hashimoto et al. (1982) defines resilience as a conditional probability as follows: 
formula
16
where S(t) is the system state variable under consideration. This definition of resilience is equal to the inverse of the mean value of the time that the system spends in an unsatisfactory state, i.e., 
formula
17
where again is the duration of the jth failure event, and M is the total number of failure events. Moy et al. (1986) defined resilience as the maximum consecutive amount of time that the system spends in an unsatisfactory state. To make this definition comparable with the definition of in Equation (16), resilience is expressed as the inverse of the maximum duration as follows: 
formula
18

Vulnerability

Vulnerability is a measure of the likely damage of a failure event and was defined by Hashimoto et al. (1982) as follows: 
formula
19
where h(j) is the most severe outcome of the jth sojourn into an unsatisfactory state, and e(j) is the probability that h(j) is the most severe outcome of a sojourn into the unsatisfactory state. Hashimoto et al. (1982) based their vulnerability measure on the total water deficit experienced during the entire jth sojourn into F, i.e. deficit volume. This definition is suited for reservoirs because the most severe outcome of a reservoir state is often the empty state, h(j) = 0. As a further simplification of Equation (18), both studies considered the probability of each event being equal, i.e. e(1) = …= e(M)=1/M, where M is the number of failure events; therefore, they estimated vulnerability as the mean value of the deficit events v(j) as follows: 
formula
20

Cumulative penalty

For the cumulative penalty incurred from 15 years of simulation, the equation for cumulative penalty may be expressed as follows: 
formula
21
where is the number of time periods of a year for monthly time step , n is the number of years of simulation (15 in this study), Rji is the actual release during the time period i of the year j (and equals the optimal release that was suggested by the model whenever possible), and Di is the demand for the time period i.

RESULTS AND DISCUSSION

This section presents the results of applying the models to the case study. A 15-year time series of stream flow (1996–2010) is used for evaluation of the model for water allocation from the Sg Langat reservoir according to the water treatment plant demands. In all models, the performance indices are used to evaluate the results of the models. These indices show how often the system does not fail (reliability), how quickly the system returns to a satisfactory state once a failure has occurred (resiliency), and the significance of the consequences of failure (vulnerability). With respect to the problem statement, two different types of state variables (Qt or Qt–1) were used when each model was tested (CSDP-RA and CSDP-ANN). From these tests, four submodels were obtained and are given in Table 3.

Table 3

Submodel descriptions

Model Description State variable 
CSDP-RA Qt 
CSDP-RA Qt−1 
MSDP-ANN Qt 
MSDP-ANN Qt−1 
Model Description State variable 
CSDP-RA Qt 
CSDP-RA Qt−1 
MSDP-ANN Qt 
MSDP-ANN Qt−1 

Conventional stochastic dynamic programing with the regression analysis model

In this study, different values of elevation, surface area, and storage capacity were subjected to RA, and the parameters yielded the following models: 
formula
22
where SA = surface area and ELE = elevation 
formula
23
where S = storage and ELE = elevation.

However, the results from these models were not as accurate compared with the results of other methods used in this study.

To generate the operating policies for reservoirs, a CSDP-based model with the RA was developed. The results of the CSDP-RA model with Qt and Qt–1 as the state variables are presented in Table 4. In this table, the reliability, resiliency, and vulnerability (used as the performance indices for supply to the water demand at the downstream section of the reservoir) for a 15-year simulation period were 0.699%, 0.302%, and 0.263 MCM, respectively, when Qt was used as state variable (Model 1). However, when the state variable was Qt–1 (Model 2), the performance indices were 0.678%, 0.289%, and 0.277 MCM, respectively, when equal numbers of inflow and reservoir storage discretization were considered. The results show that in using Model 1, the reliability of supplying the demands improved by 3.1%. The implication for this result is that out of the 175 months in the study period, the total number of nonsupplied demand months (unsatisfactory states) is 52 months using Model 1 but only 56 months using Model 2. In addition, more robust and reliable results were obtained. As given in Table 4, the resilience in Model 1 can produce a gain of 4.4% compared with that of Model 2. The vulnerability values, i.e. the expected values of nonsupplied water demand, were approximately 0.263 and 0.277 MCM for Models 1 and 2, respectively. The penalty function was used as a measure of the performance of the reservoir system and also used to identify the decision that makes the greatest contribution to the operation of the reservoir. The objective is to release the amount of water that is closest to the target demand, and thus, the operational performance also can be measured in terms of the deviation of the release decision from its target demand. The penalty function attempts to minimize the functional value and forces the release to satisfy the target demand. From Table 4, the penalty values for Models 1 and 2 are 1.59 × 10–5 and 1.67 × 10–5, respectively. This result shows that the penalty value increased by 5% for Model 1 compared with that of Model 2.

Table 4

Performance indices of supplying the demand with the CSDP-RA model

Model Reliability (%) Resilience (%) Vulnerability (MCM) Penalty 
0.699 0.302 0.263 1.59 × 10–5 
0.678 0.289 0.277 1.67 × 10–5 
Model Reliability (%) Resilience (%) Vulnerability (MCM) Penalty 
0.699 0.302 0.263 1.59 × 10–5 
0.678 0.289 0.277 1.67 × 10–5 

Conventional stochastic dynamic programing with the artificial neural network model

In this model, integration between the ANN and CSDP was used to derive the reservoir operation policy in Sg Langat. To evaluate the performance of the CSDP-ANN model, the results of the performance indices are determined and compared. The reliability, resiliency, and vulnerability of supplying the water demand at the downstream section of the reservoir are 0.76%, 0.327%, and 0.235 MCM, respectively, for Model 3, whereas the values for the same parameters are 0.734%, 0.310%, and 0.244 MCM, respectively, for Model 4, considering the same numbers of inflow and reservoir storage discretization. The results of the performance indices in supplying the demand to each sector are presented in Table 5. The results show that Model 3 improved the reliability of supply by 3.5%, which means that the total number of nonsupplied demand months (unsatisfactory states) is 47 months when Qt–1 was the state variable (Model 4) and approximately 42 months when Qt was the state variable (Model 3). The resilience in Model 3 shows a gain of 5.4% compared with that of Model 4. The vulnerability values are approximately 0.235 and 0.244 MCM, respectively, in Models 3 and 4. The penalty value in Model 3 is much better than that of Model 4, and its gain was 4.5%; this result demonstrates that Model 3 is more reliable and robust in deriving the optimal operation policy for the Sg Langat reservoir. The number of storage is 60, and from Table 6, the computation time at this storage using the CSDP-ANN model was 365.5 minutes while it was 18 minutes when the MSDP-ANN model was used.

Table 5

Performance indices of supplying the demand with the CSDP-ANN model

Model Reliability (%) Resilience (%) Vulnerability (MCM) Penalty 
0.760 0.327 0.235 6.08 × 10–6 
0.734 0.310 0.244 8.80 × 10–6 
Model Reliability (%) Resilience (%) Vulnerability (MCM) Penalty 
0.760 0.327 0.235 6.08 × 10–6 
0.734 0.310 0.244 8.80 × 10–6 
Table 6

Computational time consumption for different storage numbers

No. of storage CSDP-ANN MSDP-ANN 
10 9.5 2.8 
20 38.4 5.7 
40 152.2 11.7 
60 365.5 18.0 
80 648.3 23.6 
100 982.2 29.5 
No. of storage CSDP-ANN MSDP-ANN 
10 9.5 2.8 
20 38.4 5.7 
40 152.2 11.7 
60 365.5 18.0 
80 648.3 23.6 
100 982.2 29.5 

Comparisons between CSDP-RA and MSDP-ANN models

The CSDP-RA and MSDP-ANN were applied to generate reservoir operation policies. The results from the two models are compared in this section, as shown in Figures 7 and 8. The cumulative penalty values for each of the four models are presented in Figure 7. It can be observed that Model 3 has the lowest penalty value. In addition, from the data shown in Figure 8, Model 3 is more reliable, more resilient, and less vulnerable than the other models. To compare the accuracies of CSDP-RA and MSDP-ANN models, the state variable Qt was fixed. Comparison between Models 1 and 3 showed that the latter increased the reliability and the resilience of the system by 8.7% and 8.2%, respectively, which means that the total number of nonsupplied demand months (unsatisfactory states) is 53 months in Model 1 and approximately 42 months in Model 3. The nonlinearity of natural physical processes constitutes a major barrier to successful simulation of reservoir systems, especially if using such conventional linear methods as the RA method. To surmount this difficulty, the ANN nonlinear computational method is introduced for simulation of the system. The results of this comparison highlight the superiority of using ANN as the simulation model and Qt as the state variable in the optimization model. The result of applying this reservoir operation system to each month is a three-pronged relationship among storage number, monthly inflow, and final storage number. This relationship represents the steady-state policy for each prospective month. In Figure 9, February was taken as an example to illustrate the shape of the relationship.

Figure 7

Cumulative penalties for all models.

Figure 7

Cumulative penalties for all models.

Figure 8

Performance indices of supplying the water demand for all models.

Figure 8

Performance indices of supplying the water demand for all models.

Figure 9

Steady-state policy for February found by Model 3.

Figure 9

Steady-state policy for February found by Model 3.

Applying different demand scenarios to Model 3

From the above discussions, it can be concluded that the best model for deriving the optimal operation policy is MSDP-ANN with Qt as the state variable (Model 3). In this study, experiments were conducted to investigate the behavior of the system under different demand scenarios with increments of 5, 10, 15, 20, and 25% in the demand target, as given in Table 7. The results showed that an increase in the demand led to an increase in the vulnerability and a decrease in both the reliability and the resilience. In addition, the penalty value increased with an increase in the demand value, as shown in Figures 10 and 11. For instance, in scenario number one, when demand increased by 5%, the reliability and resilience decreased by approximately 7% and 3%, respectively, and the penalty value increased by 2.6%. The number of nonsupplied water days in scenario one was 51 months compared with 42 months prior to the increment in the demand.

Table 7

Performance indices of supplying the demand for five scenarios

Model 3 (%) Reliability Resilience Vulnerability Penalty 
0.709 0.316 0.246 6.24 × 10–6 
10 0.663 0.309 0.249 7.51 × 10–6 
15 0.633 0.305 0.253 8.29 × 10–6 
20 0.617 0.300 0.267 8.96 × 10–6 
25 0.596 0. 294 0.278 9.83 × 10–6 
Model 3 (%) Reliability Resilience Vulnerability Penalty 
0.709 0.316 0.246 6.24 × 10–6 
10 0.663 0.309 0.249 7.51 × 10–6 
15 0.633 0.305 0.253 8.29 × 10–6 
20 0.617 0.300 0.267 8.96 × 10–6 
25 0.596 0. 294 0.278 9.83 × 10–6 
Figure 10

Performance indices of supplying the water demand for different scenarios in Model 3.

Figure 10

Performance indices of supplying the water demand for different scenarios in Model 3.

Figure 11

Cumulative penalties in different scenarios for Model 3.

Figure 11

Cumulative penalties in different scenarios for Model 3.

CONCLUSIONS

In this work, an MSDP-ANN model is developed and applied in a case study, and the optimal results are compared with those of the CSDP-RA model. Both models are solved using same objective function and constraints with the same length of inflow data. To tackle the nonlinearity problems, the nonlinear computational method ANN is introduced as a simulation system. The results of the simulations highlight the superiority of using ANN as the simulation model (instead of the RA model) and using Qt as the state variable in the optimization model. This approach leads to better policy when the inflow forecast for the current period is perfect. We note that Model 3 has the least penalty value. In addition, Model 3 is more reliable, resilient, and less vulnerable than the other models. To compare the accuracies of the CSDP-RA and MSDP-ANN models, the state variable was fixed as Qt. The comparison between Models 1 and 3 showed that Model 3 increased the reliability and the resilience of the system by 8.7% and 8.2%, respectively, compared with Model 1. Therefore, it can be concluded that the MSDP-ANN model performs better than the CSDP-RA model in deriving the optimal operating policy for the reservoir system.

ACKNOWLEDGEMENTS

This research was supported by a research grant to the second author from the Ministry of Higher Education for project FRGS/1/2012/TK03/UKM/02/4 and University Kebangsaan Malaysia for project DIP-2012-03. The authors thank the Puncak Niaga (M) SDN BHD for providing the data for the Sg Langat reservoir.

REFERENCES

REFERENCES
Celeste
A.
Billib
M.
2009
Evaluation of stochastic reservoir operation optimization models
.
Adv. Water Resour.
32
(
9
),
1429
1443
.
Chandramouli
V.
Raman
H.
2001
Multireservoir modeling with dynamic programming and neural networks
.
J. Water Resour. Plan. Manag.
127
(
2
),
89
98
.
Chaves
P.
Kojiri
T.
2007a
Stochastic fuzzy neural network: case study of optimal reservoir operation
.
J. Water Resour. Plan. Manag.
133
(
6
),
509
518
.
Fayaed
S. S.
El-Shafie
A.
Jafaar
O.
2011
Performance of artificial neural network and regression techniques for simulation model in reservoir inter-relationships
.
Int. J. Phys. Sci.
6
(
34
),
7738
7748
.
Fayaed
S. S.
El-Shafie
A.
Jaafar
O.
2013a
Reservoir-system simulation and optimization techniques
.
Stochastic Environ. Res. Risk Assess.
27
(
7
),
1751
1772
.
Huang
W. C.
Harboe
R.
Bogardi
J. J.
1991
Testing stochastic dynamic programming models conditioned on observed or forecasted inflows
.
J. Water Resour. Plan. Manag.
117
(
1
),
28
36
.
Jain
S.
Das
A.
Srivastava
D.
1999
Application of ANN for reservoir inflow prediction and operation
.
J. Water Resour. Plan. Manag.
125
(
5
),
263
271
.
Labadie
J.
2004
Optimal operation of multireservoir systems: state-of-the-art review
.
J. Water Resour. Plan. Manag.
130
(
2
),
93
111
.
Liu
P.
Guo
S.
Xiong
L.
Li
W.
Zhang
H.
2006
Deriving reservoir refill operating rules by using the proposed DPNS model
.
Water Resour. Manag.
20
(
3
),
337
357
.
Moeini
R.
Afshar
A.
Afshar
M.
2011
Fuzzy rule-based model for hydropower reservoirs operation
.
Int. J. Electr. Power Energy Syst.
33
(
2
),
171
178
.
Mousavi
S.
Ponnambalam
K.
Karray
F.
2005
Reservoir operation using a dynamic programming fuzzy rule-based approach
.
Water Resour. Manag.
19
(
5
),
655
672
.
Mousavi
S.
Ponnambalam
K.
Karray
F.
2007
Inferring operating rules for reservoir operations using fuzzy regression and ANFIS
.
Fuzzy Sets Syst.
158
(
10
),
1064
1082
.
Park
K. B.
Kim
S.
Lee
Y. H.
2009
The suggestion of monthly water supply reliability indexes for the drought disaster prevention, South Korea
.
Disaster Adv.
2
(
1
),
22
30
.
Raman
H.
Chandramouli
V.
1996
Deriving a general operating policy for reservoirs and using neural networks
.
J. Water Resour. Plan. Manag.
122
(
5
),
342
347
.
Vedula
S.
Mujumdar
P. P.
2005
Water Resources System – Modelling Techniques and Analysis
.
Tata McGraw-Hill
,
New Delhi
.
Wang
Y.
Chang
J.
Huang
Q.
2010
Simulation with RBF neural network model for reservoir operation rules
.
Water Resour. Manag.
24
(
11
),
2597
2610
.
Zhao
T. T. G.
Cai
X. M.
Yang
D. W.
2011
Effect of stream flow forecast uncertainty on real-time reservoir operation
.
Adv. Water Resour.
34
(
4
),
495
504
.