In reservoir flood control operation, selection of criteria is an important part of the multi-criteria decision making (MCDM) procedure. This paper proposes a method to select criteria for MCDM of reservoir flood control operation based on the back-propagation (BP) neural network. According to the concept of ideal and anti-ideal points, we propose a method to generate training samples of the BP neural network via stochastic simulation. The topological structure of a three-layer BP neural network used for criteria selection is established. The relative importance of criteria is derived via the learned connection weights of a trained BP neural network, and its calculation method is proposed. The sensitivity curve method is employed to conduct sensitivity analysis, and the relative contribution ratio is defined to quantify the relative sensitivity strength of each criterion. We present the principle and threshold value of criteria selection based on the comprehensive discrimination index defined by the combination of the relative importance and relative contribution ratio. The Pubugou reservoir is selected as the case study. The results show that the proposed method can provide an effective tool for decision makers to select criteria before MCDM modeling of reservoir flood control operation.

## INTRODUCTION

Reservoir flood control operation is an important non-engineering measure in the flood management of river basins, which is complicated in nature because it involves many conflicting factors resulting from technical, environmental, social and political concerns (Chou & Wu 2015). Reservoir flood control operation needs to simultaneously optimize several incommensurable and often conflicting objectives, such as flood control, water supply, hydropower generation, navigation, irrigation, ecology and so on. It is impractical or even impossible to obtain a single optimal solution that simultaneously optimizes all of these objectives due to the presence of multiple conflicting objectives (Cheng & Chau 2001; Qin *et al.* 2010; Malekmohammadi *et al.* 2011). Instead, a more practical way is to generate some feasible non-inferior alternatives in advance by multi-objective optimization models (Ouyang *et al.* 2014; Luo *et al.* 2015), then a multi-criteria decision making (MCDM) model is used to rank these non-inferior alternatives against multiple criteria so that the preferred alternative can be determined and the final flood control decision can be made. Generally, the MCDM process of reservoir flood control operation comprises the following steps: (1) generate alternatives using flood control operation models; (2) select criteria; (3) calculate the performance values of alternatives against the selected criteria; (4) weight the criteria; (5) rank the alternatives via MCDM models; (6) perform sensitivity analysis; (7) make the final decision (Hajkowicz & Collins 2007).

Numerous methods have been developed to solve MCDM problems since the 1960s. Hajkowicz & Collins (2007) classified these methods into six categories: (1) multi-criteria value functions; (2) outranking approaches; (3) distance to ideal point methods; (4) pairwise comparisons; (5) fuzzy set analysis; (6) tailored methods. In recent years, many researchers have also proposed new MCDM methods or improved the existing techniques, and applied them in the field of reservoir flood control operation (Cheng & Chau 2001, 2002; Chen & Hou 2004; Yu *et al.* 2004; Fu 2008; Wang *et al.* 2011; Zhu *et al.* 2016a, 2016b). In the MCDM of reservoir flood control operation, criteria are typically employed to measure the performance of each alternative from different aspects, such as utilization efficiency of flood resources, flood control safety of reservoirs and downstream protected regions, etc. Selection of criteria is referred to structuring the MCDM problems, and is the most important part of MCDM in view of the fact that selected criteria will directly influence the MCDM results (Howard 1991; Hajkowicz & Collins 2007; Durbach & Stewart 2012; Wang *et al*. 2014). Traditionally, the criteria are selected only via subjective judgments and then used for MCDM modeling directly (Cheng & Chau 2001, 2002; Chen & Hou 2004; Yu *et al.* 2004; Fu 2008; Wang *et al.* 2011). The selection of criteria mentioned here does not refer to this subjective way of selection, but a comprehensive and quantitative examination of criteria. During the selection process, the criteria should be examined from two aspects. Firstly, each criterion has different contributions to the final MCDM results, namely, each criterion has inherently different degrees of importance. Some criteria may contribute more to the MCDM results, while some criteria may contribute less. Secondly, changes in the value of each criterion may lead to different responses of MCDM results, that is, the MCDM results have different sensitivity strength to each criterion. Before MCDM modeling, those criteria with a small importance degree and sensitivity strength can be regarded as redundant criteria and should not be selected, because they do not strongly influence the MCDM results. Although plenty of methods are available to solve an MCDM problem of reservoir flood control operation once it has been structured, little attention has been paid to help decision makers select criteria in the first place.

The back-propagation (BP) neural network is one kind of the most widely applied feedforward neural networks, in which a gradient descent algorithm is used for network training (Hassoun 1995). The BP neural network is a distributed information processing system with the ability to store experiential knowledge obtained by network learning and make it available for future use (Parasuraman *et al.* 2006). BP neural networks allow nonlinear mapping between inputs and outputs. Inputs are weighted and processed through nodes. Networks are commonly trained using the error-back propagation algorithm by adjusting connection weights to minimize errors between network outputs and target outputs. Therefore, information about the system is finally stored in connection weights. In this paper, we study the network's weights to assess the relative importance of inputs.

This paper aims to propose a method to select criteria for MCDM of reservoir flood control operation based on the BP neural network. The novel aspects and main contributions of this study are as follows: (1) the method to generate training samples of the BP neural network is developed based on the concepts of ideal and anti-ideal points; (2) the topological structure of the BP neural network used for criteria selection is established; (3) definitions and calculation methods for the relative importance and relative contribution ratio of criteria are proposed; (4) the principle and threshold value of criteria selection are presented.

The rest of this paper is organized as follows. The ‘Methodology’ section consists of five subsections: (1) generating training samples of the BP neural network; (2) designing the topological structure of the BP neural network; (3) identifying the relative importance of criteria; (4) sensitivity analysis; (5) principle of criteria selection. The next section presents the results of a case study using the proposed methodology, followed by discussions and conclusions in the final section.

## METHODOLOGY

### Generating training samples of the BP neural network

The core of criteria selection is to enable BP neural networks to evaluate flood control operation alternatives as MCDM models, and this can be achieved through network training. The training process of the BP neural network aims to establish a nonlinear mapping between the input and output layer so that each input vector can produce output values that are as close as possible to the target output desired.

The concept of ideal and anti-ideal points has been widely employed to solve MCDM problems according to the principle that the best alternative should be the one as close as possible to the ideal alternative and as far as possible from the anti-ideal alternative. The so-called ideal and anti-ideal alternatives are not real alternatives which are physically meaningful, but suppositional alternatives used to characterize the upper and lower limit state of the alternative set (Hwang & Yoon 1981; Fu 2008). In this paper, the ideal and anti-ideal alternatives are also defined for flood control operation, based on this, we propose a method for generating training samples by randomly sampling criteria values between ideal and anti-ideal alternatives in order to meet the accuracy requirement of network training. Criteria values of the training samples are chosen as the network inputs, and the closeness coefficient (Hwang & Yoon 1981), which is usually used to rank alternatives, is selected as the comprehensive evaluation index, i.e., the network output. The following steps are involved:

Use flood control operation models to generate

*m*mutually non-dominating alternatives, i.e., , where*m*and*n*represent the number of alternatives and criteria, respectively.- Generate the input data of the BP neural network through stochastic simulation. Generate k random numbers following the uniform distribution , g = 1, 2, …, k, the value of k can be determined according to the accuracy requirement of network training. For each simulation, a unique ug is used for all criteria. Randomly sample the criteria values between the ideal and anti-ideal alternatives by the following equation: In addition, take the ideal and anti-ideal alternatives as two input data, then the input data of the BP neural network can be expressed as , where
*p*is the sample size,*p**=**k*+ 2. - Calculate the closeness coefficient ct by Equation (5). The closeness coefficient ct ranges between 0 and 1, and reflects the relative distance from the
*t*th training sample to the ideal and anti-ideal alternatives. The larger the*ct*is, the*t*th training sample is closer to the ideal alternative and farther from the anti-ideal alternative, and then the better the*t*th training sample is. - In order to avoid the influence of dimension differences to the network training accuracy, the input data are scaled to the range of [0, 1] by the linear normalization approach (Shi 2000), expressed as follows: where and are the original and normalized input data, respectively; and are the maximum and minimum values within the original input data, respectively.
The normalized criteria values serve as the input data of the BP neural network, the closeness coefficient

*c*is selected as the network output, and the final training sample set can be expressed as ._{t}

### Designing the topological structure of the BP neural network

One of the important issues in developing a BP neural network is the determination of the appropriate number of hidden neurons that can satisfactorily capture the nonlinear relationship existing between the input variables and the output (Parasuraman *et al.* 2006). The number of hidden neurons is usually determined by trial and error method with the objective of minimizing the cost function. In real-world applications, the trial and error method has a large computational burden and is time-consuming. Besides, many empirical formulas are available to determine the number of hidden neurons. In order to avoid extra computational burden, this study uses the empirical formula recommended by Lippmann (1987) to determine the number of hidden neurons, i.e., .

*n*:

*h*: 1). The commonly used activation functions include linear, sigmoid and hyperbolic tangent function. The established BP neural network makes use of the linear function in the input layer, and uses the sigmoid function in the hidden and output layer. During the training process, the connection weights are updated systematically by using the error-back propagation algorithm and gradually converge to values such that each input vector produces output values that are as close as possible to the target output. The actual output is compared with the target output, then the global error of the network is calculated by Equation (7). The training process stops until the prescribed training times or the error tolerance is reached. where and are actual and target output of the

*t*th training sample, respectively.

### Identifying the relative importance of criteria

The BP neural network is commonly trained by adjusting connection weights to minimize errors between network outputs and target outputs. Therefore, for a trained BP neural network, information about the system is finally stored in connection weights. In this section, we propose a method to identify the relative importance of criteria from a trained BP neural network via the learned connection weights.

*z*represents the actual output of the BP neural network. The

*j*th input neuron and

*s*th hidden neuron are connected with the connection weight . Similarly, is the connection weight between the

*s*th hidden neuron and the output neuron. Parameters and represent the bias of corresponding neurons in the hidden and output layer, respectively. The initial connection weights and biases are given randomly from the range (−1, 1). The inputs are transformed to output by the following equations: where represents the inputs of the hidden layer; is the inputs of the output layer; is the sigmoid function, i.e., , and .

*j*th criterion to the network output is derived by the partial derivative of actual output

*z*to the

*j*th component of the input vector

*X*, shown as follows: For a given input vector

*X*, and in Equation (10) are constants. Comparing the importance degree of the

*j*th criterion and (

*j*+ 1)th criterion:

*j*th criterion as follows: Considering that the initial connection weights are given randomly before network training, the learned connection weights and the calculated always show slight differences during each training cycle. Consequently, the mean value of is used to represent the relative importance of the

*j*th criterion as follows: where

*N*is the total number of training times; is the relative importance of the

*j*th criterion during the

*l*th training cycle.

The larger the value of , the more the *j*th criterion contributes to the MCDM results, making it inherently more important than the other criteria.

### Sensitivity analysis

In this section, we used the closeness coefficient equation (i.e., Equation (5)) to conduct sensitivity analysis, and quantified the relative contribution ratio of criteria changes to the change of MCDM results.

Plenty of sensitivity analysis approaches, such as the method of Morris (Morris 1991), the Sobol’ sensitivity analysis (Sobol’ 2001) and so on, are available to assess the impact of one variable to model outputs. In this paper, a simple but practical method, sensitivity curve method (Paturel *et al.* 1995), is used to calculate and plot the relative changes of an input variable against the relative changes of the closeness coefficient. The sensitivity curve method is similar to the method of Morris, which is a so-called one-step-at-a-time method, meaning that in each run only one criterion is given a new value and other criteria remain at their original values. The procedures of conducting the sensitivity analysis can be described as follows:

(1) Assume the original input vector is

*X*= . For the value of the*j*th criterion (denoted as ), we set nine scenarios for changes in by adjusting via delta changes, i.e., , .(2) The corresponding new input vector is expressed as . Then, the closeness coefficient is recalculated (denoted as ) under each changing scenario.

(3) The relative changes of each criterion against the relative changes of the closeness coefficient are calculated and plotted as sensitivity curves.

*c*is the original closeness coefficient). Then the relative change is expressed as: The relative contribution ratio of the

*j*th criterion is defined as: The relative contribution ratio measures the relative influence of criteria changes to the closeness coefficient, namely, the relative sensitivity strength of MCDM results to each criterion. The larger the , the stronger the relative sensitivity of the

*j*th criterion. Therefore, those criteria with large should be selected for MCDM modeling.

### Principle of criteria selection

*j*th criterion, respectively. and range from 0 to 1, so the use of multiplying operator can make the comprehensive index easier to be discriminated.

*j*th criterion is defined as: The comprehensive discrimination index can reflect the difference between the order of magnitude of and . If , the

*j*th criterion's is larger than the average level; if , the

*j*th criterion's is smaller than the average level; particularly, if , the

*j*th criterion's is significantly smaller than the average level over one order of magnitude, and this criterion should be deleted from the original criteria system. Therefore, we choose as the threshold value of criteria selection, this threshold value can avoid the influence of criteria selection on MCDM results, but it cannot guarantee the original criteria system to be simplified to the greatest extent. This paper tries to transform the criteria selection process from subjective judgment to quantitative calculation process, however, subjectivity is still unavoidable when choosing the appropriate threshold value of criteria selection. In real-world applications, we can determine the ultimate threshold value by decreasing the threshold value step by step until a change of MCDM results occur.

## CASE STUDY

### Overview of the Pubugou reservoir

^{2}. Floods, mainly caused by rainfall, occur frequently during the flood season from June to August. The Pubugou reservoir is built for multiple purposes including flood control, hydropower generation, irrigation, water supply and navigation. The total storage capacity, flood limited water level, design flood water level and check flood control water level are 5.06 billion m

^{3}, 841.00 m, 850.24 m and 853.78 m, respectively. The Leshan city is located at the downstream of the Pubugou reservoir, which is an important flood control protected region with a population size of 3,544,000 and GDP of 113.48 billion CNY. Locations of the Daduhe River basin, the Pubugou reservoir and the Leshan city are shown in Figure 2.

### Establishment of the original criteria system

In reservoir flood control operation, decision makers are required to schedule releases considering the safety of reservoirs, the safety of downstream protected regions and flood resources utilization efficiency. Under the premise of flood control safety, the overall benefits should be maximized as much as possible. Due to the fact that it is difficult to quantify flood damage (e.g., economic loss, population impacted, etc.) especially for the real-time scale, flood control target factors (e.g., the highest water level, the terminal water level, the peak discharge of outflow and etc.) are usually chosen as criteria to assess alternatives instead of using flood damage directly (Yu *et al.* 2004; Fu 2008; Wang *et al.* 2011; Zhu *et al.* 2016a).

In this case study, we establish the original criteria system consisting of 10 criteria. This first criterion is the difference between the check flood water level and the highest water level (denoted as *Z _{ch}*–

*Z*), which is used to reflect the reservoir's own safety during flood control operation. The terminal water level is an important criterion used to balance flood control and hydropower generation, a large value of the terminal water level is beneficial to hydropower generation but will lead to future flood control risks, while a small value corresponds to a large flood control storage for future use but will result in a low productive head for hydropower generation. Dynamic control of flood limited water level has been found to be an effective way to increase hydropower generation without causing extra flood control risks during flood seasons (Li

_{max}*et al.*2010; Ding

*et al.*2015). In this case study, the difference between the terminal water level and ideal water level (denoted as

*Z*–

_{e}*Z*) is chosen as a second criterion, and the ideal water level (

_{id}*Z*) equals the flood limited water level, i.e.,

_{id}*Z*= 841.00 m. Furthermore, we use the volume of abandoned water (denoted as

_{id}*W*) to reflect the wasting degree of flood resources. The peak discharge in the downstream flood control section (denoted as

_{ab}*Q*) is chosen to reflect the safety of the downstream protected regions. The duration of reservoir outflow exceeding the safety discharge in the downstream flood control section (denoted as

_{max}*T*) is used to reflect the duration of the downstream flood control section being damaged by the flood event. We use spillover volume exceeding the safety discharge in the downstream flood control section (denoted as

*W*) to reflect the degree of the downstream flood control section being damaged by the flood event. Moreover, different operation alternatives may lead to different risks for both downstream flood the control section and the reservoir itself. Therefore, we consider two criteria for the risk of failure of the dam and its structures (denoted as

_{ex}*Z*) as well as the risk of flooding in the downstream flood control section (denoted as

_{f}*Q*). The risks (i.e.,

_{f}*Z*and

_{f}*Q*) are calculated using the methods proposed by Chen

_{f}*et al.*(2014). Due to the sediment problem, the flood control capability of the reservoir has been reduced to a great extent, thus in the flood control process, we should consider the sediment load (denoted as

*S*) to expand the reservoir's life cycle. The Pubugou reservoir is also built for shipping, which requires the reservoir outflow to be as stable as possible. Therefore, we choose the standard deviation of outflows (denoted as

_{l}*Q*) as an initial criterion to measure the influence of different alternatives to shipping. Among all of the 10 criteria,

_{sd}*Z*–

_{ch}*Z*is a benefit criterion, and other criteria are cost criteria.

_{max}In previous studies (Cheng & Chau 2001, 2002; Chen & Hou 2004; Yu *et al.* 2004; Fu 2008; Wang *et al.* 2011; Zhu *et al.* 2016a, 2016b), the criteria system is subjectively developed and used for MCDM modeling without conducting the criteria selection procedure. In this case study, we apply the proposed methodology to examine the relative importance and relative contribution ratio of each criterion, and further help decision makers to select appropriate criteria for MCDM modeling.

### Generation of flood control operation alternatives

*t*and

*T*are the time sequence and the number of time periods, is reservoir outflow, and represents the lateral inflow between the reservoir and Leshan city.

*t*, is the time interval, and denote the water level and the upper limit of water level at time

*t*, respectively, and represent the terminal water level and ideal water level, respectively, and is the permissible outflow variation limit (500 m

^{3}/s).

An actual flood event is used as the input to the flood control system consists of the Pubugou reservoir and Leshan city. The stepwise trial-and-error algorithm (Zhong *et al.* 2003) is employed to solve the established optimization operation model. By setting different upper limits of water level, 10 non-dominated flood control operation alternatives are generated and their criteria values are obtained after reservoir flood routing, as shown in Table 1.

Alternative no. | Z–_{ch}Z (m) _{max} | Z–_{e}Z (m) _{id} | W (10_{ab}^{6} m^{3}) | Q (m_{max}^{3}/s) | T (h) | W (10_{ex}^{6}m^{3}) | Z _{f} | Q _{f} | S (t) _{l} | Q (m_{sd}^{3}/s) |
---|---|---|---|---|---|---|---|---|---|---|

1 | 4.78 | 5.44 | 1970 | 4810 | 51 | 40 | 0.28 | 0 | 290 | 508 |

2 | 5.28 | 5.10 | 2100 | 4880 | 56 | 48 | 0.13 | 0 | 277 | 561 |

3 | 5.78 | 4.58 | 2220 | 4940 | 59 | 54 | 0.08 | 0 | 260 | 614 |

4 | 6.28 | 4.02 | 2350 | 5020 | 66 | 61 | 0 | 0 | 250 | 667 |

5 | 6.78 | 3.46 | 2480 | 5090 | 72 | 70 | 0 | 0 | 237 | 720 |

6 | 7.28 | 3.06 | 2620 | 5180 | 76 | 74 | 0 | 0 | 224 | 773 |

7 | 7.78 | 2.45 | 2750 | 5270 | 83 | 82 | 0 | 0 | 211 | 826 |

8 | 8.28 | 2.11 | 2880 | 5350 | 87 | 90 | 0 | 0.09 | 198 | 879 |

9 | 8.78 | 1.62 | 2940 | 5450 | 91 | 100 | 0 | 0.16 | 183 | 932 |

10 | 9.28 | 1.20 | 3010 | 5520 | 96 | 120 | 0 | 0.32 | 170 | 985 |

Alternative no. | Z–_{ch}Z (m) _{max} | Z–_{e}Z (m) _{id} | W (10_{ab}^{6} m^{3}) | Q (m_{max}^{3}/s) | T (h) | W (10_{ex}^{6}m^{3}) | Z _{f} | Q _{f} | S (t) _{l} | Q (m_{sd}^{3}/s) |
---|---|---|---|---|---|---|---|---|---|---|

1 | 4.78 | 5.44 | 1970 | 4810 | 51 | 40 | 0.28 | 0 | 290 | 508 |

2 | 5.28 | 5.10 | 2100 | 4880 | 56 | 48 | 0.13 | 0 | 277 | 561 |

3 | 5.78 | 4.58 | 2220 | 4940 | 59 | 54 | 0.08 | 0 | 260 | 614 |

4 | 6.28 | 4.02 | 2350 | 5020 | 66 | 61 | 0 | 0 | 250 | 667 |

5 | 6.78 | 3.46 | 2480 | 5090 | 72 | 70 | 0 | 0 | 237 | 720 |

6 | 7.28 | 3.06 | 2620 | 5180 | 76 | 74 | 0 | 0 | 224 | 773 |

7 | 7.78 | 2.45 | 2750 | 5270 | 83 | 82 | 0 | 0 | 211 | 826 |

8 | 8.28 | 2.11 | 2880 | 5350 | 87 | 90 | 0 | 0.09 | 198 | 879 |

9 | 8.78 | 1.62 | 2940 | 5450 | 91 | 100 | 0 | 0.16 | 183 | 932 |

10 | 9.28 | 1.20 | 3010 | 5520 | 96 | 120 | 0 | 0.32 | 170 | 985 |

### Training the BP neural network

Based on the generated 10 alternatives, ideal and anti-ideal alternatives are determined using Equations (1) and (2). The criteria values are randomly sampled and normalized between the ideal and anti-ideal alternatives by Equations (3) and (6). Then the closeness coefficient is calculated through Equations (4) and (5), and 30 training samples are obtained, as shown in Table 2. The sample series from 1 to 25 are used for training, and the sample series from 26 to 30 are used for verification.

Sample no. | Inputs | Target outputs | Actual outputs | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Z–_{ch}Z (m) _{max} | Z–_{e}Z (m) _{id} | W (10_{ab}^{6} m^{3}) | Q (m_{max}^{3}/s) | T (h) | W (10_{ex}^{6} m^{3}) | Z _{f} | Q _{f} | S (t) _{l} | Q (m_{sd}^{3}/s) | Closeness coefficient | |||

Training samples | 1 | 6.09 | 4.21 | 2708 | 5314 | 83 | 97 | 0.2 | 0.23 | 255 | 847 | 0.286 | 0.269 |

2 | 7.98 | 2.43 | 2272 | 5016 | 64 | 63 | 0.08 | 0.09 | 205 | 646 | 0.714 | 0.741 | |

3 | 5.86 | 4.42 | 2760 | 5350 | 85 | 101 | 0.21 | 0.24 | 261 | 871 | 0.246 | 0.222 | |

… | … | … | … | … | … | … | … | … | … | … | … | … | |

24 | 9.05 | 1.41 | 2022 | 4846 | 53 | 44 | 0.01 | 0.02 | 176 | 532 | 0.95 | 0.914 | |

25 | 5.73 | 4.55 | 2792 | 5371 | 87 | 103 | 0.22 | 0.25 | 265 | 885 | 0.214 | 0.200 | |

Verification samples | 26 | 4.96 | 5.27 | 2968 | 5492 | 94 | 117 | 0.27 | 0.31 | 285 | 966 | 0.036 | 0.111 |

27 | 7.66 | 2.73 | 2344 | 5066 | 67 | 69 | 0.1 | 0.12 | 213 | 680 | 0.636 | 0.666 | |

28 | 6.94 | 3.4 | 2511 | 5179 | 74 | 82 | 0.15 | 0.17 | 232 | 756 | 0.472 | 0.472 | |

29 | 9.05 | 1.41 | 2022 | 4846 | 53 | 44 | 0.01 | 0.02 | 176 | 532 | 0.95 | 0.918 | |

30 | 7.17 | 3.19 | 2459 | 5144 | 72 | 78 | 0.13 | 0.15 | 226 | 732 | 0.532 | 0.530 |

Sample no. | Inputs | Target outputs | Actual outputs | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Z–_{ch}Z (m) _{max} | Z–_{e}Z (m) _{id} | W (10_{ab}^{6} m^{3}) | Q (m_{max}^{3}/s) | T (h) | W (10_{ex}^{6} m^{3}) | Z _{f} | Q _{f} | S (t) _{l} | Q (m_{sd}^{3}/s) | Closeness coefficient | |||

Training samples | 1 | 6.09 | 4.21 | 2708 | 5314 | 83 | 97 | 0.2 | 0.23 | 255 | 847 | 0.286 | 0.269 |

2 | 7.98 | 2.43 | 2272 | 5016 | 64 | 63 | 0.08 | 0.09 | 205 | 646 | 0.714 | 0.741 | |

3 | 5.86 | 4.42 | 2760 | 5350 | 85 | 101 | 0.21 | 0.24 | 261 | 871 | 0.246 | 0.222 | |

… | … | … | … | … | … | … | … | … | … | … | … | … | |

24 | 9.05 | 1.41 | 2022 | 4846 | 53 | 44 | 0.01 | 0.02 | 176 | 532 | 0.95 | 0.914 | |

25 | 5.73 | 4.55 | 2792 | 5371 | 87 | 103 | 0.22 | 0.25 | 265 | 885 | 0.214 | 0.200 | |

Verification samples | 26 | 4.96 | 5.27 | 2968 | 5492 | 94 | 117 | 0.27 | 0.31 | 285 | 966 | 0.036 | 0.111 |

27 | 7.66 | 2.73 | 2344 | 5066 | 67 | 69 | 0.1 | 0.12 | 213 | 680 | 0.636 | 0.666 | |

28 | 6.94 | 3.4 | 2511 | 5179 | 74 | 82 | 0.15 | 0.17 | 232 | 756 | 0.472 | 0.472 | |

29 | 9.05 | 1.41 | 2022 | 4846 | 53 | 44 | 0.01 | 0.02 | 176 | 532 | 0.95 | 0.918 | |

30 | 7.17 | 3.19 | 2459 | 5144 | 72 | 78 | 0.13 | 0.15 | 226 | 732 | 0.532 | 0.530 |

### Selection of criteria

Z_{ch}–Z_{max} | Z_{e}–Z_{id} | W_{ab} | Q_{max} | T | W_{ex} | Z_{f} | Q_{f} | S_{l} | Q_{sd} | |
---|---|---|---|---|---|---|---|---|---|---|

0.346 | 0.030 | 0.021 | 0.062 | 0.036 | 0.325 | 0.248 | 0.293 | 0.037 | 0.024 | |

16.51% | 3.50% | 7.41% | 24.80% | 4.70% | 5.41% | 12.92% | 14.91% | 3.80% | 6.04% | |

0.519 | −1.217 | −1.046 | −0.051 | −1.010 | 0.007 | 0.268 | 0.402 | −1.090 | −1.077 | |

Decision | Select | Delete | Delete | Select | Delete | Select | Select | Select | Delete | Delete |

Z_{ch}–Z_{max} | Z_{e}–Z_{id} | W_{ab} | Q_{max} | T | W_{ex} | Z_{f} | Q_{f} | S_{l} | Q_{sd} | |
---|---|---|---|---|---|---|---|---|---|---|

0.346 | 0.030 | 0.021 | 0.062 | 0.036 | 0.325 | 0.248 | 0.293 | 0.037 | 0.024 | |

16.51% | 3.50% | 7.41% | 24.80% | 4.70% | 5.41% | 12.92% | 14.91% | 3.80% | 6.04% | |

0.519 | −1.217 | −1.046 | −0.051 | −1.010 | 0.007 | 0.268 | 0.402 | −1.090 | −1.077 | |

Decision | Select | Delete | Delete | Select | Delete | Select | Select | Select | Delete | Delete |

According to Table 3, the ranking of the relative importance is determined as: *Z _{ch}*–

*Z*>

_{max}*W*>

_{ex}*Q*>

_{f}*Z*>

_{f}*Q*>

_{max}*S*>

_{l}*T*>

*Z*–

_{e}*Z*>

_{id}*Q*>

_{sd}*W*. The ranking of the relative contribution ratio is:

_{ab}*Q*>

_{max}*Z*–

_{ch}*Z*>

_{max}*Q*>

_{f}*Z*>

_{f}*W*>

_{ab}*Q*>

_{sd}*W*>

_{ex}*T*>

*S*>

_{l}*Z*–

_{e}*Z*. Furthermore, the ranking of the comprehensive discrimination index is then determined as:

_{id}*Z*–

_{ch}*Z*>

_{max}*Q*>

_{f}*Z*>

_{f}*W*>

_{ex}*Q*>

_{max}*T*>

*W*>

_{ab}*Q*>

_{sd}*S*>

_{l}*Z*–

_{e}*Z*. It can be seen from Table 3 that the comprehensive discrimination indices of five criteria (i.e.,

_{id}*T*;

*W*;

_{ab}*Q*;

_{sd}*S*;

_{l}*Z*–

_{e}*Z*) are smaller than −1, indicating that the relative importance and relative contribution ratio of these five criteria can be considered significantly smaller than the average level of all criteria over one order of magnitude. According to the principle and threshold value of criteria selection, they should be deleted from the original criteria system. The comprehensive discrimination indices of the other five criteria (i.e.,

_{id}*Z*–

_{ch}*Z*;

_{max}*Q*;

_{f}*Z*;

_{f}*W*;

_{ex}*Q*) are greater than −1 and these criteria should be selected for further MCDM modeling.

_{max}### Rationality analysis of criteria selection

According to Table 3, it is reasonable to delete *T*, *W _{ab}*,

*Q*,

_{sd}*S*and

_{l}*Z*–

_{e}*Z*, because they get the worst performances on the relative importance as well as relative contribution ratio, and their comprehensive discrimination indices are smaller than the threshold value as well. The comprehensive discrimination indices of

_{id}*Z*–

_{ch}*Z*,

_{max}*Q*and

_{f}*Z*are greater than zero, indicating that the relative importance and relative contribution ratio of these three criteria are better than the average level of all criteria because they get relatively good performances on the relative importance and relative contribution ratio, therefore, it is reasonable to select these three criteria. It can be found that

_{f}*Q*has a large relative contribution ratio but a small relative importance, while

_{max}*W*has a large relative importance but a small relative contribution ratio. Therefore, if we only examine the relative importance when selecting criteria, those criteria with large relative contribution ratios may be deleted by mistake, and vice versa. In this paper, we recommend using the comprehensive discrimination index, which combines both the relative importance and relative contribution ratio, for criteria selection.

_{ex}In hydrological forecasting problems, forecasted results are usually compared with the benchmark results (i.e., the measures values) to test the performance of forecasting models (Zhu *et al.* 2016a). However, such a benchmark alternative in MCDM problems of flood control operation is difficult to determine to examine the effectiveness and rationality of the proposed methodology. Instead, multiple MCDM methods should be simultaneously used to test the sensitivity and rationality of the results (Hajkowicz & Collins 2007). Therefore, we use the TOPSIS model (Hwang & Yoon 1981), fuzzy matter-element model (Cai 1994) and fuzzy optimization model (Fu 2008) to evaluate the 10 flood control operation alternatives (listed in Table 1) simultaneously. These three models differ basically depending on how they: (1) determine marginal evaluation on each criterion; and (2) aggregate marginal evaluations across criteria to achieve a global evaluation (Durbach & Stewart 2012). The TOPSIS model determines the best alternative which is simultaneously closest to the ideal point and farthest from the anti-ideal point. The fuzzy matter-element model is a classical MCDM method based on the theory of matter element analysis. The fuzzy optimization model uses fuzzy ideal and anti-ideal weight distances to calculate fuzzy membership degrees, by which the rank of candidate alternatives are determined directly without a need to compare fuzzy numbers. The original criteria system and the selected criteria system are used as the inputs of the three MCDM models, and each criterion is assigned with equal weight. The MCDM results of the three models before and after criteria selection are compared in Table 4. It can be seen from Table 4 that all these three MCDM models get consistent ranking orders of the 10 alternatives before and after criteria selection. The results demonstrate that deleting *T*, *W _{ab}*,

*Q*,

_{sd}*S*and

_{l}*Z*–

_{e}*Z*does not influence the MCDM results because these criteria are inherently unimportant to the MCDM results as well as having a weak sensitivity. Before selection, some criteria may be highly correlated and measure the same underlying factor, this may be another reason for deleting some criteria without changing the MCDM results.

_{id}Alternative no. | TOPSIS model | Fuzzy matter-element model | Fuzzy optimization model | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|

Before | After | Before | After | Before | After | |||||||

c _{i} | Rank | c _{i} | Rank | ρH _{i} | Rank | ρH _{i} | Rank | u _{i} | Rank | u _{i} | Rank | |

1 | 0.515 | 9 | 0.508 | 9 | 0.215 | 9 | 0.201 | 9 | 0.480 | 9 | 0.488 | 9 |

2 | 0.698 | 7 | 0.698 | 7 | 0.398 | 7 | 0.372 | 7 | 0.584 | 7 | 0.591 | 7 |

3 | 0.784 | 5 | 0.786 | 5 | 0.486 | 5 | 0.492 | 5 | 0.671 | 5 | 0.683 | 5 |

4 | 0.892 | 1 | 0.889 | 1 | 0.628 | 1 | 0.644 | 1 | 0.739 | 1 | 0.742 | 1 |

5 | 0.887 | 2 | 0.884 | 2 | 0.593 | 2 | 0.602 | 2 | 0.734 | 2 | 0.736 | 2 |

6 | 0.880 | 3 | 0.881 | 3 | 0.581 | 3 | 0.571 | 3 | 0.730 | 3 | 0.731 | 3 |

7 | 0.872 | 4 | 0.871 | 4 | 0.561 | 4 | 0.510 | 4 | 0.723 | 4 | 0.726 | 4 |

8 | 0.772 | 6 | 0.772 | 6 | 0.443 | 6 | 0.451 | 6 | 0.643 | 6 | 0.651 | 6 |

9 | 0.658 | 8 | 0.666 | 8 | 0.329 | 8 | 0.318 | 8 | 0.555 | 8 | 0.562 | 8 |

10 | 0.499 | 10 | 0.492 | 10 | 0.183 | 10 | 0.162 | 10 | 0.398 | 10 | 0.412 | 10 |

Alternative no. | TOPSIS model | Fuzzy matter-element model | Fuzzy optimization model | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|

Before | After | Before | After | Before | After | |||||||

c _{i} | Rank | c _{i} | Rank | ρH _{i} | Rank | ρH _{i} | Rank | u _{i} | Rank | u _{i} | Rank | |

1 | 0.515 | 9 | 0.508 | 9 | 0.215 | 9 | 0.201 | 9 | 0.480 | 9 | 0.488 | 9 |

2 | 0.698 | 7 | 0.698 | 7 | 0.398 | 7 | 0.372 | 7 | 0.584 | 7 | 0.591 | 7 |

3 | 0.784 | 5 | 0.786 | 5 | 0.486 | 5 | 0.492 | 5 | 0.671 | 5 | 0.683 | 5 |

4 | 0.892 | 1 | 0.889 | 1 | 0.628 | 1 | 0.644 | 1 | 0.739 | 1 | 0.742 | 1 |

5 | 0.887 | 2 | 0.884 | 2 | 0.593 | 2 | 0.602 | 2 | 0.734 | 2 | 0.736 | 2 |

6 | 0.880 | 3 | 0.881 | 3 | 0.581 | 3 | 0.571 | 3 | 0.730 | 3 | 0.731 | 3 |

7 | 0.872 | 4 | 0.871 | 4 | 0.561 | 4 | 0.510 | 4 | 0.723 | 4 | 0.726 | 4 |

8 | 0.772 | 6 | 0.772 | 6 | 0.443 | 6 | 0.451 | 6 | 0.643 | 6 | 0.651 | 6 |

9 | 0.658 | 8 | 0.666 | 8 | 0.329 | 8 | 0.318 | 8 | 0.555 | 8 | 0.562 | 8 |

10 | 0.499 | 10 | 0.492 | 10 | 0.183 | 10 | 0.162 | 10 | 0.398 | 10 | 0.412 | 10 |

*Note:**c _{i}*,

*ρH*and

_{i}*u*represent the comprehensive evaluation index of TOPSIS model, fuzzy matter-element model and fuzzy optimization model, respectively.

_{i}## DISCUSSION AND CONCLUSIONS

In reservoir flood control operation, candidate alternatives are ranked through MCDM approaches, and selecting criteria is the most important part of the MCDM process. This paper proposed a method to select criteria for MCDM of reservoir flood control operation based on the BP neural network. The main conclusions are summarized as follows.

Based on the concepts of ideal and anti-ideal points, we proposed a method to generate training samples of the BP neural network via stochastic simulation. The topological structure of a three-layer BP neural network used for criteria selection was established. The relative importance of criteria was derived via the learned connection weights of a trained BP neural network, and its calculation method was proposed.

The sensitivity curve method was employed to conduct sensitivity analysis, and the relative contribution ratio was defined to quantify the relative sensitivity strength of each criterion.

The principle and threshold value of criteria selection were presented based on the comprehensive discrimination index defined by the combination of the relative importance and relative contribution ratio.

We applied the proposed methodology to a case study. The original criteria system consisting of 10 criteria was established. Then, the flood control optimization operation model based on the rule of maximum flood peak reduction was developed to generate 10 flood control operation alternatives. The results from the case study indicate that the method of generating training samples via stochastic simulation can guarantee a high accuracy of network training. The relative importance based on the learned connection weights can provide valuable information about the inherent contribution of each criterion to the MCDM results. This paper transforms the criteria selection process from subjective judgment to quantitative calculation process, and provides an effective tool for decision makers to select criteria before MCDM modeling of reservoir flood control operation.

It should be mentioned that the case study performed in the flood control system consists of a single reservoir and a single flood control section. However, compared with multi-reservoir systems, the original criteria system will be more complicated since more criteria are required to assess the performances of candidate alternatives, and the complexity will increase with the increasing number of reservoirs. Consequently, for large-scale multi-reservoir systems, the proposed methodology may have more effective applications to select appropriate criteria for MCDM modeling, which can further reduce the dimensionality of the original criteria system and simplify the MCDM modeling process.

## ACKNOWLEDGEMENTS

This study was supported by the National Natural Science Foundation of China (Grant No. 51579068), the Special Fund for Public Welfare Industry of the Ministry of Water Resources of China (Grant No. 201501007), the Major Science and Technology Program for Water Pollution Control and Treatment (Grant No. 2014ZX07405002), the Fundamental Research Funds for the Central Universities (Grant No. 2017B40614; 2016B40214; 2015B05414), and the Research Innovation Program for College Graduates in Jiangsu Province of China (Grant No. KYLX16_0738; KYZZ15_0135). The first author was also supported by a fellowship from the China Scholarship Council for his visit to the University of California, Los Angeles.