An artificial neural network (ANN) is a powerful data-driven modeling tool. The selection of the input variable is an important task in the development of an ANN model. However, at present in ANN modeling, the input variables are usually determined by trial and error methods. Further, the ANN modeler usually selects a single ‘good’ result, and accepts it as the final result without detailed explanation of the initial weight parameter. In this way, the ANN model cannot guarantee that the model will produce the optimal result for later predictions. In this study, the ANN ensemble model with exploratory factor analysis (EFA) was developed and applied to three stations in the Nakdong River, Korea for the 1-day ahead streamflow forecasting. EFA was used to select the input variables of the ANN model, and then the ensemble modeling was applied to estimate the performance of the ANN to remove the influence of initial weight parameters on the model results. In the result, the ANN ensemble model with the input variables proposed by EFA produced more accurate and reliable forecasts than other models with several combinations of input variables. Nash–Sutcliffe efficiency (NSE) results in the validation were 0.92, 0.95, and 0.97, respectively.

INTRODUCTION

An artificial neural network (ANN) is a powerful data-driven modeling tool that can capture the complex and non-linear relationships between inputs and outputs. Thus, an ANN has become a new tool as an efficient model in hydrological forecasts due to the uncertainties of which hydrological processes inherently have. However, in spite of the increasing studies for hydrological forecasts using the ANN model, the modeler often developed the ANNs without considering the determination of appropriate input variables. The selection of input variables for the present ANN modeling is generally based on a priori knowledge of causal variables and statistical analysis of potential inputs and outputs, and is usually determined by trial and error methods by changing the causal variables affecting the output variables (e.g., Maier et al. 2010; May et al. 2011). Maier & Dandy (2000), Bowden (2003), Maier et al. (2010) and May et al. (2011) reviewed the input variable selection methods which have been applied to ANN applications. In the reviews, input variable selection methods were classified into two main classes which were model-based and model-free (filtered) methods. Model-based approaches rely on the development of a number of ANN models with different inputs to determine which of the input candidates should be included. The primary disadvantage of this approach is that it is time-consuming, as the results of ANNs with all possible combinations of the input candidates have to be compared. In contrast to model-based approaches, model-free approaches to input selection do not rely on the performance of trained ANN models. Model-free approaches are usually based on bivariate statistics such as correlation and mutual information. This approach can reduce the computational burden and the overall effort of ANN development. However, the problem of this approach is the redundancy of input variables and which makes it difficult to assess the optimality of the input variables obtained since this approach gives the relevance between the input candidates and the desired output, not how well the input candidates represent the information of the desired output. The authors previously mentioned above concluded that no consensus had been reached regarding suitable methods for input variable selection and there was a need for a more considered approach.

Another problem of ANNs is that the ANN model gives different results for the same original inputs depending on the initial weight parameter set before training the neural network. Since the backpropagation (BP) optimization method used in the training of the networks is unstable, a different weight parameter set will be obtained each time the optimization process is launched (Boucher et al. 2009). This led to research for the optimal initial weights of neural networks (Kolen & Pollack 1990; Yam & Chow 1995). Several methods involving genetic algorithms have been implemented to find the optimal initial weight parameters and enhance the accuracy of ANN models (Venkatesan et al. 2009; Chang et al. 2012; Mulia et al. 2013). These studies reached a consensus that the optimal initial weight parameters were very sensitive to training algorithms and data structures, and there were no fixed optimal initial weight parameters which are universally applicable to the variety of data structures and training algorithms. Therefore, ensemble techniques have been applied due to the basic fact that selection of the weights is an optimization problem with many local minima (Hansen & Salamon 1990). Laucelli et al. (2007) applied ensemble modeling and genetic programing to hydrological forecasts and showed the error due to the variance is effectively eliminated by using an average model (ensemble model) as the resulting model of many runs. Boucher et al. (2009) developed the 1-day ahead ensemble ANN model for streamflow forecast. This study showed that the random initialization of the weight parameters mainly accounted for the uncertainty linked to the optimization of the model's parameters and ensemble modeling can reduce the uncertainty using the proper assessment tools for the performance of ensemble models. Zamani et al. (2009) developed an ensemble ANN model with the Kalman filter which corrects the outputs of the ANNs to find the best estimate of the wave height, and showed the prediction results were improved as the number of ensemble members increased. These studies indicate the ANN model cannot guarantee that the model will produce an optimal result without considering appropriate methods for the initial weight parameters.

This study attempted to employ exploratory factor analysis (EFA) to propose a useful input variable selection method for the ANN model. The EFA used in this study, which is classified to a model-free approach, can select the input variables required to describe little or no predictive information of the desired output, considering the degree of redundancy. In this study, in addition to EFA application, the ensemble modeling for various initial weight parameter sets was applied in order to estimate the ANN performance without the effect of initial weight parameters on the variance of ANN model results. Figure 1 illustrates the research questions and approaches applied in this study (e.g., Maier et al. 2010). To estimate the performance of ANN models with the proposed approaches, the one-step ahead forecasting ANN ensemble model was developed and applied to Nakdong River in Korea.

Figure 1

Research questions and approaches applied in this study.

Figure 1

Research questions and approaches applied in this study.

MODEL DEVELOPMENT

ANN model

The ANN model is a computational model inspired by the biological neural network in the human brain. The concept of artificial neurons was first introduced in 1943 (McCulloch & Pitts 1990), and applications of ANNs in research areas began with the introduction of the BP training algorithm for feedforward ANNs in 1986 (Rumelhart et al. 1986). The main difference between the various types of ANN model involves the network architecture and the method to determine the weights (Caudill & Butler 1992). The way in which the neurons of the ANN are structured and the neural networks learn determines the architectures of ANN. In general, there are three fundamentally different classes of network architectures based on the way by which the neurons of ANNs are structured (Haykin 1999). The first is a single-layer feedforward network without hidden layers. The second is a multilayer feedforward network with more than one hidden layer. The third is a recurrent neural network with at least more than one feedback loop.

In this study, the multilayer feedforward neural network (MFNN) with one hidden layer was used because it is able to approximate most of the non-linear functions demanded by practice (Mulia et al. 2013). The most common and standard training algorithm is the BP training algorithm, the central idea of which is that the error for the neurons of the hidden layer are determined by backpropagating the error of the neurons of the output layer, as shown in Figure 2. There are a number of variations in BP training algorithms on the basic algorithm that are based on other standard optimization techniques, such as steepest descent algorithm, conjugate gradient algorithm, and Newton's method. The steepest descent algorithm is the simplest and often the slowest. The conjugate gradient algorithm and Newton's method generally provide faster convergence. Especially, the Levenberg–Marquardt (LM) algorithm has been very successfully applied to hydrological forecasts, providing significant speedup and faster convergence than the steepest descent algorithm and conjugate gradient-based algorithms (Zamani et al. 2009). In this study, the LM algorithm was applied for training the network.

Figure 2

Backpropagation training algorithm of MFNN.

Figure 2

Backpropagation training algorithm of MFNN.

The LM algorithm is a variation of Newton's method designed to minimize functions that are sums of squares of other non-linear functions. This is very well suited to neural networks training, where the performance function is the sum of squared error. Newton's method is based on the second-order Taylor series expansion, while the derivation of the steepest descent algorithm was based on the first-order Taylor series expansion (Hagan et al. 1996). The LM algorithm is given in Equation (1) 
formula
1
where w is the weight vector and v is the error vector. J is the Jacobian matrix. I is the identity matrix. is the damping factor. Subscript k is an epoch or iteration number.
This algorithm has the very useful feature that as increases, it approaches the steepest descent algorithm with a small learning rate. 
formula
2
 
formula
3
where is a performance function (the sum of squared error), N is the number of the data set.

As decreases to zero, the algorithm becomes Gauss–Newton. The algorithm begins with set to some small value (e.g., ). If a step does not yield a smaller value for , then the step is repeated with multiplied by some factor (e.g., ). Eventually should decrease, since we would be taking a small step in the direction of steepest descent. If a step does produce a smaller value for , then is divided by for the next step, so that the algorithm will approach Gauss–Newton, which should provide faster convergence. The algorithm provides a nice compromise between the speed of Newton's method and the guaranteed convergence of steepest descent (Hagan et al. 1996).

EFA

The EFA is one of the factor analysis techniques whose goal is to identify the underlying relationships between measured variables (Norris & Lecavalier 2010). It is generally used when the researcher has no a priori hypothesis about factors or patterns of measured variables (Finch & West 1997). EFA has been applied to research in the field of psychology, economics (business and marketing), medicine, and biology (Fabrigar et al. 1999; Peterson 2000; Kelton et al. 2010; Lucas et al. 2010). EFA in water resource management was applied to interpret the relationships between the observed data set and variables from monitoring stations at various locations in rivers. Liu et al. (2003) attempted to apply the EFA to groundwater samples. The results showed that two factors affecting the seawater salinization and arsenic pollution were defined, and describe the main hydrochemical processes and identify the possible cause of groundwater salinization and arsenic pollution. Boyacioglu (2006) applied EFA to analyze the surface water quality data and identified the practical pollution indicators to delineate the domestic and agricultural pollution in the basin.

EFA assumes that the variance in the observed variables is due to the presence of one or more latent variables (common factors) that exert causal influence on these observed variables, and distinguishes the contributions of individual variables to the variance of output variables from the interpretation of the latent factors in each variable. A large number of input variables (redundant input variables) might include irrelevant and noisy variables, and lead to increasing the modeling error. Thus, the input variable set will contain the fewest input variables required to describe the variance of the output variable, with a minimum degree of redundancy (May et al. 2011). From this point of view, the EFA is very well suited to determine input variables of the ANN model that significantly affect the output variables.

The common EFA model is defined as 
formula
4
where the observations with p variables are and is the common factor matrix for the number of factor m. is the factor loading matrix. is the specific factor or residual errors. From this model, the factor loading value L for common factors F that affect the original variables X can be obtained with a factoring method. The procedure of EFA consists of five steps, as shown in Figure 3. The first is standardizing the row data. The second is making the covariance matrix. The third is calculating the eigenvalues and eigenvectors and selecting the number of common factors. The fourth is calculating the factor loading value of the selected common factors. The last is interpretation of the common factors. In this study, the principal component factoring method was used for calculating the factor loading value. It is the most widely used and basic factoring method in EFA. This factoring method has similarities to principal component analysis (PCA). Thus, EFA is sometimes confused with PCA. However, EFA and PCA are very different. EFA assumes that the covariance in the observed variables is due to the presence of one or more latent variables (common factors) that exert causal influence on these observed variables. Therefore, it enables the contributions of individual variables to the variance of the desired variables by the interpretation of common factors to be distinguished. In contrast, PCA makes no assumption about an underlying causal model. PCA is simply a variable reduction analysis that typically results in a relatively small number of principal components (PCs) which are the linear compound of the variables instead of the direct use of variables. Therefore, it is not possible to distinguish the contributions of individual variables to the variance of the desired variables.
Figure 3

Overview of the steps in EFA.

Figure 3

Overview of the steps in EFA.

The common EFA model Equation (4) is based on two assumptions. First, the error terms are independent of one another, and such that and , where E denotes the expected value (mean value) of the variable for all observed data set and is the specific variance. Second, common factors are independent of one another and of the error terms, and are such that and . From these assumptions, the covariance matrix in EFA can be defined as 
formula
5
where .
A vector of observations with p variables is 
formula
6
The covariance matrix ∑ is expressed as 
formula
7
 
formula
8
The eigenvalues and corresponding eigenvectors can be obtained from Equation (8). In PCA, the covariance matrix can be expressed as the sum of p eigenvalues multiplied by their eigenvectors and their transposes. The idea behind the principal component factoring method is approximating this expression. 
formula
9
This result yields the following estimator for the ijth factor loading value, which indicates the effect of the jth factor on the ith variables. 
formula
10
where is an ith eigenvector and is a jth component of the ith eigenvector.
Equation (4) can be rewritten as 
formula
11
In the case , the residual error terms become zero and in the case , the terms of the right-hand side for become the residual error term . From Equation (11) and the assumption of the EFA model, the variance of the original variable can be defined as 
formula
12

The first term, communality of the variables, is the part that is explained by the common factor and means how well the common factors explain the total variance of each variable, and is calculated as the square of each factor loading value. The second term, the specific variance, is the part of the variance that is not accounted for by the common factors. As communality is close to 1, it means almost all variances of the variable are explained by the common factors.

Ensemble modeling

Ensemble techniques have been applied in hydrology and environmental science as an approach to enhance the skill of forecasts with considerable success (Maqsood et al. 2004; Jeong & Kim 2005; Araghinejad et al. 2011). The motivation for this procedure is based on the idea that by combining the outputs of several individual predictors one might improve on the performance of a single generic one (Wolpert 1992; Krogh & Vedelsby 1995). The technique of combining multiple models into a single one is referred to as ensemble modeling (Laucelli et al. 2007). The most common practice of ensemble modeling in ANN is to average the ensemble members to obtain a forecast result (Hansen & Salamon 1990; Zhou et al. 2002; Zhang 2007). This averaged output of many neural networks trained on the same data has less variability than the outcome of a single network (Boucher et al. 2009). The application of an ensemble technique is divided into two steps. The first step is to create individual ensemble members, and the second step is the appropriate combination of outputs of the ensemble members to produce the most optimal output (Araghinejad et al. 2011).

In this study, the ensemble modeling technique was applied to estimate the performance of the ANN without the influence of initial weight parameters on the model results. For the first step, neural networks with 100 randomly generated initial weight sets for the same structure were used in parallel for the same original training data set. For the second step, Nash–Sutcliffe efficiency (NSE), which is given in Equation (13), was used. NSE is a normalized statistic that determines the relative magnitude of the residual variance (‘noise’) compared to the measured data variance (‘information’) (Nash & Sutcliffe 1970). NSE indicates how well a mean result of the ANN ensemble model fitted a set of test or validation output data. 
formula
13
 
formula
14
 
formula
15
where is the total sum of squared, and is the sum of squares of residuals. is an ith observed value or target value. is a mean value of for all observed data set. is a mean result of an ANN ensemble model for the ith data set.
In addition to NSE, the root-mean-square error (RMSE) and inter-quartile range (IQR, distance between 25th and 75th percentile) were used for a global estimate of the results of ensemble modeling. RMSE was used to measure the whole bias error between mean results of the ANN ensemble model and observed values. 
formula
16
where n is the number of observed data set. IQR, shown in Figure 4, was used to measure variance error of the ANN ensemble model itself. 
formula
17
where and are 25th and 75th percentile value of ANN ensemble model results for the ith data set, respectively.
Figure 4

IQR of ANN ensemble model results.

Figure 4

IQR of ANN ensemble model results.

The effect of initial weight parameters was removed through the aggregation of the results of individual ensemble members. Figure 5 illustrates the ANN ensemble modeling technique used in this study.

Figure 5

Schematic of ANN ensemble modeling for random initial weight parameters.

Figure 5

Schematic of ANN ensemble modeling for random initial weight parameters.

MODEL APPLICATION

Study site and input data

The Nakdong River, one of the four major rivers in South Korea, is located in the southeastern part of the Korean peninsula. The river is about 525 km long and has a drainage area of 23,817 km2. Busan, one of the biggest cities in South Korea, is located near the river mouth. About 7 million people reside within the basin, and more than 13 million people intake drinking water from this river. Moreover, river flow and environmental characteristics have been changed drastically after the Four Major River Restoration Project, constructed from 2010 to 2012. Thus, management of water uses and the flood control in this area has been a major national project.

The daily streamflow data from 23 stations and the daily rainfall data from 12 rainfall stations in the Nakdong River basin for the years 2003–2009 were used in this study to develop the 1-day ahead streamflow forecasting ANN model. The locations of the streamflow and rainfall stations in the Nakdong River basin are shown in Figure 6. Rainfall (P) and streamflow (Q) are essential input variables for the streamflow forecasting ANN model (Minns & Hall 1996; Campolo et al. 1999; Anctil & Rat 2005). The time range of rainfall and streamflow inputs should ideally be adjusted according to the time of concentration or the traveling time of each predicted station in the basin (Anctil & Rat 2005). Considering the critical rainfall duration 40 hours for a 200-year design flood of the Nakdong River (MOLIT 2004) and the preceding rainfall, input variables of daily rainfall and streamflow were discretized into input variable candidates with time t(day), t − 1, t − 2 for the rainfall and the streamflow and the desired output time t + 1 for the streamflow. Thus, in this study, the desired output Qt+1 of each stream gaging station has the input variable candidates Qt, Qt−1, Qt−2, Pt, Pt−1, and Pt−2. One hundred and twenty-eight variables (105 input variable candidates and 23 desired output) are shown in Table 1. Symbols instead of variable names were used for convenience in this study.

Table 1

Total inputs for EFA

Symbol Name Type Symbol Name Type Symbol Name Type 
v1 Andong Qt+1 v44 Samlangjin Qt v87 Jindong Qt−2 
v2 Dalge Qt+1 v45 Yangsan Qt v88 Susan Qt−2 
v3 Sabor Qt+1 v46 Gupo Qt v89 Milyang Qt−2 
v4 Yonggok Qt+1 v47 Andong Qt−1 v90 Samlangjin Qt−2 
v5 Nakdong Qt+1 v48 Dalge Qt−1 v91 Yangsan Qt−2 
v6 Ilsungyo Qt+1 v49 Sabor Qt−1 v92 Gupo Qt−2 
v7 Sunsan Qt+1 v50 Yonggok Qt−1 v93 Andong Pt 
v8 Gumi Qt+1 v51 Nakdong Qt−1 v94 Deagu Pt 
v9 Wegwan Qt+1 v52 Ilsungyo Qt−1 v95 Ueisung Pt 
v10 Sungju Qt+1 v53 Sunsan Qt−1 v96 Guchang Pt 
v11 Sungseo Qt+1 v54 Gumi Qt−1 v97 Gumi Pt 
v12 Hwawon Qt+1 v55 Wegwan Qt−1 v98 Habcheon Pt 
v13 Koryunggyo Qt+1 v56 Sungju Qt−1 v99 Jinju Pt 
v14 Hyunpung Qt+1 v57 Sungseo Qt−1 v100 Milyang Pt 
v15 Jucko Qt+1 v58 Hwawon Qt−1 v101 Sanchung Pt 
v16 Jukpogyo Qt+1 v59 Koryunggyo Qt−1 v102 Yungcheon Pt 
v17 Jungam Qt+1 v60 Hyunpung Qt−1 v103 Youngju Pt 
v18 Jindong Qt+1 v61 Jucko Qt−1 v104 Mungyeoung Pt 
v19 Susan Qt+1 v62 Jukpogyo Qt−1 v105 Andong Pt−1 
v20 Milyang Qt+1 v63 Jungam Qt−1 v106 Deagu Pt−1 
v21 Samlangjin Qt+1 v64 Jindong Qt−1 v107 Ueisung Pt−1 
v22 Yangsan Qt+1 v65 Susan Qt−1 v108 Guchang Pt−1 
v23 Gupo Qt+1 v66 Milyang Qt−1 v109 Gumi Pt−1 
v24 Andong Qt v67 Samlangjin Qt−1 v110 Habcheon Pt−1 
v25 Dalge Qt v68 Yangsan Qt−1 v111 Jinju Pt−1 
v26 Sabor Qt v69 Gupo Qt−1 v112 Milyang Pt−1 
v27 Yonggok Qt v70 Andong Qt−2 v113 Sanchung Pt−1 
v28 Nakdong Qt v71 Dalge Qt−2 v114 Yungcheon Pt−1 
v29 Ilsungyo Qt v72 Sabor Qt−2 v115 Youngju Pt−1 
v30 Sunsan Qt v73 Yonggok Qt−2 v116 Mungyeoung Pt−1 
v31 Gumi Qt v74 Nakdong Qt−2 v117 Andong Pt−2 
v32 Wegwan Qt v75 Ilsungyo Qt−2 v118 Deagu Pt−2 
v33 Sungju Qt v76 Sunsan Qt−2 v119 Ueisung Pt−2 
v34 Sungseo Qt v77 Gumi Qt−2 v120 Guchang Pt−2 
v35 Hwawon Qt v78 Wegwan Qt−2 v121 Gumi Pt−2 
v36 Koryunggyo Qt v79 Sungju Qt−2 v122 Habcheon Pt−2 
v37 Hyunpung Qt v80 Sungseo Qt−2 v123 Jinju Pt−2 
v38 Jucko Qt v81 Hwawon Qt−2 v124 Milyang Pt−2 
v39 Jukpogyo Qt v82 Koryunggyo Qt−2 v125 Sanchung Pt−2 
v40 Jungam Qt v83 Hyunpung Qt−2 v126 Yungcheon Pt−2 
v41 Jindong Qt v84 Jucko Qt−2 v127 Youngju Pt−2 
v42 Susan Qt v85 Jukpogyo Qt−2 v128 Mungyeoung Pt−2 
v43 Milyang Qt v86 Jungam Qt−2 – – – 
Symbol Name Type Symbol Name Type Symbol Name Type 
v1 Andong Qt+1 v44 Samlangjin Qt v87 Jindong Qt−2 
v2 Dalge Qt+1 v45 Yangsan Qt v88 Susan Qt−2 
v3 Sabor Qt+1 v46 Gupo Qt v89 Milyang Qt−2 
v4 Yonggok Qt+1 v47 Andong Qt−1 v90 Samlangjin Qt−2 
v5 Nakdong Qt+1 v48 Dalge Qt−1 v91 Yangsan Qt−2 
v6 Ilsungyo Qt+1 v49 Sabor Qt−1 v92 Gupo Qt−2 
v7 Sunsan Qt+1 v50 Yonggok Qt−1 v93 Andong Pt 
v8 Gumi Qt+1 v51 Nakdong Qt−1 v94 Deagu Pt 
v9 Wegwan Qt+1 v52 Ilsungyo Qt−1 v95 Ueisung Pt 
v10 Sungju Qt+1 v53 Sunsan Qt−1 v96 Guchang Pt 
v11 Sungseo Qt+1 v54 Gumi Qt−1 v97 Gumi Pt 
v12 Hwawon Qt+1 v55 Wegwan Qt−1 v98 Habcheon Pt 
v13 Koryunggyo Qt+1 v56 Sungju Qt−1 v99 Jinju Pt 
v14 Hyunpung Qt+1 v57 Sungseo Qt−1 v100 Milyang Pt 
v15 Jucko Qt+1 v58 Hwawon Qt−1 v101 Sanchung Pt 
v16 Jukpogyo Qt+1 v59 Koryunggyo Qt−1 v102 Yungcheon Pt 
v17 Jungam Qt+1 v60 Hyunpung Qt−1 v103 Youngju Pt 
v18 Jindong Qt+1 v61 Jucko Qt−1 v104 Mungyeoung Pt 
v19 Susan Qt+1 v62 Jukpogyo Qt−1 v105 Andong Pt−1 
v20 Milyang Qt+1 v63 Jungam Qt−1 v106 Deagu Pt−1 
v21 Samlangjin Qt+1 v64 Jindong Qt−1 v107 Ueisung Pt−1 
v22 Yangsan Qt+1 v65 Susan Qt−1 v108 Guchang Pt−1 
v23 Gupo Qt+1 v66 Milyang Qt−1 v109 Gumi Pt−1 
v24 Andong Qt v67 Samlangjin Qt−1 v110 Habcheon Pt−1 
v25 Dalge Qt v68 Yangsan Qt−1 v111 Jinju Pt−1 
v26 Sabor Qt v69 Gupo Qt−1 v112 Milyang Pt−1 
v27 Yonggok Qt v70 Andong Qt−2 v113 Sanchung Pt−1 
v28 Nakdong Qt v71 Dalge Qt−2 v114 Yungcheon Pt−1 
v29 Ilsungyo Qt v72 Sabor Qt−2 v115 Youngju Pt−1 
v30 Sunsan Qt v73 Yonggok Qt−2 v116 Mungyeoung Pt−1 
v31 Gumi Qt v74 Nakdong Qt−2 v117 Andong Pt−2 
v32 Wegwan Qt v75 Ilsungyo Qt−2 v118 Deagu Pt−2 
v33 Sungju Qt v76 Sunsan Qt−2 v119 Ueisung Pt−2 
v34 Sungseo Qt v77 Gumi Qt−2 v120 Guchang Pt−2 
v35 Hwawon Qt v78 Wegwan Qt−2 v121 Gumi Pt−2 
v36 Koryunggyo Qt v79 Sungju Qt−2 v122 Habcheon Pt−2 
v37 Hyunpung Qt v80 Sungseo Qt−2 v123 Jinju Pt−2 
v38 Jucko Qt v81 Hwawon Qt−2 v124 Milyang Pt−2 
v39 Jukpogyo Qt v82 Koryunggyo Qt−2 v125 Sanchung Pt−2 
v40 Jungam Qt v83 Hyunpung Qt−2 v126 Yungcheon Pt−2 
v41 Jindong Qt v84 Jucko Qt−2 v127 Youngju Pt−2 
v42 Susan Qt v85 Jukpogyo Qt−2 v128 Mungyeoung Pt−2 
v43 Milyang Qt v86 Jungam Qt−2 – – – 
Figure 6

Study site: the Nakdong River.

Figure 6

Study site: the Nakdong River.

Results of EFA

One hundred and twenty-eight inputs with a 2,554 data set (the daily data from 2003 to 2009) were analyzed by EFA to define what factors affect the desired output. The number of common factors was determined by the number of eigenvalues of the covariance matrix that exceeded 1. This criterion was proposed by Kaiser (1958) and is widely used. The number of common factors was selected as 15, and the cumulated proportion of 15 eigenvalues accounts for about 87% of the total variance of the inputs. The principal component factoring method was used for the factoring method. From the results of Table 2, each common factor can be categorized into three groups and interpreted as follows. The first group is Factor 1 to Factor 4, which gives information for the time range of inputs. The second group is Factor 5 and Factor 6, which gives information for whether a stream gaging station is more affected by the upstream or downstream part of the basin, such that application of rainfall inputs P* for upstream and P of all rainfall stations for downstream can be set. The third group is Factor 7 to Factor 15, which gives information for variables affected by specific stations. From the third group of factors, Q or P inputs of specific stations can be added to the modeling for a desired output.

Table 2

Variables included in each factor and group

Group Factor Variables/Contents 
Group 1 Factor 1 v35 v41 v42 v44 v55 v56 v58 v59 v60 v61 v62 v63 v64 v65 v66 v67 v69 v71 v72 v74 v75 v76 v77 v78 v79 v80 v81 v82 v83 v84 v85 v86 v87 v88 v89 v90 v92 
→ almost all Qt−1 and all Qt−2 
Factor 2 v105 v106 v107 v108 v109 v110 v111 v112 v113 v114 v115 v116 v25 v26 v28 v29 v31 v32 v33 v36 v37 v38 v39 v40 v43 
→all Pt−1 and Qt−1 
Factor 3 v48 v49 v51 v52 v54 v117 v118 v119 v120 v121 v122 v123 v124 v125 v126 v127 v128 
→ all Pt−2 and Q*t−1 where Q* is denoted as upstream gaging station on main stream which are Dalge, Sabor, Nakdong, Ilsungyo, Gumi and Wegwan 
Factor 4 v93 v94 v95 v96 v97 v98 v99 v100 v101 v102 v103 v104 
all Pt 
Group 2 Factor 5 v25 v26 v28 v29 v31 v32 v93 v95 v97 v102 v103 v104 
Q*t and P*t where P* is denoted as rainfall station in upstream part of basin which are Andong, Ueisung, Gumi, Youngju, Youncheon, and Mungyeoung 
Factor 6 v30 v38 v39 v40 v42 v43 v44 v46 v53 v61 v63 
Qt of stream gaging station located at downstream part of Sungju 
Group 3 Factor 7 v25 v26 v28 v29 v31 v32 v48 v49 v51 v52 v54 
Q*t and Q*t−1 
Factor 8 v25 v26 v28 v29 v31 v32 
Q*t 
Factor 9 v68 v45 v91 
Qt , Qt−1, and Qt−2 of Yansan stream gaging station 
Factor 10 v24 v47 v70 
Qt , Qt−1, and Qt−2 of Andong stream gaging station 
Factor 11 v27 v50 v73 
Qt−1 and Qt−2 of Yonggok stream gaging station 
Factor 12 v30 v34 v57 v76 v80 
Qt, Qt−1, and Qt−2 of Susan and Sungseo stream gaging station 
Factor 13 v46 
Qt of Gupo stream gaging station 
Factor 14 v71 v72 v74 v75 v77 v78 v86 v88 v89 v90 v92 
Qt−2 of stream gaging station on the main stream 
Factor 15 v93 v95 v104 
Pt of Andong, Ueisung, and Mungyeoung rainfall station in most upstream part of the basin 
Group Factor Variables/Contents 
Group 1 Factor 1 v35 v41 v42 v44 v55 v56 v58 v59 v60 v61 v62 v63 v64 v65 v66 v67 v69 v71 v72 v74 v75 v76 v77 v78 v79 v80 v81 v82 v83 v84 v85 v86 v87 v88 v89 v90 v92 
→ almost all Qt−1 and all Qt−2 
Factor 2 v105 v106 v107 v108 v109 v110 v111 v112 v113 v114 v115 v116 v25 v26 v28 v29 v31 v32 v33 v36 v37 v38 v39 v40 v43 
→all Pt−1 and Qt−1 
Factor 3 v48 v49 v51 v52 v54 v117 v118 v119 v120 v121 v122 v123 v124 v125 v126 v127 v128 
→ all Pt−2 and Q*t−1 where Q* is denoted as upstream gaging station on main stream which are Dalge, Sabor, Nakdong, Ilsungyo, Gumi and Wegwan 
Factor 4 v93 v94 v95 v96 v97 v98 v99 v100 v101 v102 v103 v104 
all Pt 
Group 2 Factor 5 v25 v26 v28 v29 v31 v32 v93 v95 v97 v102 v103 v104 
Q*t and P*t where P* is denoted as rainfall station in upstream part of basin which are Andong, Ueisung, Gumi, Youngju, Youncheon, and Mungyeoung 
Factor 6 v30 v38 v39 v40 v42 v43 v44 v46 v53 v61 v63 
Qt of stream gaging station located at downstream part of Sungju 
Group 3 Factor 7 v25 v26 v28 v29 v31 v32 v48 v49 v51 v52 v54 
Q*t and Q*t−1 
Factor 8 v25 v26 v28 v29 v31 v32 
Q*t 
Factor 9 v68 v45 v91 
Qt , Qt−1, and Qt−2 of Yansan stream gaging station 
Factor 10 v24 v47 v70 
Qt , Qt−1, and Qt−2 of Andong stream gaging station 
Factor 11 v27 v50 v73 
Qt−1 and Qt−2 of Yonggok stream gaging station 
Factor 12 v30 v34 v57 v76 v80 
Qt, Qt−1, and Qt−2 of Susan and Sungseo stream gaging station 
Factor 13 v46 
Qt of Gupo stream gaging station 
Factor 14 v71 v72 v74 v75 v77 v78 v86 v88 v89 v90 v92 
Qt−2 of stream gaging station on the main stream 
Factor 15 v93 v95 v104 
Pt of Andong, Ueisung, and Mungyeoung rainfall station in most upstream part of the basin 

Determination of input variables

In this study, three stream gaging stations located in the upstream, middle, and downstream part of the Nakdong River were selected to verify the streamflow forecasting ANN modeling result for the input variables selected by EFA. These are Nakdong, Koryunggyo, and Susan stream gaging stations located in the major metropolitan area. Table 3 shows the EFA result for the desired output Qt+1 of Nakdong (v5), Koryunggyo (v13), and Susan (v19) stream gaging stations. The total communality result shows that 94.7% variance of the variable v5, 96.2% variance of the variable v13, and 94.8% variance of the variable v19 can be described by 15 common factors.

Table 3

Communality results and selected input variables for the desired output v5, v13, and v19

  vv13 
Order Factor Cumulative communality Cumulative percent of variance (%) Factor Cumulative communality Cumulative percent of variance (%) 
0.586 62 0.391 41 
0.767 81 0.549 57 
0.825 87 0.695 72 
0.880 93 0.781 81 
0.900 95 0.839 87 
10 0.915 97 0.887 92 
15 0.926 98 0.921 96 
0.934 99 0.937 97 
11 0.939 99 11 0.947 98 
10 0.942 99 12 0.952 99 
11 0.943 100 13 0.957 99 
12 0.944 100 14 0.958 100 
13 14 0.944 100 10 0.959 100 
14 12 0.944 100 0.959 100 
15 13 0.947 100 15 0.962 100 
  v19       Selected input variables 
Order Factor Cumulative communality Cumulative percent of variance (%) Desired output Streamflow Precipitation 
0.236 25 vQt , Qt−1, Qt−2 of Nakdong P*t, P*t−1 
0.397 42 
0.543 57 
0.685 72 
0.792 84 
0.852 90 v13 Qt , Qt−1, Qt−2 of Koryunggyo and Qt of Wegwan P*t, P*t−1 
0.888 94 
0.919 97 
12 0.925 98 
10 11 0.930 98 
11 10 0.934 98 v19 Qt, Qt−1, Qt−2 of Susan Pt, Pt−1, Pt−2 of all rainfall stations 
12 0.935 99 
13 13 0.936 99 
14 14 0.936 99 
15 15 0.948 100 
  vv13 
Order Factor Cumulative communality Cumulative percent of variance (%) Factor Cumulative communality Cumulative percent of variance (%) 
0.586 62 0.391 41 
0.767 81 0.549 57 
0.825 87 0.695 72 
0.880 93 0.781 81 
0.900 95 0.839 87 
10 0.915 97 0.887 92 
15 0.926 98 0.921 96 
0.934 99 0.937 97 
11 0.939 99 11 0.947 98 
10 0.942 99 12 0.952 99 
11 0.943 100 13 0.957 99 
12 0.944 100 14 0.958 100 
13 14 0.944 100 10 0.959 100 
14 12 0.944 100 0.959 100 
15 13 0.947 100 15 0.962 100 
  v19       Selected input variables 
Order Factor Cumulative communality Cumulative percent of variance (%) Desired output Streamflow Precipitation 
0.236 25 vQt , Qt−1, Qt−2 of Nakdong P*t, P*t−1 
0.397 42 
0.543 57 
0.685 72 
0.792 84 
0.852 90 v13 Qt , Qt−1, Qt−2 of Koryunggyo and Qt of Wegwan P*t, P*t−1 
0.888 94 
0.919 97 
12 0.925 98 
10 11 0.930 98 
11 10 0.934 98 v19 Qt, Qt−1, Qt−2 of Susan Pt, Pt−1, Pt−2 of all rainfall stations 
12 0.935 99 
13 13 0.936 99 
14 14 0.936 99 
15 15 0.948 100 

Considering the reduction of input variables and redundancy, the input variables for v5, v13, and v19 were determined by the factors of which accumulated communality exceeds 90% in Table 3. The variance of v5 can be explained more than 90% by Factor 5 in the second group and Factor 4, Factor 2 and Factor 1 in the first group. From the interpreted factor results, we can guess v5 was affected strongly by the upstream part of the basin, because Factor 5 is bigger than Factor 6. The time step of variables was affected by Factor 4, Factor 2, and Factor 1. The meaningful variables for v5 can be selected as Qt, Qt−1, Qt−2, P*t, and P*t−1. The variance of v13 can be explained more than 90% by Factor 5, Factor 6 in the second group, Factor 2, Factor 4, and Factor 1 in the first group, and Factor 8 in the third group. As Factor 5 is bigger than Factor 6, we can guess v13 was affected more by the upstream part of the basin than by the downstream part of the basin. The time step of variables was affected by Factor 4, Factor 2, and Factor 1. From Factor 8, a specific station upstream affected v13. The meaningful variables can be selected as Qt, Qt−1, Qt−2, P*t, P*t−1, and Q*t of a specific station, which is selected as the Wegwan stream gaging station, since it was the closest upstream stream gaging station to Koryunggyo station among the gaging stations in Q*. The variance of v19 can be explained more than 90% by Factor 6, Factor 5 in the second group, Factor 1, Factor 2, Factor 4, and Factor 3 in the first group. Since Factor 6 is bigger than Factor 5, we can guess that the v19 was located in the downstream part of the basin. The time step of variables is affected by all factors in the first group. Therefore, the meaningful variables for v19 would be Qt, Qt−1, Qt−2, Pt, Pt−1, and Pt−2.

RESULT OF STREAMFLOW FORECASTING

To verify that the input variables selected by the EFA give a good streamflow forecasting ANN modeling result, several cases consisting of various combinations of input variables selected without application of the EFA were compared to the case of proposed input variables by EFA. In the ANN modeling, SMFNNs with one hidden layer were used, and the number of hidden neurons was set by less than or several times the number of input variables (input neurons). The LM algorithm was used to train the network with 100 randomly generated weight parameter sets to reach the required training goal of 0.001 in 500 epochs. The tangent sigmoid activation function for the hidden layer and a linear activation function for the output layer were used. Among the 2,554 data sets, 1,824 data sets in 2003, 2004, 2007, 2008, and 2009 were used to train the network, and 365 data sets in 2005 were used to test what number of hidden neurons gives a good result; 365 data sets in 2006 were used to validate the neural network selected from the test.

Simple autoregressive model

Simple autoregressive (AR) models with antecedents of each variables in Equation (18), were also developed for training data set: 
formula
18
where is a desired output, , , and are the antecedents of the desired output, and is a constant or noise.

AR models for each desired output are shown in Table 4. In the result, AR(3) model using Qt, Qt−1, Qt−2 shows better results than AR(1) and AR(2) models. Developed AR models using training data were applied to test and validation data set to compare the result with ANN ensemble models.

Table 4

Results of AR models for each desired output variable

 AR model     RMSE (m³/s) NSE 
Nakdong (v5) AR(1) 50.529 0.675   262.599 0.456 
AR(2) 49.564 0.662 0.019  262.551 0.456 
AR(3) 40.334 0.659 −0.104 0.186 257.959 0.475 
Koryunggyo (v13) AR(1) 38.390 0.808   330.542 0.653 
AR(2) 49.809 1.048 −0.297  315.578 0.684 
AR(3) 34.714 1.139 −0.615 0.303 300.731 0.713 
Susan (v19) AR(1) 75.313 0.873   424.331 0.762 
AR(2) 112.048 1.300 −0.490  369.952 0.819 
AR(3) 74.050 1.467 −0.932 0.340 347.922 0.840 
 AR model     RMSE (m³/s) NSE 
Nakdong (v5) AR(1) 50.529 0.675   262.599 0.456 
AR(2) 49.564 0.662 0.019  262.551 0.456 
AR(3) 40.334 0.659 −0.104 0.186 257.959 0.475 
Koryunggyo (v13) AR(1) 38.390 0.808   330.542 0.653 
AR(2) 49.809 1.048 −0.297  315.578 0.684 
AR(3) 34.714 1.139 −0.615 0.303 300.731 0.713 
Susan (v19) AR(1) 75.313 0.873   424.331 0.762 
AR(2) 112.048 1.300 −0.490  369.952 0.819 
AR(3) 74.050 1.467 −0.932 0.340 347.922 0.840 

Qt+1 of Nakdong stream gaging station (v5)

Several cases of various combinations of input variables affected by Factor 5, Factor 4, Factor 2, and Factor 1 that affected v5 are compared. Case 1 is the neural network model with input variables affected by Factor 5 and Factor 4. Case 2 is the model with input variables affected by Factor 5, Factor 4, and Factor 2. Case 3 is the proposed model by EFA in which the input variables are affected by Factor 5, Factor 4, Factor 2, and Factor 1. Case 4 is the model in which the input variable affected by Factor 3 is added to the input variables of Case 3. Case 5 and Case 6 are the models with/without Qt−1, Qt−2, and P*t−2 of Case 3. The test and validation results of various numbers of hidden neurons for each case and AR models are shown in Table 5.

Table 5

Test and validation results of each case of neural network model for v5

    Test result Validation result 
Case Input variables (total number) No. of hidden neurons RMSE (m³/s) Mean IQR (m³/s) NSE RMSE (m³/s) Mean IQR (m³/s) NSE 
Case 1 Qt, P*t (7) 580.073 57.875 −0.382 314.319 49.74 0.313 
14 405.832 230.981 0.371 180.421 121.616 0.418 
28 408.766 302.233 0.262 229.62 181.392 0.449 
42 433.791 265.193 0.063 231.316 179.943 0.434 
56 377.79 249.325 0.176 234.498 160.417 0.31 
Case 2 Qt, Qt−1, P*t, P*t−1(14) 10 124.419 67.72 0.7 161.089 107.101 0.83 
14 104.36 68.354 0.723 170.899 109.628 0.765 
28 101.747 73.646 0.723 210.73 102.786 0.58 
56 120.038 82.127 0.654 215.899 117.205 0.515 
84 133.231 84.088 0.619 234.725 121.206 0.403 
Case 3 Qt, Qt−1, Qt−2, P*t, P*t−1 (15)–proposed by EFA 10 115.289 77.576 0.746 114.086 106.501 0.919 
15 123.725 75.051 0.709 125.868 94.937 0.88 
30 135.505 79.089 0.65 151.68 98.647 0.798 
60 139.763 87.179 0.606 148.191 104.828 0.812 
90 151.111 91.759 0.573 170.18 116.325 0.739 
Case 4 Qt, Qt−1, Qt−2, Pt, Pt−1, Pt−2 (21) 10 151.021 66.678 0.679 135.517 98.475 0.903 
21 99.638 70.119 0.767 160.643 92.764 0.759 
42 111.351 79.323 0.722 156.221 104.127 0.772 
63 116.986 79.016 0.671 170.85 107.139 0.698 
84 136.954 100.923 0.586 177.251 129.957 0.687 
Case 5 Qt, P*t, P*t−1 (13) 10 158.702 72.183 0.656 220.083 111.426 0.652 
13 141.84 80.653 0.66 203.089 117.301 0.583 
26 135.525 81.211 0.639 244.863 110.573 0.381 
52 147.771 83.738 0.613 254.627 109.625 0.401 
84 165.534 88 0.551 307.041 118.065 0.077 
Case 6 Qt, P*t, P*t−1, P*t−2 (19) 10 100.637 62.297 0.757 193.066 100.853 0.735 
19 107.003 64.037 0.716 220.12 102.074 0.606 
38 128.131 74.985 0.639 236.762 108.273 0.47 
76 139.223 79.794 0.599 280.975 124.741 0.082 
114 148.022 87.821 0.575 300.185 134.579 0.071 
AR model Qt AR(1) 110.865 – 0.548 265.014 – 0.606 
Qt Qt−1, AR(2) 111.126 – 0.545 265.317 – 0.605 
Qt Qt−1 Qt−2 AR(3) 112.189 – 0.537 271.961 – 0.585 
    Test result Validation result 
Case Input variables (total number) No. of hidden neurons RMSE (m³/s) Mean IQR (m³/s) NSE RMSE (m³/s) Mean IQR (m³/s) NSE 
Case 1 Qt, P*t (7) 580.073 57.875 −0.382 314.319 49.74 0.313 
14 405.832 230.981 0.371 180.421 121.616 0.418 
28 408.766 302.233 0.262 229.62 181.392 0.449 
42 433.791 265.193 0.063 231.316 179.943 0.434 
56 377.79 249.325 0.176 234.498 160.417 0.31 
Case 2 Qt, Qt−1, P*t, P*t−1(14) 10 124.419 67.72 0.7 161.089 107.101 0.83 
14 104.36 68.354 0.723 170.899 109.628 0.765 
28 101.747 73.646 0.723 210.73 102.786 0.58 
56 120.038 82.127 0.654 215.899 117.205 0.515 
84 133.231 84.088 0.619 234.725 121.206 0.403 
Case 3 Qt, Qt−1, Qt−2, P*t, P*t−1 (15)–proposed by EFA 10 115.289 77.576 0.746 114.086 106.501 0.919 
15 123.725 75.051 0.709 125.868 94.937 0.88 
30 135.505 79.089 0.65 151.68 98.647 0.798 
60 139.763 87.179 0.606 148.191 104.828 0.812 
90 151.111 91.759 0.573 170.18 116.325 0.739 
Case 4 Qt, Qt−1, Qt−2, Pt, Pt−1, Pt−2 (21) 10 151.021 66.678 0.679 135.517 98.475 0.903 
21 99.638 70.119 0.767 160.643 92.764 0.759 
42 111.351 79.323 0.722 156.221 104.127 0.772 
63 116.986 79.016 0.671 170.85 107.139 0.698 
84 136.954 100.923 0.586 177.251 129.957 0.687 
Case 5 Qt, P*t, P*t−1 (13) 10 158.702 72.183 0.656 220.083 111.426 0.652 
13 141.84 80.653 0.66 203.089 117.301 0.583 
26 135.525 81.211 0.639 244.863 110.573 0.381 
52 147.771 83.738 0.613 254.627 109.625 0.401 
84 165.534 88 0.551 307.041 118.065 0.077 
Case 6 Qt, P*t, P*t−1, P*t−2 (19) 10 100.637 62.297 0.757 193.066 100.853 0.735 
19 107.003 64.037 0.716 220.12 102.074 0.606 
38 128.131 74.985 0.639 236.762 108.273 0.47 
76 139.223 79.794 0.599 280.975 124.741 0.082 
114 148.022 87.821 0.575 300.185 134.579 0.071 
AR model Qt AR(1) 110.865 – 0.548 265.014 – 0.606 
Qt Qt−1, AR(2) 111.126 – 0.545 265.317 – 0.605 
Qt Qt−1 Qt−2 AR(3) 112.189 – 0.537 271.961 – 0.585 

Each neural network was trained into RMSE, 65.7 m3/s, IQR 29.3 m3/s, and over 0.96 NSE on average. According to the test result, the optimum number of hidden neurons for each model was selected as 14 hidden neurons for Case 1, 28 hidden neurons for Case 2, 10 hidden neurons for Case 3, 21 hidden neurons for Case 4, 26 hidden neurons for Case 5, and 10 hidden neurons for Case 6. Validation results of Case 1, Case 3, and Case 6 also showed good results in the model with the optimum number of hidden neurons of each case in the test. However, Case 2, Case 4, and Case 5 showed good validation results in the model with different numbers of hidden neurons with an optimum number of hidden neurons selected in the test. This means that these models may give a different result according to the input data presented, so these models can be regarded as unstable. As the variables affected by factors with high communality are added to the input variables, RMSE in the test and the validation result decrease. Case 4 is the model in which Pt−2 is added to the input variables of Case 3. As shown in Table 3, the 2% of communality is added by Pt−2 which is the variable affected by Factor 3 and leads to the lowest RMSE in the test result and generally lower RMSE than other cases in the validation result. However, Case 4 shows higher RMSE than Case 3, which is thought to be caused by the data redundancy 2% of necessary information was given by Factor 3 but, more unnecessary information was included. As shown in Figure 7, Case 3 shows relatively higher RMSE 115.3 m3/s, IQR 77.6 m3/s than Case 2, Case 4, and Case 6 in the test result, but gives the lowest RMSE 114.1 m3/s and the highest NSE 0.92 in the validation result. And Case 3 shows better results both in the test and validation than AR models. Therefore, the Case 3 neural network with proposed input variables from EFA can be selected as a reliable neural network model for v5. Figure 8 shows the validation result of Case 3.

Figure 7

Comparison of the test and validation results of each case for v5.

Figure 7

Comparison of the test and validation results of each case for v5.

Figure 8

Validation result of Case 3 for v5.

Figure 8

Validation result of Case 3 for v5.

Qt+1 of Koryunggyo stream gaging station (v13)

Several cases of various combinations of input variables affected by Factor 5, Factor 2, Factor 4, Factor 8, Factor 1, and Factor 6 that affect v13 are compared. Case 1 is the model with input variables affected by Factor 5, Factor 2, Factor 4, and Factor 6. Case 2 is the model with input variables affected by Factor 5, Factor 2, Factor 4, Factor 1, and Factor 6. Case 3 is the proposed model by EFA in which the input variables are affected by Factor 5, Factor 4, Factor 2, Factor 1, Factor 8, and Factor 6. Case 4 is the model in which the input variable affected by Factor 3 is added to the input variables of Case 3. Case 5 and Case 6 are the models with/without Qt−1, Qt−2, and P*t−1 of Case 3. The test and validation results of various numbers of hidden neurons for each case and AR models are shown in Table 6.

Table 6

Test and validation results of each case of neural network model for v13

   Test result Validation result 
Case Input variables (total number) No. of hidden neurons RMSE (m³/s) Mean IQR (m³/s) NSE RMSE (m³/s) Mean IQR (m³/s) NSE 
Case 1 Qt, Qt−1, P*t, P*t−1 (14) 10 86.266 84.931 0.887 173.827 88.752 0.922 
14 85.9 87.069 0.888 184.068 91.067 0.908 
28 94.437 90.879 0.872 243.282 95.911 0.824 
42 104.188 98.529 0.843 257.091 104.418 0.813 
56 115.859 99.38 0.806 283.565 105.134 0.774 
Case 2 Qt, Qt−1, Qt−2, P*t, P*t−1 (15) 10 75.612 82.207 0.903 171.076 86.443 0.926 
15 75.388 85.399 0.915 173.379 89.275 0.921 
30 86.639 91.812 0.875 206.102 96.004 0.888 
45 91.233 95.295 0.876 249.022 101.697 0.825 
60 127.355 96.714 0.763 286.134 103.634 0.784 
Case 3 Qt, Qt−1, Qt−2, P*t, P*t−1, Q*t of Wegwan (16)–proposed by EFA 10 74.322 91.766 0.906 143.265 97.151 0.954 
16 78.825 96.216 0.896 155.164 100.237 0.942 
32 83.792 102.054 0.878 189.992 106.786 0.906 
48 95.104 107.584 0.831 208.418 114.03 0.891 
64 112.713 109.327 0.751 218.799 117.886 0.892 
Case 4 Qt, Qt−1, Qt−2, P*t, P*t−1, P*t−2 (21) 10 76.857 105.779 0.89 148.343 110.595 0.95 
21 79.752 110.184 0.874 174.719 115.1 0.916 
42 152.603 121.237 0.31 217.441 126.304 0.87 
63 192.606 123.994 0.009 287.817 130.445 0.763 
84 254.958 128.676 −0.601 301.634 137.371 0.764 
Case 5 Qt, Qt−1, Qt−2, P*t (9) 110.191 49.398 0.839 173.612 54.123 0.931 
82.662 50.326 0.901 205.587 54.272 0.877 
18 86.693 53.924 0.893 246.804 58.668 0.802 
27 95 56.156 0.885 264.115 60.556 0.76 
36 95.931 56.588 0.882 285.251 62.57 0.693 
Case 6 Qt, P*t, P*t−1 (13) 10 84.618 73.134 0.883 177.602 77.065 0.922 
13 85.771 75.459 0.884 199.018 79.267 0.891 
26 88.616 81.332 0.874 228.2 85.5 0.855 
39 100.275 84.745 0.835 264.835 89.335 0.785 
52 120.227 86.882 0.79 302.987 91.923 0.713 
AR model Qt AR(1) 166.650 – 0.489 379.352 – 0.725 
Qt Qt−1 AR(2) 161.820 – 0.518 333.582 – 0.787 
Qt Qt−1 Qt−2 AR(3) 158.391 – 0.539 325.480 – 0.798 
   Test result Validation result 
Case Input variables (total number) No. of hidden neurons RMSE (m³/s) Mean IQR (m³/s) NSE RMSE (m³/s) Mean IQR (m³/s) NSE 
Case 1 Qt, Qt−1, P*t, P*t−1 (14) 10 86.266 84.931 0.887 173.827 88.752 0.922 
14 85.9 87.069 0.888 184.068 91.067 0.908 
28 94.437 90.879 0.872 243.282 95.911 0.824 
42 104.188 98.529 0.843 257.091 104.418 0.813 
56 115.859 99.38 0.806 283.565 105.134 0.774 
Case 2 Qt, Qt−1, Qt−2, P*t, P*t−1 (15) 10 75.612 82.207 0.903 171.076 86.443 0.926 
15 75.388 85.399 0.915 173.379 89.275 0.921 
30 86.639 91.812 0.875 206.102 96.004 0.888 
45 91.233 95.295 0.876 249.022 101.697 0.825 
60 127.355 96.714 0.763 286.134 103.634 0.784 
Case 3 Qt, Qt−1, Qt−2, P*t, P*t−1, Q*t of Wegwan (16)–proposed by EFA 10 74.322 91.766 0.906 143.265 97.151 0.954 
16 78.825 96.216 0.896 155.164 100.237 0.942 
32 83.792 102.054 0.878 189.992 106.786 0.906 
48 95.104 107.584 0.831 208.418 114.03 0.891 
64 112.713 109.327 0.751 218.799 117.886 0.892 
Case 4 Qt, Qt−1, Qt−2, P*t, P*t−1, P*t−2 (21) 10 76.857 105.779 0.89 148.343 110.595 0.95 
21 79.752 110.184 0.874 174.719 115.1 0.916 
42 152.603 121.237 0.31 217.441 126.304 0.87 
63 192.606 123.994 0.009 287.817 130.445 0.763 
84 254.958 128.676 −0.601 301.634 137.371 0.764 
Case 5 Qt, Qt−1, Qt−2, P*t (9) 110.191 49.398 0.839 173.612 54.123 0.931 
82.662 50.326 0.901 205.587 54.272 0.877 
18 86.693 53.924 0.893 246.804 58.668 0.802 
27 95 56.156 0.885 264.115 60.556 0.76 
36 95.931 56.588 0.882 285.251 62.57 0.693 
Case 6 Qt, P*t, P*t−1 (13) 10 84.618 73.134 0.883 177.602 77.065 0.922 
13 85.771 75.459 0.884 199.018 79.267 0.891 
26 88.616 81.332 0.874 228.2 85.5 0.855 
39 100.275 84.745 0.835 264.835 89.335 0.785 
52 120.227 86.882 0.79 302.987 91.923 0.713 
AR model Qt AR(1) 166.650 – 0.489 379.352 – 0.725 
Qt Qt−1 AR(2) 161.820 – 0.518 333.582 – 0.787 
Qt Qt−1 Qt−2 AR(3) 158.391 – 0.539 325.480 – 0.798 

Each neural network was trained into RMSE 80.8 m3/s, IQR 73.0 m3/s, and over 0.98 NSE on average. According to the test result, the optimum number of hidden neurons for each case was selected as 14 hidden neurons for Case 1, 15 hidden neurons for Case 2, 10 hidden neurons for Case 3, 10 hidden neurons for Case 4, nine hidden neurons for Case 5, and 10 hidden neurons for Case 6. Case 1, Case 2, and Case 5 showed good validation results in different numbers of hidden neurons with the optimum number of hidden neurons selected in the test. So these models can be regarded as unstable. As the variables affected by factors with high communality in Table 3 are added to input variables, RMSE in the test and the validation result decrease. Case 3 is the model in which Qt of Wegwan is added to the input variables of Case 2. In Table 3, the 9% of communality is added by Qt of Wegwan affected by Factor 8. This leads to lower RMSE than Case 2 in the test and validation results. Case 4 is the model in which Pt−2 is added to input variables of Case 2. In Table 3, the 4% of communality is added by Factor 3. As shown in Figure 9, this leads to lower RMSE than Case 2 in the test and validation results. Case 3 and Case 4 show similar and lowest RMSE results. However, Case 3 is a more efficient model than Case 4, because Case 3 has five fewer input variables than Case 4. Therefore, the Case 3 neural network with input variables proposed by EFA can be selected as an appropriate neural network model for v13. And Case 3 shows better results both in the test and validation than AR models. Figure 10 shows the validation result of Case 3.

Figure 9

Comparison of the test and validation results of each case for v13.

Figure 9

Comparison of the test and validation results of each case for v13.

Figure 10

Validation result of Case 3 for v13.

Figure 10

Validation result of Case 3 for v13.

Qt+1 of Susan stream gaging station (v19)

Several cases of various combinations of input variables affected by Factor 6, Factor 1, Factor 5, Factor 2, Factor 4, and Factor 3 that affect v19 are compared. Case 1 is the model with input variables affected by Factor 6, Factor 1, and Factor 5. Case 2 is the model with input variables affected by Factor 6, Factor 1, Factor 5, Factor 2, and Factor 4. Case 3 is the model proposed by EFA in which the input variables are affected by Factor 6, Factor 1, Factor 5, Factor 2, Factor 4, and Factor 3. Case 4 is the model in which the input variable Qt of Wegwan affected by Factor 8 is added to the input variables of Case 3. Case 5, Case 6, and Case 7 are the models without Qt−1, Qt−2, P*t−1, and P*t−2 of Case 3. The test and validation results of various numbers of hidden neurons for each case and AR models are shown in Table 7.

Table 7

Test and validation results of each case of neural network model for v19

   Test result Validation result 
Case Input variables (total number) No. of hidden neurons RMSE (m³/s) Mean IQR (m³/s) NSE RMSE (m³/s) Mean IQR (m³/s) NSE 
Case 1 Qt, Qt−1, Qt−2, Pt, (15) 10 158.093 444.502 0.792 443.504 495.434 0.802 
15 150.531 532.83 0.829 374.746 581.562 0.885 
30 177.009 473.777 0.787 375.027 530.893 0.9 
45 172.843 615.287 0.796 402.859 676.038 0.883 
60 209.357 593.384 0.727 428.852 656.438 0.879 
Case 2 Qt, Qt−1, Qt−2, Pt, Pt−1 (27) 14 122.833 443.996 0.896 291.15 493.281 0.932 
27 140.568 553.205 0.88 236.712 601.274 0.961 
54 185.514 562.801 0.822 218.984 633.523 0.971 
81 218.025 607.841 0.769 263.889 681.463 0.96 
108 245.991 579.726 0.724 266.197 659.502 0.96 
Case 3 Qt, Qt−1, Qt−2, Pt, Pt−1, Pt−2 (39)–proposed by EFA 10 116.878 119.019 0.91 218.673 162.337 0.968 
20 130.519 198.785 0.893 235.204 241.644 0.962 
39 219.729 230.73 0.728 239.847 277.096 0.962 
78 324.125 250.983 0.456 286.408 321.01 0.952 
117 404.314 268.625 0.342 333.092 354.902 0.932 
Case 4 Qt, Qt−1, Qt−2, Pt, Pt−1, Pt−2 Qt of Wegwan (40) 10 102.516 152.727 0.931 235.985 159.56 0.959 
20 110.975 161.813 0.922 255.385 171.02 0.949 
40 203.293 177.936 0.772 263.493 188.851 0.948 
60 275.614 172.35 0.604 272.121 186.625 0.95 
80 313.001 181.035 0.528 296.98 194.032 0.942 
Case 5 Qt, Pt (13) 10 192.478 424.829 0.641 472.428 473.747 0.799 
13 173.956 428.603 0.712 465.114 482.466 0.821 
26 200.675 465.174 0.691 479.43 516.432 0.849 
39 204.754 563.38 0.693 532.744 618.835 0.836 
52 228.921 537.468 0.678 592.473 598.424 0.821 
Case 6 Qt, Pt, Pt−1 (25) 10 137.966 399.838 0.863 379.405 456.787 0.867 
25 339.426 484.384 0.194 377.843 528.543 0.895 
50 218.882 538.94 0.763 264.1 584.892 0.961 
75 301.079 347.814 0.677 308.545 692.805 0.947 
100 309.412 327.362 0.668 358.758 698.094 0.937 
Case 7 Qt, Pt, Pt−1, Pt−2 (37) 10 168.335 355.765 0.819 455.114 414.135 0.767 
20 157.151 442.567 0.845 320.809 488.406 0.914 
37 177.288 541.639 0.824 224.342 590.567 0.968 
74 274.841 577.531 0.621 276.861 653.069 0.954 
111 347.485 654.709 0.51 283.886 712.045 0.957 
AR model Qt AR(1) 205.296 – 0.695 613.122 – 0.792 
Qt Qt−1 AR(2) 174.262 – 0.780 492.277 – 0.866 
Qt, Qt−1 Qt−2 AR(3) 155.570 – 0.825 449.022 – 0.888 
   Test result Validation result 
Case Input variables (total number) No. of hidden neurons RMSE (m³/s) Mean IQR (m³/s) NSE RMSE (m³/s) Mean IQR (m³/s) NSE 
Case 1 Qt, Qt−1, Qt−2, Pt, (15) 10 158.093 444.502 0.792 443.504 495.434 0.802 
15 150.531 532.83 0.829 374.746 581.562 0.885 
30 177.009 473.777 0.787 375.027 530.893 0.9 
45 172.843 615.287 0.796 402.859 676.038 0.883 
60 209.357 593.384 0.727 428.852 656.438 0.879 
Case 2 Qt, Qt−1, Qt−2, Pt, Pt−1 (27) 14 122.833 443.996 0.896 291.15 493.281 0.932 
27 140.568 553.205 0.88 236.712 601.274 0.961 
54 185.514 562.801 0.822 218.984 633.523 0.971 
81 218.025 607.841 0.769 263.889 681.463 0.96 
108 245.991 579.726 0.724 266.197 659.502 0.96 
Case 3 Qt, Qt−1, Qt−2, Pt, Pt−1, Pt−2 (39)–proposed by EFA 10 116.878 119.019 0.91 218.673 162.337 0.968 
20 130.519 198.785 0.893 235.204 241.644 0.962 
39 219.729 230.73 0.728 239.847 277.096 0.962 
78 324.125 250.983 0.456 286.408 321.01 0.952 
117 404.314 268.625 0.342 333.092 354.902 0.932 
Case 4 Qt, Qt−1, Qt−2, Pt, Pt−1, Pt−2 Qt of Wegwan (40) 10 102.516 152.727 0.931 235.985 159.56 0.959 
20 110.975 161.813 0.922 255.385 171.02 0.949 
40 203.293 177.936 0.772 263.493 188.851 0.948 
60 275.614 172.35 0.604 272.121 186.625 0.95 
80 313.001 181.035 0.528 296.98 194.032 0.942 
Case 5 Qt, Pt (13) 10 192.478 424.829 0.641 472.428 473.747 0.799 
13 173.956 428.603 0.712 465.114 482.466 0.821 
26 200.675 465.174 0.691 479.43 516.432 0.849 
39 204.754 563.38 0.693 532.744 618.835 0.836 
52 228.921 537.468 0.678 592.473 598.424 0.821 
Case 6 Qt, Pt, Pt−1 (25) 10 137.966 399.838 0.863 379.405 456.787 0.867 
25 339.426 484.384 0.194 377.843 528.543 0.895 
50 218.882 538.94 0.763 264.1 584.892 0.961 
75 301.079 347.814 0.677 308.545 692.805 0.947 
100 309.412 327.362 0.668 358.758 698.094 0.937 
Case 7 Qt, Pt, Pt−1, Pt−2 (37) 10 168.335 355.765 0.819 455.114 414.135 0.767 
20 157.151 442.567 0.845 320.809 488.406 0.914 
37 177.288 541.639 0.824 224.342 590.567 0.968 
74 274.841 577.531 0.621 276.861 653.069 0.954 
111 347.485 654.709 0.51 283.886 712.045 0.957 
AR model Qt AR(1) 205.296 – 0.695 613.122 – 0.792 
Qt Qt−1 AR(2) 174.262 – 0.780 492.277 – 0.866 
Qt, Qt−1 Qt−2 AR(3) 155.570 – 0.825 449.022 – 0.888 

Each neural network was trained into RMSE 145.8 m3/s, IQR 402.3 m3/s and over 0.95 NSE on average. According to the test result, the optimum number of hidden neurons for each case was selected as 15 hidden neurons for Case 1, 27 hidden neurons for Case 2, 10 hidden neurons for Case 3, 10 hidden neurons for Case 4, 13 hidden neurons for Case 5, 10 hidden neurons for Case 6, and 20 hidden neurons for Case 7. Validation results of Case 2, Case 6, and Case 7 showed good results in the model with the different numbers of hidden neurons of the test. As the variables affected by factors with high communality in Table 3 are added to the input variables, RMSE of Case 1–Case 4 in the test result decreases. Case 4 is the case in which Qt of Wegwan affected by Factor 8 is added to input variables of Case 3. In Table 3, the 4% of communality is added by the variable affected by Factor 8. As shown in Figure 11, this leads to lower RMSE 102.5 m3/s than RMSE 116.9 m3/s of Case 3 in the test result. However, Case 4 shows higher RMSE 236.0 m3/s than RMSE 218.7 m3/s of Case 3 in validation results. Therefore, the Case 3 neural network with proposed input variables by the EFA can be selected as an appropriate neural network model for v19. And Case 3 also shows better results both in the test and validation than AR models. Figure 12 shows the validation result of Case 3.

Figure 11

Comparison of the test and validation results of each case for v19.

Figure 11

Comparison of the test and validation results of each case for v19.

Figure 12

Validation result of Case 3 for v19.

Figure 12

Validation result of Case 3 for v19.

DISCUSSION

The ANN model could give very different results according to the initial weight parameters. This means that the optimal model can be changed according to the initial weight parameters. Based on the ensemble modeling, the global performance of the ANN model can be estimated, and the generalization ability of ANNs was improved by combining individual ANNs (ensemble members). Figure 13 and Table 8 show the comparison of validation results between many individual ANN models and ensemble ANN models with a 100 ensemble members. ANN ensemble models show a better NSE result than averaged individual ANN models. An individual ANN model shows NSE result of Min. −29.22 and Median 0.46 for v5, Min. 0.74 and Median 0.91 for v13 and Min. 0.63 and Median 0.93 for v19. All ensemble models show better results (0.92 for v5, 0.95 for v13, 0.97 for v19) than the averaged NSE of individual ANN models.

Table 8

NSE of individual ANN models and ANN ensemble models for validation result

    Individual model 
  Ensemble model NSE 
Desired output NSE Min. Averaged Median Max. 
v5 0.92 −29.22 −0.06 0.46 0.84 
v13 0.95 0.74 0.90 0.91 0.97 
v19 0.97 0.63 0.91 0.93 0.98 
    Individual model 
  Ensemble model NSE 
Desired output NSE Min. Averaged Median Max. 
v5 0.92 −29.22 −0.06 0.46 0.84 
v13 0.95 0.74 0.90 0.91 0.97 
v19 0.97 0.63 0.91 0.93 0.98 
Figure 13

Comparison of the validation results between individual and ANN ensemble models.

Figure 13

Comparison of the validation results between individual and ANN ensemble models.

IQR is the indicator of the variance or uncertainty of ANN ensemble model result according to the initial weight parameters. The result of ensemble modeling showed that the IQR increased as the number of hidden neurons was increasing, and all cases gave good RMSE and NSE results, when the ratio between the number of hidden neurons and input neurons (input variable) was less than or equal to 1. As the number of hidden neurons is increasing, the number of local optima of weight parameters in the network is increasing. This means that the variance of ANN model results for the initial weight parameters can be increased more by the increase of the number of hidden neurons, which led to the increase of the uncertainty of ANN model results. Figure 14 shows the NSE results of each optimal ANN ensemble model for the ratio between the number of hidden neurons and input neurons.

Figure 14

NSE results of each optimal ANN ensemble model for the ratio.

Figure 14

NSE results of each optimal ANN ensemble model for the ratio.

The optimal ANN ensemble models showed big errors in the forecast for the high discharge flow period shown in Figure 15. The RMSE in the high discharge flow period is 334.22 m3/s for v5, 467.35 m3/s for v13, and 693.48 m3/s for v19, which is about three times bigger than each total RMSE 114.09 m3/s for v3, 143.27 m3/s for v13, and 218.67 m3/s for v19. The error in the high discharge flow period is thought to come from the unbalance of the training data set itself. The neural network is trained by the patterns of the given historical data set to minimize the sum of squared errors. Naturally, the network is optimized by the data in the range of low discharge in which a large amount of data is distributed, since the amount of data in the range of high discharge is much less than the amount of data in low discharge.

Figure 15

Validation result for v5, v13, and v19 in high discharge period.

Figure 15

Validation result for v5, v13, and v19 in high discharge period.

CONCLUSIONS

In this study, the ANN ensemble models using EFA for input variables selection and simple AR models were developed to forecast streamflow of three stream gaging stations in Nakdong River, Korea. EFA was performed on 128 variable candidates (105 input variable candidates and 23 desired output) of the 23 streamflow stations and 12 rainfall stations in the Nakdong River basin. In the EFA result, the communality result by 15 factors had an average 87% for the total variance of the 128 variables. The communality result for the three stream gaging stations of Nakdong (v5), Koryunggyo (v13), and Susan (v19) showed 94.7%, 96.2%, and 94.8%, respectively. From the interpretation of the common factors, input variables of the ANN model to forecast streamflow of the three stations were selected as the variables affected by factors of which cumulative communality was more than 90% of each total communality of v5, v13, and v19. Qt , Qt−1, Qt−2 of Nakdong and P*t, P*t−1 for v5, Qt, Qt−1, Qt−2 of Koryunggyo, Qt of Wegwan and P*t, P*t−1 for v13, Qt , Qt−1, Qt−2 of Susan and Pt, Pt−1, Pt−2 of all rainfall stations for v19 were selected.

In the streamflow forecasting results for v5, v13, and v19, RMSE, IQR, and NSE results of the ANN ensemble models with the input variables from EFA for v5, v13, and v19 in the validation were (114.09 m3/s, 106.50 m3/s, 0.92), (143.27 m3/s, 97.15 m3/s, 0.95) and (218.67 m3/s, 162.33 m3/s, 0.97), respectively. And the ANN ensemble models show better result than the simple AR models. When the variables affected by factors of which the cumulative communality was over 90% were added to the input variables, the test result was better than the model with the proposed input variables but the validation result was not better or similar. Considering the robustness and efficiency of the model, the model with the variables affected by factors for which the cumulative communality was more than 90% of the total communality could be selected as an appropriate neural network model.

This study presents an application of ANN with the ensemble modeling technique and the input variables selection technique by EFA. Using the ensembles modeling technique, the global performance of the ANN model considering the variance of ANN results for various initial weight parameters was estimated, and the optimal ANN forecasting model for streamflow of each station can be selected. Through the input variable selection technique by EFA, the ANN model can be more accurate, efficient, cost-effective, and interpretable. EFA enables the selection of the input variables required to describe the variance of the output variable, considering the degree of redundancy and variables that have little or no predictive information. Also, this enables the development of an efficient, cost-effective ANN model, since the process of the input variable selection by EFA is model-free and multiobjective.

However, a relatively high error occurred in the high discharge flow period due to the unbalanced distribution of the training data set. Generally, the distribution of flow data is unbalanced. Thus, future applications of the proposed ANN ensemble model with EFA for streamflow forecasting will include the data preprocessing technique, such as the clustering method, to construct the balanced training data set.

ACKNOWLEDGEMENTS

This research was supported by a grant (11-TI-C06) from the Advanced Water Management Research Program funded by the Ministry of Land, Infrastructure and Transport of the Korean government, and conducted at the Engineering Research Institute and the Integrated Research Institute of Construction and Environment in Seoul National University, Seoul, Korea.

REFERENCES

REFERENCES
Araghinejad
S.
Azmi
M.
Kholghi
M.
2011
Application of artificial neural network ensembles in probabilistic hydrological forecasting
.
J. Hydrol.
407
(
1–4
),
94
104
.
Boucher
M. A.
Perreault
L.
Anctil
F.
2009
Tools for the assessment of hydrological ensemble forecasts obtained by neural networks
.
J. Hydroinform.
11
(
3–4
),
297
307
.
Bowden
G. J.
2003
Forecasting water resources variables using artificial neural techniques.
PhD thesis
,
University of Adelaide
,
Australia
.
Boyacioglu
H.
2006
Surface water quality assessment using factor analysis
.
Water SA
32
(
3
),
389
393
.
Campolo
M.
Andreussi
P.
Soldati
A.
1999
River flood forecasting with a neural network model
.
Water Resour. Res.
35
(
4
),
1191
1197
.
Caudill
M.
Butler
C.
1992
Understanding Neural Networks: Computer Explorations: A workbook in two volumes with software for the Macintosh and PC compatibles
.
MIT Press
,
Cambridge, MA
,
USA
.
Chang
Y.-T.
Lin
J.
Shieh
J.-S.
Abbod
M. F.
2012
Optimization the initial weights of artificial neural networks via genetic algorithm applied to hip bone fracture prediction
.
Adv. Fuzzy Syst.
2012
,
9
.
Fabrigar
L. R.
Wegener
D. T.
MacCallum
R. C.
Strahan
E. J.
1999
Evaluating the use of exploratory factor analysis in psychological research
.
Psychol. Methods
4
,
272
299
.
Finch
J. F.
West
S. G.
1997
The investigation of personality structure: statistical models
.
J. Res. Personality
31
(
4
),
439
485
.
Hagan
M. T.
Demuth
H. B.
Beale
M. H.
1996
Neural Network Design
,
1st edn
.
PWS Publishing Company
,
Boston, MA
,
USA
.
Hansen
K. L.
Salamon
P.
1990
Neural network ensembles
.
IEEE Trans. Pattern Anal. Machine Intell.
12
(
10
),
993
1001
.
Haykin
S.
1999
Neural Networks: A Comprehensive Foundation
,
2nd edn
.
Prentice Hall
,
Upper Saddle River, NJ
,
USA
,
842
pp.
Kelton
M. L.
LeardMann
C. A.
Smith
B.
Boyko
E. J.
Hooper
T. I.
Gackstetter
G. D.
Bliese
P. D.
Hoge
C. W.
Smith
T. C.
2010
Exploratory factor analysis of self-reported symptons in a large, population-based military cohort
.
BMC Med. Res. Method.
10
,
94
.
Kolen
J. F.
Pollack
J. B.
1990
Back Propagation is Sensitive to Initial Conditions
.
Morgan Kaufmann Publishers
,
San Francisco, CA
,
USA
, pp.
860
867
.
Krogh
A.
Vedelsby
J.
1995
Neural network ensembles, cross validation and active learning
.
Adv. Neural Inform. Process. Syst.
7
,
231
238
.
Laucelli
D.
Babovic
V.
Keijzer
M.
Giustolisi
O.
2007
Ensemble modeling approach for rainfall/groundwater balancing
.
J. Hydroinform.
9
(
2
),
95
106
.
Lucas
J. E.
Kung
H. N.
Chi
J. T.
2010
Latent factor analysis to discover pathway-associated putative segmental aneuploidies in human cancers
.
PLOS Computat. Biol.
6
(
9
),
1
15
.
Maqsood
I.
Khan
M. R.
Abraham
A.
2004
An ensemble of neural networks for weather forecasting
.
Neural Comput. Appl.
13
(
2
),
112
122
.
May
R.
Maier
H. R.
Dandy
G. C.
2011
Review of input variable selection methods for artificial neural networks
.
Artificial Neural Networks – Methodological Advances and Biomedical Application
(
Suzuki
K.
, ed.),
Minns
A. W.
Hall
M. J.
1996
Artificial neural networks as rainfall-runoff models
.
Hydrol. Sci. J.
41
(
3
),
399
417
.
MOLIT
2004
Comprehensive Flood Control Plan of Nakdong River Basin
.
Ministry of Land, Infrastructure and Transport
,
Korea
.
Mulia
I. E.
Tay
H.
Roopsekhar
K.
Tkalich
P.
2013
Hybrid ANN-GA model for predicting turbidity and chlorophyll-a concentration
.
J. Hydro-Env. Res.
7
,
279
299
.
Rumelhart
D. E.
Hinton
G. E.
Williams
R. J.
1986
Learning representations by back-propagating errors
.
Nature
323
,
533
536
.
Wolpert
D. H.
1992
Stacked generalization
.
Neural Netw.
5
(
2
),
241
259
.
Zhou
Z. H.
Wu
J.
Tang
W.
2002
Ensembling neural networks: many could be better than all
.
Artif. Intell.
137
,
239
263
.