Comparing machine-learning-based black box techniques and white box models to predict rainfall-runoff in a northern area of Iraq, the Little Khabur River

Runoff from rainfall is one of the most significant sources of water supply in most regions of the world, particularly in arid and low-rainfall regions. In areas with arid to semi-arid climatic conditions, scientific methods of collecting and controlling surface water to provide water and prevent damage become increasingly essential. Hence, a more accurate estimation of the runoff resulting from rainfall in catchment basins is crucial (Yavari et al. 2022). Knowledge of the natural power of runoff production in basins is one of the essentials for basic planning for the optimal use of runoff (Li et al. 2018; Liu & Lu 2022). In dry countries where environmental issues and water scarcity are increasing, it is necessary to know about water resources in order to analyze the relationship between food production, economic activities, and the ecosystem (Khan & Coulibaly 2006). Also, many studies have been conducted on the impacts of runoff on the water quality of reservoirs and rivers and the resulting problems. It is easier to predict floods by simulating the rainfall-runoff process. Approximately 40% of natural hazards are related to floods, and every year 20–300 million people are affected by problems caused by floods. It is predicted that by 2050, floods will cause a thousand billion dollars in annual damage (Młyński et al. 2018; Youssef et al. 2021). These issues show the importance of rainfall-runoff predicting.

Forecasting the rainfall-runoff process has been one of the complex issues in water resources management. Studies have shown that optimization algorithms like Genetic Algorithm (GA) and Particle Swarm Optimization (PSO), neural networks, machine learning, and artificial intelligence can be suitable solutions to solve complex problems related to water resources like, simulation rainfall-runoff, coastal engineering, and safety in water supply networks (Kamranzad & Samadi 2013; Nourani et al. 2019b; Shahmirnoori et al. 2022). In recent years, various rainfall-runoff models have been developed and modified. Runoff rainfall models include artificial intelligence, conceptual, experimental, and physical models. Conceptual or white box models are based on physical regulations and can represent hydrological behaviors with practical expression. Compared to physical models, white box models require less data, and also, these models require less calculation volume and time (Kamranzad & Samadi 2013; Bartoletti et al. 2018). In modeling hydrological phenomena, there are two conceptual approaches: white box and black box models. White box models are presented based on governing mathematical relationships, equations, and physical parameters in the phenomenon. Models such as HEC-HMS, TOPMODEL, and HBV are conceptual models. These models can use fewer physical parameters than other models to estimate the appropriate response of the catchment area against precipitation, which is an advantage of these models (Abdessamed & Abderrazak 2019). White box models are presented based on governing mathematical relationships, equations, and physical parameters in the phenomenon. The purpose of white box models is to conduct scientific research on each component of the hydrological cycle's main operation to fully understand the mechanism and how the components work with each other. In black box models, it is impossible to present equations and mathematical relationships. In these models, the physical parameters affecting it cannot be estimated easily. This group aims to develop reliable connections between the measured parameters of the hydrological cycle, which are used to solve technical and scientific problems (Nourani et al. 2019a).

In black box models, it is impossible to present equations and mathematical relationships. In these models, the physical parameters affecting it cannot be estimated easily. This group aims to develop reliable connections between the measured parameters of the hydrological cycle, which are used to solve technical and scientific problems. Black box models estimate the intended output by receiving input and performing mathematical operations. All black box models have parameters and coefficients estimated according to the input and output data of observations. Therefore, black box models depend on input and output data in quantity and quality (Andrews & Boyne 2010; Nourani et al. 2019b). Due to the nonlinear nature of most hydrological processes, nowadays, artificial neural networks (ANN) and machine learning as self-constructed estimator functions are widely used in modeling and forecasting nonlinear time series in hydrological processes. In general, the primary and most obvious advantages of black box models compared to other statistical and conceptual methods can be categorized in the form of things such as not requiring prior knowledge, applying a nonlinear filter under the title of stimulus function, and the ability to process multivariate inputs with different characteristics (Molajou et al. 2021).

In the study of Hamdan et al. (2021) which took place in northern Iraq, the researchers used the HEC model to assess rainfall-runoff. The results confirmed the performance of the model for rainfall-runoff simulation. Jaber et al. (2017) analyzed rainfall-runoff in a part of Iraq. They used Geographic Information System (GIS) and remote sensing to analyze the data and to produce the runoff depth map for the study area. Misra et al. (2009) compared the support vector machine model with an artificial neural network to forecast runoff and sediment amounts in India. The results showed that the support vector machine and neural network performed well predicting runoff and sediment load, but the support vector machine showed relatively better performance. Adamowski (2013) compared a support vector machine and an artificial neural network in rainfall-runoff modeling in India by calculating RMSE, R, Slop, intercept, and MBE. The results revealed that both models were suitable. Veintimilla-Reyes et al. (2016) used two neural network types in their study, and Anusree & Varghese (2016) used three neural network models to predict rainfall-runoff. The results showed the neural networks’ positive performance in their research. Nourani (2017) employed a new type of neural network to forecast rainfall-runoff in their research. Humphrey et al. (2016) predicted rainfall-runoff by combining a conceptual model and a neural network. Miguélez et al. (2009) predicted urban runoff by combining ANN with the GA. This study used an artificial intelligence technique to model precipitation and surface runoff production for an urban area. The results of their work showed that runoff modeling using ANN had satisfactory results.

The literature review shows that in most research, only one conceptual model has been compared with the artificial intelligence model, and more models have not been used in the studies. However, examining more models and examining them with one another seems necessary because, based on the conditions in a region, different models can have different results in rainfall-runoff results, which should be considered. Also, considering that in most research, only two separate models have been compared, it is impossible to analyze the performance of two groups of conceptual models (white box) and black box with high accuracy. In fact, in a watershed, one conceptual model may be less accurate than a black box model but higher than another. In addition to the mentioned points, in previous research, ANN and support vector machines have been mainly used to investigate rainfall-runoff. The application of two ANFIS and GEP methods together has not been studied in past studies, which is considered in this study. Therefore, in this research, rainfall-runoff in Iraq is simulated with two conceptual models, HEC-HMS and TOPMODEL, and two black box models, Adaptive Neural Fuzzy Inference System (ANFIS) and Gene Expression Programming (GEP). In this study, statistical data of 5 years (2015–2020) have been used to estimate rainfall-runoff on a daily scale.

HEC-HMS is one of the most reliable software in the field of hydrology, which uses several mathematical models to calculate the rainfall-runoff hydrological model, flood trends, infiltration, and other cases. This model is one of the mathematical models with several sub-models in the sections of surface runoff, base flow, and channel flow and is used to simulate the hydrological behavior of watersheds. HEC-HMS has the main sections, including the general outline of the watershed, characteristics of sub-basins such as the area and type of hydrograph and flows modeling, the essential characteristics of waterways such as loss rates and trending methods, characteristics of control sites, climate model flow; Including meteorological stations, the method of effecting the amount of precipitation of each station on the amount of water and control indicators (Halwatura & Najim 2013; Wildhaber et al. 2017). Lu et al. (2013) and Wildhaber et al. (2017) used this program in their research and reported its results as favorable. In general, HEC-HMS software is one of the most well-known programs for runoff simulation, and it is widely used in the United States. Different methods include the Clark hydrograph, the Schneider unit hydrograph, and the SCS unit hydrograph to convert rainfall into runoff. It is inexpensive to use and has the ability for automatic calibration (Wildhaber et al. 2017).

TOPMODEL is a semi-distributive model in which the concept of a hydrological response unit based on topographic moisture index is used for basin homogenization. The topographical index usually introduces the topographical information used in this model. This index shows the tendency of flow accumulation and its movement in the direction of the downstream slope by the force of gravity. In this model, it is assumed that the hydraulic conductivity of the saturated zone can be estimated using the topographic slope of the earth's surface, and the relationship between the soil transfer ability and the depth is a power function proportional to the lack of saturation. It has been widely used to model precipitation and runoff in the past decades. This model's inputs include evaporation and transpiration, basin runoff height potential, precipitation, basin delay function, and topographic humidity index. In order to optimize the model parameters, the random jump method was used. TOPMODEL model is one of the standard models in hydrological analysis in many European countries (Campling et al. 2002; Khan & Coulibaly 2006).

ANFIS combines the advantages of ANN and fuzzy logic (FL) in a single framework. It provides rapid learning and adaptive interpretation capabilities for modeling complex patterns and understanding nonlinear relationships. ANFIS has been applied and practiced in various fields and has provided solutions to common iterative problems with improved temporal and spatial complexity. Therefore, it can effectively simulate rainfall-runoff, which is a complex problem. If the output of each neuro-fuzzy network layer is shown as ⁠, which is the output of the ith group in the first layer, then the performance of the different layers can be expressed as follows (Yaseen et al. 2018; Adnan et al. 2021):

Gene Expression Programming is a generalized GA that was first proposed based on Darwin's theory. In this way, in the direction of evolution, a population selectively abandons the unsuitable population and creates modified children. Tree structures are created from the set of functions (mathematical operators used in formulas) and terminals (problem variables and fixed numbers). Before the execution steps of GEP, the following preliminary steps must be determined by the user: 1 – The set of terminals (random fixed number problem variables), 2 – The set of mathematical operators used in the formulas, 3 – Selecting the fit function to measure the fit of the formulas, 4 – Determining Controlling parameters of the program execution, 5 – Criterion of the end and presentation of the results of the program execution, such as the number of new population production, determining a specific value for fitting the formulas, if the reasonable rate is equal to or greater than that value, the execution will be stopped. One of the positive points of the GEP method is that the search operators of GEP always generate functional structures and are fitted to genetic diversity. Another positive point of GEP is its unique, multigenic nature which permits the development of more complex programs composed of several subprograms. So GEP can be a suitable method for estimating rainfall-runoff simulation, considered a complex problem (Fernando et al. 2012; Molajou et al. 2021). In this research, in order to simulate the rainfall-runoff process with the GEP model, GeneXproTools 4.0 software is utilized.

The data include the daily runoff, rainfall, and temperature during the 5-year period. The inputs of the black box models are based on previous studies and information from relevant authorities. Based on this, seven scenarios have been considered. Table 1 shows the composition of input data. P is rainfall, T is temperature, and Q is stream flow. 80% of the total data was used for training and the remaining 20% was used for testing.

Table 1

Combination of inputs

Scenarios .	Input .	Output .
1
2
3
4
5
6
7

Scenarios .	Input .	Output .
1
2
3
4
5
6
7

View Large

The results of the evaluation criteria of HEC-HMS and TOPMODEL models are shown in Table 2. A 5-year data was used for training, and 1-year data for testing. The error values of both models obtained in the test phase were lower than in training, which shows that the models could not be trained well and could not simulate the runoff with the accuracy of the training phase. The error values of both HEC-HMS and TOPMODEL models are close to each other in the test phase, and both models predict runoff with little accuracy. The RMSE of HEC-HMS is higher than TOPMODEL. The most significant difference is related to the MBE, which indicates that TOPMODEL estimated the rainfall-runoff much more than the actual value.

The results of the performance of ANFIS and GEP with different scenarios are displayed in Table 3. The results in this table demonstrate that the discharge input one day before cannot predict the rainfall-runoff well, and with the addition of the discharge 2 days before, the ANFIS error value stays constant, and GEP shows a decrease. According to the results of scenarios 1 and 2, both rainfall models of the previous day were added to scenario 2, and scenario 3 was implemented. Comparing the error rate of scenario 3 with scenario 2 in the test phase shows that ANFIS and GEP models have reduced the error amount, which is better results, and rainfall and discharge significantly affected the accuracy of runoff estimation.

Scenario 4 was done by mixing the flow rate input up to two previous time steps and the previous day's temperature. The results revealed that the ANFIS model simulated the runoff with the lowest error value and the highest R in the test phase. Nevertheless, the accuracy of the GEP simulation with scenario 4 and the addition of the previous day's temperature were similar to the previous day's rainfall (scenario 3).

Scenarios 5, 6, and 7 with four input parameters could not provide more favorable results. In some cases, they provided similar or weaker results than scenarios 3 and 4 with fewer input parameters. It should be noted that with the increase in the number of inputs, the computing speed of the model decreased, so running the model with a smaller number of inputs is desirable if the results are similar. Adding the rainfall 2 days before to the ANFIS model (scenario 5) improved the results, but its effect was less than the previous day's temperature. Also, by adding the temperature two days before (scenario 6), the accuracy of the simulation decreased. By adding the rainfall two days before to scenario 3, scenario 7 was created, which slightly reduced the accuracy of GEP and increased the accuracy of ANFIS.

The previous day's temperature and rainfall in scenarios (3 and 4) had a similar effect on the simulation of rainfall-runoff in the GEP model. However, in the ANFIS model, the effect of the previous day's temperature on the improvement of runoff simulation results was more significant than the previous day's rainfall. ANFIS in scenario 4 and GEP in scenarios 3 and 4 were selected as acceptable scenarios. Scenario 4 in ANFIS with RMSE 1.32, R² 0.82, and MBE −0.06 provided the best result in the test stage. The RMSE of the best GEP scenario is about 16% higher than that of ANFIS, and ANFIS has been able to simulate the runoff more accurately.

Table 4 illustrates the performance of four study models in simulating the rainfall-runoff process. The error values of rainfall-runoff process simulation in black box models are lower than in white box models. Also, black box models had lower R² values than white box models. Figure 3 demonstrates that ANFIS and GEP could estimate the rainfall-runoff process more accurately, showing the power of black box models in analyzing the rainfall-runoff process and more accurate simulation. While the RMSE of ANFIS was lower than the RMSE of GEP, the accuracy of both ANFIS and GEP methods was acceptable. According to the results of this research, it can be concluded that the two white box models (HEC_HMS and TOPMODEL) could not simulate the dominant processes of the basin due to their structure. According to the available data, the black box models’ flexibility was generally higher than the white box models. One of the reasons for the good performance of black box models can be considered as their inputs, which are the flow rates in the previous time step, which significantly impact the rainfall-runoff simulation's accuracy. Due to the fact that in black box models, only input and output data are examined, the presence of noisy data significantly affects the model's performance.

In general, the results of this research showed that black box models could work well in rainfall-runoff estimation because it was shown by Adamowski (2013) and Veintimilla-Reyes et al. (2016) that machine learning is a suitable tool for predicting rainfall and runoff. This study also demonstrated that the ANFIS model is suitable for rainfall-runoff estimation. The positive performance of the ANFIS model in estimating runoff precipitation was also reported in the research of Nourani & Komasi 2013. The better performance of the black box model compared to the white box model in Khan & Coulibaly (2006) study has the same result. They showed that the ANFIS model could more accurately estimate the rainfall-runoff process than the HBV model.

To investigate the effect of rainfall and temperature in different months on the accuracy of rainfall-runoff simulation, scenarios 3 and 4 were used in GEP and ANFIS models, which had the highest accuracy. Tables 5,⁶–7 show the evaluation criteria of scenarios 3 and 4 in different months for the two models. The highest R²-value of ANFIS and GEP models with scenario 4 is for March and June, respectively, which are almost similar (Table 5). The lowest R² also corresponds to August and September, and the results of both models are almost similar.

The RMSE results in Table 6 demonstrate no significant difference between GEP and ANFIS scenarios and models in October, and the rainfall-runoff prediction error was almost the same. In December, ANFIS-3 and GEP-4 provided the same results. In January, scenario 4, both models, and in February and March, the GEP-4 scenario had lower RMSE. Because in the winter and cold months of the year, sometimes the weather was snowy, the temperature parameter reduced the amount of model error.

When the temperature increased in April and May, there was no difference between the scenarios and models, and the results were almost identical. In June, in scenario 4, both models were slightly better than scenario 3. In the hot months of July, August, and September, models and scenarios did not differ much; their RMSE was almost the same. In general, from December to March, when the weather becomes cold, the temperature parameter increases the accuracy of the forecast. In spring and summer, except June, no difference was observed between scenarios 3 and 4. Table 7 shows that the models predicted rainfall-runoff less than the actual amount in March and April.

This study used HEC-HMS and TOPMODEL as white box models and ANFIS and GEP as black box models to simulate rainfall-runoff during a 5-year statistical period. A combination of runoff, rainfall, and temperature data on a daily scale was selected as input parameters. Then, seven events were defined for rainfall-runoff simulation. Using rainfall and temperature inputs of the previous day and discharge in the previous two days’ steps reduced the prediction error of both models. In this research, by investigating the rainfall and temperature factors in improving the simulations, it was found that the temperature factor as an effective parameter in the cold months reduced the amount of prediction error. Also, it was found that the accuracy of both black and white box models in the simulation of the hot months of the year is low, and the accuracy of the runoff prediction has not improved by changing the scenario. Comparing R², RMSE, and MBE demonstrated that black box models act as more effective tools in predictions, and using white box models as a suitable tool for the initial estimation was acceptable. Both black box models (ANFIS and GEP) could simulate daily rainfall-runoff. Although the accuracy of ANFIS was higher than the GEP model, using the GEP model is easier than the ANFIS model and can be used as a practical and competitive method with other methods in rainfall-runoff simulation. It should be noted that although black box models do not need equations with a physical basis, understanding the processes and the effect of the chosen factors in the input data on the output of the models is very important. In the absence of understanding, it cannot be Expected good results. If the black box models are run for a range that has not been trained, they usually use extrapolations that are not reliable. Also, some black box models, such as ANFIS, require much time to run, which makes using these models complicated and time-consuming.

Due to the importance of rainfall-runoff, it is suggested that researchers use other black box models, such as Support Vector Machine and Bayesian Network, in future studies. It is also recommended to use and examine other white box models, like HBV and HYMOD, to compare with black box models in future studies.

All relevant data are included in the paper or its Supplementary Information.

The authors declare there is no conflict.