Water level prediction model of the upper Yangtze River by cloud computing and AutoML

As the most important waterway in China's inland areas, the Yangtze River waterway flows through many important economic provinces and cities such as Chongqing, Hubei, and Shanghai. It plays an increasingly prominent role in developing China's national economy (Peng et al. 2010). The water level is the main index of waterway scale maintenance, and its timely and accurate prediction helps ships predict water level changes in advance, predict and choose routes, and ensure the navigation safety of ships (Scheepers et al. 2018; Zhou et al. 2020). With the continuous development of information technology, the short-term water level prediction and forecast technology of the Yangtze River waterway is an important means to predict waterway conditions and guide waterway maintenance. It is also an important technical support for promoting the construction of intelligent waterways and comprehensively improving the management and public service level of the Yangtze River waterway.

At present, the research methods of water level prediction can be summarized into two categories: process-driven model and data-driven model (Chen et al. 2003). The process-driven model is based on the existing physical theory and empirical formula to establish the correlation between various hydrological elements. For instance, the Stanford Model proposed by Linsley & Crawford (1960), the Xin'an River model proposed by Zhao & Wang (1988), and the Institute of Hydrology Distributed Model (IHDM) proposed by Morris (1980). Furthermore, advancements have been observed in hydrodynamic modeling, particularly those grounded in the Navier–Stokes equations, concerning water level prediction (Medeiros & Hagen 2012; Woodruff et al. 2013). Its advantage is that the complex water level process can be described by some functional relationships of physical processes. However, obtaining detailed hydrophysical process data in the project is difficult, so the model is subject to many empirical assumptions, and the prediction effect is not ideal (Jiang et al. 2014). Simultaneously, the model's accuracy is subject to uncertainties in boundary conditions, parameterization, and structural errors inherent in coupled systems (Alizadeh et al. 2021). However, waterway managers often face challenges when utilizing the complex physical model, as it encompasses numerous intricate formulas that are difficult to apply in their daily tasks (Dou et al. 2007; Xu et al. 2016).

Data-driven models, functioning as intelligent models, typically disregard the physical mechanisms of hydrological processes, focusing instead on discovering the optimal relationship between inputs and outputs, thereby exhibiting high predictive accuracy and the capacity to model complex scenarios (Phan & Nguyen 2020; Dash et al. 2021; Wang et al. 2022). Commonly used models include time series analysis based on mathematical statistics and machine learning (ML) represented by artificial neural networks (ANNs) and support vector machines (SVMs). The evolution of ML has progressed through two waves: shallow learning and deep learning. Shallow learning models exhibit constrained representation and generalization abilities when handling complex high-dimensional functions and intricate problems, often resulting in lengthy computation times. Conversely, deep learning focuses on leveraging deep structures and explicit feature learning to efficiently characterize complex functions with fewer parameters, even with limited samples. This approach enhances the accuracy of classification and prediction tasks in the final learning process (Guo & Ding 2015).

In recent decades, ML algorithms, an affordable and effective modeling approach, have been widely employed to learn and simulate a wide range of hydrological events characterized by high-dimensional variables, nonlinear relationships, and time series (Khan et al. 2020; Yan et al. 2021; Hilal et al. 2022). Water level prediction research has experienced substantial advancements. Gharehbaghi et al. (2022) used the gated recurrent unit (GRU) model to predict the groundwater level. Park et al. (2022) chose the GRU model to improve the accuracy of water level prediction and adopted multiple learning methods to effectively use meteorological data to improve the prediction accuracy of daily water levels. Nguyen & Le (2019) employed the SVM method for the prediction of water levels in the Tich-Bui River, Vietnam. Zhao et al. (2020) employed SVM for real-time water level prediction in the lower reaches of the Yellow River, China. Nhu et al. (2020) utilized a diverse array of regression tree models, including random forest and M5, to predict daily water levels at Zrebar Lake in Iran. Castillo-Botón et al. (2020) employed various ANNs and SVMs for reservoir water level forecasting at the Calicia Hydroelectric Power Station in Spain. Ren et al. (2020) developed a multi-layer perceptron and recurrent neural network (RNN) to forecast water levels in the South-to-North Water Diversion channels in China. Li et al. (2020) proposed a self-attention mechanism LSTM network, to effectively model and forecast water level dynamics in small- and medium-sized rivers with diverse terrain characteristics. Sun et al. (2022) incorporated precipitation, air temperature, water vapor pressure, and mining volume as input variables to formulate a groundwater level prediction model using a multivariable long short-term memory (LSTM) neural network. Chen et al. (2023) introduced a novel WaveNet-driven convolutional neural network model, demonstrating superior performance in forecasting water levels at the Waizhou gauge station along the Ganjiang River (GR), China. When utilizing ML for research purposes, users are expected to possess a certain level of programming proficiency and need to set up a high-performance computer for program execution.

The exponential growth of data and the escalating demand for computing resources have given rise to cloud computing (Surbiryala & Rong 2019). As defined by the National Institute of Standards and Technology (NIST), cloud computing is a model predicated on shared resource pools and flexible resource access (Mell & Grance 2010). Individuals can effortlessly access a versatile array of resources, including network, computational, and storage capabilities, as well as accompanying software services, from a customizable shared resource pool, with the flexibility to do so from any location and at any time. Cloud computing facilitates the rapid provisioning and delivery of resources to users. In comparison to traditional resource provisioning methods, cloud computing offers increased flexibility, enhances resource utilization efficiency, and elevates service quality (Xiao et al. 2023). Azure AutoML, a cloud computing platform designed for ML and big data analysis, represents an automated modeling technology unveiled by Microsoft in 2018 (Barga et al. 2015). This innovative platform encompasses functionalities for model structure search, hyperparameter exploration, and algorithmic support for prevalent classification, regression, and time series prediction algorithms, which can effectively improve the efficiency of data analysis using ML methods. The main advantages of the platform are the ability to try multiple models at once in a single experiment and compare the results, helping to find the most suitable solution. Specifically, the purpose of establishing a prediction model is to establish a multi-algorithm model in the same experiment, make a comparative analysis of the prediction results, select a suitable learning algorithm, and train massive data to achieve the purpose of establishing a prediction model (Trivedi 2023).In contrast to competing solutions like Google AutoML and Baidu EasyDL, Azure AutoML stands out for its user-friendly network accessibility, low-code approach, and the ability to execute most computations and operations directly within a web browser. The intelligent and digital transformation of the Yangtze River shipping route is currently in its early stages (Ma et al. 2022; Huang et al. 2024); the undergraduate major in smart water conservancy was introduced in 2022 to the directory of professional programs at ordinary higher education institutions. Due to their inherent complexity, traditional ML and deep learning approaches are less commonly employed in the routine practices of shipping route management. The application potential of AutoML, integrated with cloud computing technology, is deemed promising in this context. As a successful application, Azeem & Dev (2023) used AzureAutoML to improve the accuracy of rainfall prediction with an accuracy of 86.5%.

For promoting the practical application of ML techniques in the Yangtze River, it is necessary to compare and analyze the accuracy difference between the Azure AutoML and the traditional ML model for short-term water level prediction (Yuan et al. 2022). Using the Zhutuo Hydrological Station as a focal point, the Station, a fundamental hydrological research facility, was established to investigate the alterations in the water regime following the confluence of significant tributaries such as the Jinsha, Hengjiang, Minjiang, and Tuojiang in the upper reaches of the Yangtze (Li et al. 2022). Its primary objective is to decipher the river's hydrological characteristics. Its primary function is to serve as a data foundation for flood management, basin planning, engineering design, water resource development, utilization, hydrological analysis, water environmental protection, sediment studies, and other related fields. It also contributes to the accumulation of essential hydrological data for supporting industrial, agricultural, defense, and national economic development initiatives (Zhang et al. 2023).

This study compares and analyzes the accuracy difference between the Azure AutoML platform and the RNN model for short-term water level prediction. The comparison is based on the collection of continuous long-time series observation data from hydrological stations. Furthermore, this study discusses methods to enhance the accuracy of both the Azure AutoML platform and the RNN model. Additionally, it examines the operational difficulty and convenience of these two methods from the perspective of waterway managers, offering a novel approach to short-term water level prediction.

RNN is an ML technology built upon ANNs. Unlike traditional neural networks, RNNs process input data considering time series, leading to the interdependence between input and output. This enables the retention and utilization of output data for subsequent time steps, showcasing the persistence of information (Graves et al. 2013). RNN, characterized by loops within its structure, excels at processing sequence information by incorporating both current input samples and previous input data. Consequently, RNN emerges as a deep neural network specifically tailored for modeling sequence data.

Neural networks typically consist of an input layer, an output layer, and one or more hidden layers, each equipped with distinct activation functions regulating the output. These layers are interconnected by weights, which are adjusted during neural network training to optimize performance and accuracy. One of the key distinctions between RNNs from traditional neural networks lies in the interconnectedness of neurons within their hidden layers (Tsai et al. 2018). This unique architecture enables the conversion of information from the current hidden layer to the next, allowing for the retention and utilization of previous data to enhance the accuracy of data sequence predictions. By effectively memorizing past information and incorporating it into output calculations, RNNs establish a robust framework for data sequence forecasting (Gelenbe 1993). The presence of a cyclic hidden layer in RNN results in parameter sharing within the network architecture, significantly reducing the number of parameters requiring training. This characteristic not only simplifies the training process but also fosters efficiency in model optimization. Additionally, the nonlinear relationships captured between sequential data have found extensive application in domains such as water level prediction, showcasing the versatility and effectiveness of RNNs in modeling temporal dependencies (Wang 2019).

The output characteristic of the recurrent kernel at the current time step, denoted as y_t, where w_hy represents the weight matrix, by is the bias term, and softmax denotes the activation function, is essentially analogous to a fully connected layer. The memory capacity can be adjusted by configuring the number of memory units. When a specific number of memory units is defined, the dimensions of x_t, y_t, and the surrounding parameters to be trained are constrained. When the current propagates forward, the state information h_t stored in memory is refreshed continuously, while the three parameter matrices and two bias terms remain consistently estimated.

Section 4.1.1 compares two distinct RNN models: the preferred hyperparameter model (RNN-P), which employs optimized hyperparameters, and the specified hyperparameter model (RNN-S). Preferred hyperparameters denote the set of optimal values discovered during training within predefined ranges, while specified parameters are assigned without customization, relying on prior expertise or empirical evidence.

Azure AutoML employs computational resources at your disposal to execute a series of iterative experiments, exploring diverse algorithmic configurations and hyperparameters, with the primary objective of training a model that exhibits satisfactory performance. Once the optimal model is obtained, it becomes seamlessly integrated into any application you construct, facilitating its practical application. The fundamental process of employing a framework is divided into four sequential stages: dataset loading, computing resources configuring, AutoML running, and verification (https://learn.microsoft.com/en-us/azure/machine-learning/concept-automated-ml?view=azureml-api-2).

Azure AutoML incorporates a variety of ML algorithms, as detailed in the provided link (https://learn.microsoft.com/en-us/azure/machine-learning/concept-automated-ml?view=azureml-api-2#when-to-use-automl-classification-regression-forecasting-computer-vision–nlp). Among these, certain models share the same algorithm name but are considered distinct due to variations in their hyperparameter settings. Its configuration and operating interface are depicted in Figure 2 and Figure 3.

Section 4.1.2 compares two distinct AutoML models: one designed to handle multi-factor inputs, denoted as multi-factor AutoML (AutoML), and the other relying on a single-factor input, referred to as single-factor AutoML (AutoML-one). Multi-factor inputs encompass the integration of flow rate data from Fushun and Zhutuo, alongside water level measurements from Lizhuang, Fushun, and Zhutuo stations. In contrast, single-factor input is limited to the utilization of water level data exclusively from the Zhutuo station.

The Fushun Hydrological Station, functioning as a key control point for the Tuojiang River's mainstem downstream, is strategically located in the lower reaches, at an approximate distance of 113 km from the river's mouth, as depicted in the illustration. The Lizhuang Hydrological Station serves as a critical water monitoring station for the Yangtze River's upper mainstem, located approximately 19 km downstream from the confluence of the Jinsha and Min Rivers, and approximately 114 km upstream from the junction with the Tuojiang and the Yangtze. The Zhutuo Water Level Station, another basic station, is situated at a distance of roughly 105 km from the point where the Tuo River merges with the Yangtze, and the specific locations of each cite are shown in Figure 4. The actual conditions of Zhutuo Station are depicted in Figure 5.

The alignment between the input data employed in this investigation and the distinct model architectures is detailed in Table 1, including water level and flow data, which are crucial input factors for water level prediction. The dataset, comprising hourly observations from Lizhuang (LZ), Fushun (FS), and Zhutuo (ZT) over the past 3 months, was partitioned into three subsets in a 7:2:1 ratio (70% for training, 20% for validation, and 10% for testing). Using the single-factor and multi-factor input models developed, the next 1-, 8-, and 24-h outcomes were predicted independently. The model's accuracy and performance were assessed by comparing the predicted values with the 10% test set data. The selection of hourly data was justified by the need for hourly-scale data for navigation in the ZT section. In the context of this paper's research, the forecasting model's data can be updated hourly. In real-world applications, to cater to ship navigation needs, the forecast values are typically updated 3–6 times a day. The water level and flow data are available from the Bureau of Hydrology of Changjiang Water Resources Commission (https://www.cjh.com.cn).

To accurately and effectively assess the predictive performance of the model, three evaluation methods were utilized to measure the disparities between observed and predicted values: root mean square error (RMSE), mean absolute error (MAE), and training duration (TD). RMSE is a widely used metric that quantifies the discrepancy between observed and predicted values. A decrease in RMSE value signifies a reduced error magnitude. Conversely, MAE calculates the mean of the absolute differences between observed and predicted values. Like RMSE, a lower MAE value indicates enhanced predictive precision of the model. TD is a crucial metric for assessing the efficiency and speed of model training. A reduced training duration signifies accelerated model training on the specified dataset. The TD for AutoML models is automatically calculated by the program, whereas the TD for RNN is realized through the writing of statements related to the time module.

The evaluation of ML-based prediction models is conducted through a set of three performance metrics: RMSE, MAE, and TD. RMSE and MAE, being the prevalent methods for obtaining consistent findings, will be delved into in greater detail subsequently. Table 2 furnishes the RMSE, MAE, and TD values for each ML model across both the training and prediction stages.

In general, the efficiency of ML models tends to degrade as prediction time stretches. As depicted in Table 2, for models RNN-P, RNN-S, AutoML, and AutoML(one), the RMSE and MAE escalate with an extended prediction horizon, revealing a consistent decrease in model performance across both training and testing scenarios (see Table 2 for quantitative evidence).

An initial comparative analysis was performed between two RNN models, as shown in Figure 6. The comparative analysis in Table 2's training section reveals that the RNN-P model consistently delivers lower RMSE and MAE across all time horizons (1, 8, and 24 h), albeit with a longer training period. The RNN-P model demonstrates a clear advantage over RNN-S during the prediction phase, with nearly equivalent prediction times. Consequently, the RNN-P model's overall performance is superior.

In the subsequent analysis, a comparative evaluation is conducted between two distinct AutoML models, as shown in Figure 7. As evident from columns 3 and 4 of Table 1 in the training section, the AutoML model consistently outperforms AutoML-one in terms of RMSE and MAE, except for the 1-h RMSE results, where AutoML-one performs better. Both models exhibit identical training durations of 300 s. During the prediction phase, the AutoML model consistently exhibits better performance than AutoML-one, except for marginally higher 1-h RMSE and MAE values. In summary, the multi-factor input AutoML model displays a superior overall performance when compared to its single-factor counterpart.

In this comparative analysis, we opt for the most proficient models by examining the RNN model with fine-tuned hyperparameters (RNN-P) against an AutoML model that leverages multi-factor inputs for enhanced performance, as shown in Figure 8. Comparing the 1 and 3 columns of the training section of Table 1, it is evident that the AutuML model exhibits a notable advantage in terms of the three evaluation metrics: RMSE, MAE, and TD, for the 1, 8, and 24 h forecast horizons. This advantage is particularly pronounced in the TD metric, suggesting that the AutuML model effectively captures patterns and regularities within the training data. In the testing section, the values of RMSE and MAE for the three forecast horizon predictions are generally comparable, with the AutoML model slightly outperformed by the RNN-P model. Specifically, in the 1-h and 8-h predictions, the AutoML model's RMSE is marginally higher than that of the RNN-P model by 0.015 and 0.017, respectively; in the 24-h prediction, the two models are at the same level. Similarly, the MAE values of the AutoML model for the three forecast horizons are all slightly higher than those of the RNN-P model by 0.003, 0.001, and 0.001, respectively. In terms of efficiency, the AutoML model still maintains a certain advantage. The comparative analysis indicates that both models possess distinct strengths, with the AutoML model demonstrating a notable efficiency advantage, particularly in scenarios where the precision differences are minimal. This advantage is particularly beneficial for managers and operational personnel.

Subsequently, a comparative analysis will be undertaken for the three models, with a common focus on the input data, specifically the Zhuotuo water level as the input variable. During the training phase, the RNN-P model demonstrates comparable performance to the AutoML-one model across both the RMSE and MAE metrics. In contrast, the RNN-S model exhibits a notably higher error rate. Regarding training efficiency, the AutoML-one and RNN-S models outperform in terms of computational time, as they require less time for training compared to the RNN-P model, which incurs a longer duration due to its inherent process of optimizing hyperparameter configurations. In the evaluation of model predictions, the RNN-P model consistently outperforms with the lowest RMSE and MAE values, followed by the AutoML-one model in a closely competitive second place. In terms of computational efficiency, the AutoML-one model outperforms with a prediction time of less than 2 s, contrasting with the other two models, which require approximately 10 s per prediction. This analysis indicates that, when focusing on single variable inputs, the AutoML-one exhibits a marginal decrease in accuracy compared to the RNN-P model, while showcasing a substantial boost in computational speed.

As a stand-alone model, the RNN model possesses three main advantages: (1) Easy parameter adjustment, allowing for the adaptation of test parameters based on demand, and the selection of optimal hyperparameters through output result comparisons. The hyperparameters primarily encompass the learning rate, batch size, and number of epochs. The learning rate, for instance, governs the magnitude of weight adjustments. An excessively high learning rate can impede the network's ability to converge, whereas a learning rate that is too low can result in a sluggish learning process. The batch size specifies the quantity of samples utilized for each weight adjustment. A larger batch size can offer more reliable gradient estimates, yet it may also give rise to memory constraints and necessitate additional iterations for convergence. The number of epochs signifies the frequency with which the entire dataset is traversed. Incrementing the number of epochs can enhance network performance but may also contribute to overfitting. (2) Straightforward to comprehend and remember. This mechanism facilitates the retention of previous information through cyclic connections within the network, enabling the network to identify dependencies between elements in the sequence, thereby enhancing understanding. (3) Easy to rewrite the program, allowing users to optimize and adjust the code according to their needs. (4) Modifications and adjustments to the input data are minimal. Users can directly access the data and execute the program without the need to alter the input data format or manually segment the input data. However, RNNs encounter several challenges, including the issues of gradient vanishing and gradient explosion, which can impede the network's ability to capture long-range dependencies. Furthermore, the utilization of an RNN model necessitates a thorough understanding of its structure and the functionality of each component, a process that demands considerable time and a high level of programming expertise from the user. Moreover, the operation of this model requires a robust computational setup and incurs significant runtime expenses.

The AutoML platform built on cloud computing offers four key advantages: (1) Easy operation and low threshold of use. Users are not required to have in-depth knowledge of ML algorithms, even non-professionals can also directly construct models by following operational instructions, and gradually complete the steps of data input, model training, and result output. (2) Spend less time and resources. the automated platform minimizes time and resource consumption through its automated process, accelerating model construction and optimization, thereby conserving valuable time and resources. The ability to automatically read data, train, and generate models, and evaluate model performance is a significant advantage. However, data collection still requires manual effort. Fortunately, the data collection process for this study is convenient, as we can obtain data in real-time through the Bureau of Hydrology of Changjiang Water Resources Commission. (3) Demonstrate high performance. The AutoML system identifies optimal-performance models by examining the hyperparameter combinations of hundreds of models provided by different vendors, including Azure OpenAI Services, Mistral, Meta, Cohere, Nvidia, and Hugging Face. (4) Transparency and explainability. Platforms provide users with comprehensive insights into the models and the computational processes. This allows users to easily understand how the model was created and what it contains, fostering a deeper comprehension of the model's inner workings. (5) The computer configuration does not have excessively high requirements. Since training and calculations are conducted on the cloud platform, eliminating the need for local configuration consumption, the AutoML platform is more suitable for water resources and waterway managers than for RNNs.

The field of water resources has seen some relevant research on AutoML. For instance, Guo et al. (2022) developed AutoML-based models for urban inundation and early warning analysis, offering more efficient solutions to the multivariable and nonlinear relationship between building configuration and urban inundation. Wang et al. (2024) utilized AutoML to enhance the prediction of groundwater potential in Qinghai Province. Venkata Vara Prasad et al. (2021) employed AutoML models to assess water quality, noting improvements of 1.4 and 0.5%, respectively, over traditional ML methods for binary and multi-class water quality data. To our knowledge, this paper represents the first application of AutoML technology to navigable water level forecasting. In essence, AutoML represents a novel approach with potential utility within the domain of water resources. The method's specific utility is contingent upon the modeling and computational efforts applied to various problems. Despite not guaranteeing optimal outcomes, AutoML's simplicity and user-friendliness render it a suitable benchmark for evaluating model performance.

In conclusion, both methods offer distinct advantages, as shown in Table 3. The editable nature of the RNN model program provides model developers and researchers with greater flexibility to conduct thorough research and analysis. On the other hand, for managers and scheduling decision-makers, the AutoML platform offers ease of operation and use. The faster computing speed it provides can offer timely insights for daily management and decision-making.

The proposed two models were evaluated through practical experiments conducted using the water level dataset from the Zhutuo Station. The results suggest that: (1) A shorter prediction time horizon for AutoML and RNN models yields a smaller discrepancy between MAE and RMSE, thereby enhancing the accuracy of predictions. In other words, short-term forecasting outperforms long-term forecasting. (2) The two models effectively predict short-term water level data changes. Compared to the RNN model, the AutoML model exhibits a higher computational efficiency while maintaining a similar overall accuracy. Moreover, the method achieves a good balance between model complexity, ease of modeling, and hardware requirements, rendering the AutoML model more suitable for use by front-line workers in navigation.

The findings of this study have validated the utility of AutoML in the upper reaches of the Yangtze River, specifically for short-term water level forecasting. It remains to be seen, however, whether AutoML is equally effective in other regions, particularly those focused on medium- to long-term forecasting outcomes. This paper offers a straightforward, swift, and cost-effective approach to model construction, which is highly recommended for researchers across various disciplines seeking to address their unique challenges.

The work presented in this paper is financially supported by the Science and Technology Program of the Yangtze River Waterway Bureau (202223001), the National Key R&D Program of China (2023YFC3209502), and the National Natural Science Foundation of China (U2340217). And the author would like to thank Microsoft's AZURE platform for providing computing power support.

Data cannot be made publicly available; readers should contact the corresponding author for details.

The authors declare there is no conflict.