The normal probability density function (PDF) is widely used in parameter estimation in the modeling of dynamic systems, assuming that the random variables are distributed at infinite intervals. However, in practice, these random variables are usually distributed in a finite region confined by the physical process and engineering practice. In this study, we address this issue through the application of truncated normal PDF. This method avoids a non-differentiable problem inherited in the truncated normal PDF at the truncation points, a limitation that can limit the use of analytical methods (e.g., Gaussian approximation). A data assimilation method with the derived formula is proposed to describe the probability of parameter and measurement noise in the truncated space. In application to a water distribution system (WDS), the proposed method leads to estimating nodal water demand and hydraulic pressure key to hydraulic and water quality model simulations. Application results to a hypothetical and a large field WDS clearly show the superiority of the proposed method in parameter estimation for WDS simulations. This improvement is essential for developing real-time hydraulic and water quality simulation and process control in field applications when the parameter and measurement noise are distributed in the finite region.

  • The truncated normal probability density functions (PDFs) are developed.

  • A new data assimilation method utilizing truncated normal PDF is proposed.

  • The method is used for demand estimation in water distribution systems.

Parameter estimation and calibration play a crucial role in the modeling and management of water resources systems, as emphasized in several studies (Savic et al. 2009; Beckers et al. 2020; Scott et al. 2022; Yoon et al. 2022). The main objective is to minimize discrepancies between model outputs and measured values by adjusting network parameters, including nodal water demand and pipe roughness. However, this task can be challenging due to the large number of parameters involved and the limited availability of measured data. Consequently, parameter estimation often faces ill-conditioning issues, where the insufficient number of measurements leads to non-unique solutions within the search domain (Shao et al. 2019). In addition, uncertainties in both the measurement and model itself can significantly impact the parameter estimation accuracy. Often time parameter estimation from such a noisy environment is a major challenge for the modeling of water resources systems (Chu et al. 2021a, 2021b). Data assimilation with consideration of various uncertainties has been used for parameter estimation in model simulations (Vrugt et al. 2005; Hutton et al. 2014; Zhou et al. 2020).

Data assimilation estimates the state of a process based on time-series measurements. The state estimator relies on the knowledge of the posterior probability distribution function (PDF) of the state given the real-time measurements (Bar-shalom et al. 2001). Generally, the posterior PDF is approximated in a recursion, which involves two stages: the prediction stage and the update stage (Garcia-Fernandez et al. 2012; Hutton et al. 2014). In the prediction stage, the PDF of the state at the current time step is predicted from the historical data, referred to as the prior PDF. In the update stage, the likelihood describes the probabilistic relationship between the state and measurement. The likelihood and prior PDF are combined based on the Bayesian rule to approximate the posterior PDF (Garcia-Fernandez et al. 2012).

Numerous data assimilation methods have been proposed to solve the state estimation problem through the sample-based and analytical methods (e.g., Gaussian approximation). The sampling-based, such as particle filter, Markov Chain Monte Carlo, stipulates that one or more samples (particles) are sampled from the prior PDF, and then the likelihood or posterior PDF evaluates the sampled value. If the sampled value agrees with the likelihood or posterior PDF, it is retained, and the algorithm proceeds to the next variable in turn (Bishop 2006). However, sampling-based methods may be time-consuming for large-scale non-linear systems because they require frequent evaluation of the samples' probability density. For this reason, there is considerable interest in computationally efficient analytical methods (e.g., Gaussian approximations) (Garcia-Fernandez et al. 2012). Kalman filter is the most well-known analytical method for state estimation in the linear system. For the non-linear system, the extended Kalman filter (Singh et al. 2022) and the iterative Kalman filter (Huang et al. 2022) have been developed, in which a linearized approximation of the system function is required (Garcia-Fernandez et al. 2012; Shao et al. 2019). In the linearization of the system, the first-order gradient (Jacobian matrix) or second-order gradient (Hessian matrix) information is utilized to search for the optimal solution. This analytical method is more computationally efficient.

One common assumption in the analytical method is that the prior PDF of the state variable conforms to a normal PDF over the full range (−∞, ∞) (Law et al. 2015; Yang et al. 2018; Shao et al. 2019). In practice, however, the state variable is usually distributed in the finite region. Besides, no sensors can provide an infinitely large measurement (Garcia-Fernandez et al. 2012), for which the measurement noise should be bounded in the reasonable region. This practice is not consistent with the unbounded likelihood assumed in most models (Garcia-Fernandez et al. 2012). Therefore, the state variable or measurement constraints to confine them in the feasible domain must be considered in the data assimilation process (Lauvernet et al. 2009), a difficult requirement that calls for developing efficient algorithms to handle the constraints.

A popular method to address the problem is by incorporating constraints in data assimilation (Ko & Bitmead 2007; Lauvernet et al. 2009; Garcia-Fernandez et al. 2012; Xu et al. 2013). Ko & Bitmead (2007) proposed a constrained Kalman filter based on the projected system method. The method's superiority was demonstrated by comparing the magnitude of the estimation error covariance matrix with those of the unconstrained Kalman filters. Garcia-Fernandez et al. (2012) developed a method to solve the state estimation problem under bounded measurement noise. The boundary information of the measurement noise is used to modify the prior PDF of the state variable based on the Bayesian rule, while the Kalman filter is used to fuse the modified prior PDF with the likelihood. Simon & Simon (2010) described a PDF truncation method by incorporating constraints into the Kalman filter and applied this method to an aircraft turbofan engine health estimation problem. Andersson et al. (2019) developed a linear state estimation method with linear equality constraints for time-variant systems. Xu et al. (2013) constructed a linear equality constraint dynamic model by incorporating the constraints as the prior information about the states into the dynamics modeling. Overall, the above methods address the constraint problem by introducing equality or inequality constraints to data assimilation.

Another approach to address the problem is to modify the normal PDF, referred to as truncated normal PDF, to restrict the value range of state variables. The truncated normal PDF has many application scenarios in Bayesian inference for truncated parameter space problems. Robert (1995) proposed an efficient algorithm for unidimensional truncated normal variables and a multidimensional extension. Generally, the theoretical truncated normal PDF is equal to the normal distribution PDF in the feasible region, and is directly equal to 0 outside the feasible region (Burkardt 2014).

The theoretical truncated normal PDF has been widely used in sample-based state estimation problems (Zhou et al. 2018). However, it is rarely used in the analytical method since the non-differentiable truncation points lead to intricate numerical integration (Robert 1995). For the analytical method, due to the requirement of the linearization of the system, the function should be differentiable, and the first-order gradient (Jacobian matrix) or second-order gradient (Hessian matrix) information is utilized. The difficulty in applying the truncated normal PDF to the analytical method is the non-differentiability of a truncated normal PDF at the truncation points.

In a water distribution system (WDS) simulation, the hydraulic modeling faces the same difficulty in the state estimation using the truncated normal PDF. The hydraulic models have been widely implemented to address the system analysis, design, and operation of a WDS (Zeng et al. 2016; Oikonomou et al. 2018; Harmouche & Narasimhan 2020). For real-time control purposes, the model parameter such as the nodal water demand is a time-dependent state variable that should be estimated given real-time measurements, for which the analytical method is suggested for its superiority in terms of computational efficiency. When the PDF of nodal water demand is assumed as a standard normal PDF, the variable ranges from negative infinity to positive infinity (Shao et al. 2019). The probability of negative or excessive values is not equal to zero. Therefore, the search space ranges from negative infinity to positive infinity. In addition, when the measured values of some pressure sensors are inaccurate or have large uncertainty, the nodal water demands may adjust to negative or excessive values, which is an infeasible condition in practice. A possible solution to this problem is to introduce a truncated normal PDF. However, the PDF used in analytical methods must be differentiable, whereas the truncated normal PDF cannot meet this condition. As shown in Figure 1, the PDF value drops sharply to 0 on the truncation points, at which the PDF is non-differentiable. Currently, one strategy to solve this problem is to set all negative values to 0 when finishing all the nodal water demand estimations. Another strategy is reassigning the PDF in the infeasible region to the feasible region (Shimada et al. 1998; Simon & Simon 2010). Nevertheless, both strategies mentioned above are too inefficient and do not solve the fundamental problem of the non-differentiability of the truncation points.
Figure 1

The truncated normal PDF with different λ for conditions (). Please refer to the online version of this paper to see this figure in colour: https://dx.doi.org/10.2166/hydro.2023.250.

Figure 1

The truncated normal PDF with different λ for conditions (). Please refer to the online version of this paper to see this figure in colour: https://dx.doi.org/10.2166/hydro.2023.250.

Close modal

The primary aim of this study is to propose a truncated normal PDF to overcome the difficulty of non-differentiable at truncated points. Then, the truncated normal PDF is used to describe the prior PDF and likelihood based on the probability modeling of state variable and measurement noise in the truncated space. The prior PDF and likelihood are fused in the data assimilation framework, and analytical solutions for state estimation in a non-linear system are developed. Furthermore, we have applied the developed method to estimate the nodal water demand in a hypothetical WDS and indirectly through nodal pressure estimate in a field WDS. The results show that the method can deal with the state estimation problem under the condition that the state variable and measurement noise are distributed in the finite region. Moreover, the developed method can be effectively applied to estimate a wide range of parameters in WDS models, such as pipe roughness.

Modeling of truncated normal PDF

Two-side truncated normal PDF

The proposed normal PDF truncated to some interval has the following form:
formula
(1)
where is the random variable; and are the mean and standard deviation of the normal PDF; is a constant and ; C is the normalization constant, which ensures that Equation (1) yields a valid probability density and integrates to a unit one.
The shapes of the developed truncated normal PDF for different are shown in Figure 1, when . The PDF for the theoretical truncated normal PDF (black line) drops sharply to 0 at the truncation points (Figure 1), at which the PDF is non-differentiable. For a fully developed PDF, the probability density values transition to 0 continuously in close approximation to the theoretical truncated distribution (Figure 1). These properties are important for the analytical method in the state variable estimation:
formula
(2)

The proposed PDF (red line in Figure 1) has modeling errors in approximation compared to the theoretical truncated normal PDF (black line in Figure 1). Importantly, the magnitude of the modeling error is mainly controlled by the parameter . With a decreasing value, the proposed PDF better replicate the theoretical PDF. The smaller the , the smaller the error of the truncated normal PDF. Also notable is that the parameter λ affects the convergence of results in the state estimation for non-linear systems, for which the recommended range is .

One-side truncated normal PDF

Equation (1) can be easily applied to the one-side truncated normal PDF (:
formula
(3)
formula
(4)
The shapes of the developed one-side truncated normal PDF are shown in Figure 2.
Figure 2

One-side truncated normal PDF: .

Figure 2

One-side truncated normal PDF: .

Close modal

Data assimilation via truncated normal PDF

Probability formulation for data assimilation

Consider a non-linear dynamic system given by Equations (5) and (6):
formula
(5)
formula
(6)
where t is the time index; is the state variable to be estimated; is the prediction matrix and is the prediction error; is the measurement; is the non-linear function mapping to ; is the measurement noise; m and n are the dimensions of and . In the classical data assimilation algorithm, and are uncorrelated zero mean normal noise input sequences:
formula
(7)
where is the normal PDF; and are the covariance matrix for and , respectively.
Based on Equations (5)–(7), we can write that
formula
(8)
formula
(9)
where and
Equation (8) is the prior PDF for the state variable and Equation (9) is the likelihood. The data assimilation method aims to estimate by assimilating the prior and likelihood. Based on Bayesian theory, the posterior PDF for can be written as
formula
(10)
Then, the state variable is estimated by maximizing the posterior PDF:
formula
(11)

Equation (11) can be solved by a sampling-based method (Do et al. 2017; Zhou et al. 2018) or an analytical method (Shao et al. 2019; Singh et al. 2022). This paper focuses on the use of analytical methods to estimate the state variable.

Here we assume that: (1) the elements in are independent of each other, thus ; (2) the measurement noise from different sensors is independent of each other, thus . In this situation, and R are diagonal matrices. The prior PDF and likelihood can be written as Equations (12) and (14), respectively:
formula
(12)
formula
(13)
formula
(14)
formula
(15)
where , , , and are the element of , , , and , respectively; and are the diagonal value of and R, respectively.
By substituting Equations (12) and (14) into Equation (10), the posterior PDF has the following form:
formula
(16)
Equations (13) and (15) are normal PDFs, given the dynamic system described by Equations (5) and (6). In normal PDF, the theoretical values of random variables are distributed at infinite intervals (Tokmachev 2018). However, in the realistic problem, and are usually distributed in the finite region. This issue is addressed by the use of the proposed truncated normal PDF.

Truncated prior PDF and likelihood

Assuming that , the truncated normal prior PDF has the following form:
formula
(17)
Assuming that , we can conclude that . The likelihood has the following form:
formula
(18)
where and .

Substituting Equations (17) and (18) into Equation (16) yields the truncated posterior PDF. In the following section, an analytical method is adopted to estimate the state variable by maximizing the truncated posterior PDF refer to Equation (16).

Analytical solutions for non-linear system

The objective function

Maximizing the truncated posterior PDF , as shown in Equation (16), is equivalent to maximizing the logarithm of :
formula
(19)
Considering that
formula
formula
Equation (19) can be written as
formula
(20)

As shown in Equation (20), when the parameter or approaches the boundary from the inner of the feasible domain, e.g., [,] for and [,] for , the optimization objective function will increase sharply. Then the variables can be effectively constrained to the feasible domain by minimizing the objective function. This strategy is similar to the barrier method to solve non-linearly constrained optimization problems, in which is the barrier parameter affects the number of iterations and the convergence effect (Nocedal & Wright 2006).

Non-linear solution

Considering that is a non-linear function, Equation (20) is also non-linear, for which Newton's iteration method is adopted in this study to minimize Equation (20). The corrector computed by Newton's iteration method can be expressed as
formula
(21)
formula
(22)
where is the correction vector of the kth iteration; and are the first-order gradient and the second-order gradient of , respectively. and can be derived as follows:
formula
(23)
formula
(24)
where and , and . These vectors can be computed by
formula
(25)
formula
(26)
formula
(27)
Considering that , Equation (24) can be written as
formula
(28)
The state variable can be estimated iteratively by substituting Equations (23) and (28) to Equations (21) and (22). The covariance of the state variable, , can be computed by
formula
(29)
Equation (29) can be solved based on Woodbury-Matrix-Identity Formula. The steps of the iteration are shown in Figure 3 and in a flow chart in the Supplementary Material.
Figure 3

Computational procedure for parameter estimation using the truncated normal PDF.

Figure 3

Computational procedure for parameter estimation using the truncated normal PDF.

Close modal

The proposed method is applied to the nodal water demand estimation in two water distribution networks. The first one is a hypothetical simple network, and the other is a large-scale field network located in eastern China. Case study 1 is used to verify the proposed method with a hypothetical simple network. In Case study 2, the performance of the proposed method is evaluated when used to solve a real large-scale state estimation.

Case study 1: Simple hypothetical network

The simple hypothetical network consists of 8 nodes, 11 pipes, and 1 reservoir, with two pressure sensors and three flow sensors (Figure 4). We use two methods to estimate the nodal water demand. One is the method proposed in this study, and another is proposed by Shao et al. (2019). The main difference between the two methods is whether the truncated normal distribution is used. A 3 days simulation with a time step of 20 min is carried out.
Figure 4

Schematic diagram of the hypothetical network.

Figure 4

Schematic diagram of the hypothetical network.

Close modal

Random noise is added to the theoretical value to obtain the observed value and the noise is normally distributed with a variance of R. The variance for the pressure and flow sensors are and 1 , respectively. The prior nodal water demand is generally predicted from historical data (Chu et al. 2021a). In this case study, we assume that the mean value of prior nodal water demand is equal to the estimated nodal water demand in the previous time step (). In the first time-step, the total water demand is equally allocated to each node as the prior node water demand, with . The covariance () is assumed to be a constant, with . The truncation point for the prior PDF is computed as , where is the theoretical value of the nodal water demand. For the pressure sensor, the truncation points for the likelihood are determined by ; for the flow sensor, , where is the observed value of the sensor. The constant parameter = 1 and the maximum number of allowed iterations .

Nodal water demand estimation

Step 1: Set Estimation Parameters

The pressure at nodes 3 and 7 and the pipe flow at pipes [8], [10], and [11] are selected as the measured values. The observed value and their variance are ,,, and . The truncation points for the likelihood are ; For the prior PDF of nodal water demand , . The truncation points for the prior PDF are . . In each time step, the nodal water demand is estimated after serval iterations. The prior value of the nodal water demand is set as the value at the first iteration ()

Step 2: Compute the Model Outputs

The nodal water demand () at current iteration () is as the model input, then the model output is calculated as
formula

Step 3: Calculate the Jacobian Matrix

By substituting the WDS hydraulic model input, can be calculated as
formula
Step 4: Calculateand Hessian Matrix
formula
formula
Step 5: Calculate
formula
Step 6: Update the Nodal Water Demand for the Next Iteration
formula
If , then . If , then .
formula
formula

Step 7: Reach Termination Conditions

The next step is to repeat Steps 2–6 until . The comparison of the nodal water demands and nodal pressure between theoretical values and estimated results are discussed.

Case study 1: Discussion

The estimated nodal water demands are illustrated in Figure 5. It is evident that the error in nodal water demand estimated by Shao et al. (2019) is significantly higher as compared to the method proposed in this study. For example, Shao et al. (2019) estimated the nodal water demand at N4 with an error of 2.30 L/s, whereas the approach presented in this study achieved an error of only 0.24 L/s. The accuracy of the two algorithms is evaluated by calculating the average deviation (error) between the estimated values and the theoretical values. Similar observations can be found in N1, N2, and N5. It is apparent that the proposed method can effectively avoid unrealistic estimates of excessively large or small nodal water demands. This is primarily due to the use of the truncated normal PDF, which restricts the parameters to a reasonable range. As shown in Figure 5, the estimated nodal water demand is confined within the domain defined by the truncation points (gray dashed line). In contrast, Shao et al. (2019) used a normal PDF, which allows the nodal water demand to be estimated over the full range . This assumption is not consistent with the actual situation, leading to an excessive value of the nodal water demand at N1, N2, N4, and N5. More importantly, these unrealistic nodal demands without the use of the proposed truncated normal PDF cannot be easily uncovered by the use of limited pressure and flow measurements.
Figure 5

Estimated nodal water demand.

Figure 5

Estimated nodal water demand.

Close modal
As shown in Figure 6, the two models yield pressure and flow rates close to the measured values by the two pressure sensors and three flow meters. This disparity in results suggests that demand estimation with a limited number of field measurements is an ill-conditioning problem. The insufficient number of measurements leads to non-unique solutions in the search domain, thereby resulting in errors in the nodal water demand. For the most agreement with the observed value, the search space of the nodal water demand depends on the truncation of the PDF, which serves as a constraint in the modeling process. The utilization of a truncated normal PDF can significantly narrow down the search space for nodal water demand, thereby ensuring that they lie within a reasonable domain. By employing a truncated PDF, it is possible to transform the ill-conditioned problem into a solution with modeling constraints. The potential of this approach is analyzed in Case study 2.
Figure 6

Measured and estimated values of the extended period simulation.

Figure 6

Measured and estimated values of the extended period simulation.

Close modal

Case study 2: Large-scale city network

Case 2 involves a large-scale WDS located in a city in eastern China, as shown in Figure 7. The network contains three reservoirs, 4,242 nodes, 4,841 pipes, and 77 pressure sensors installed in the network. Among the 77 pressure sensors, 61 of them are used to estimate the nodal demand, while the remaining 16 sensors are compared with the model output value to validate the estimation accuracy. The pressure values of the sensors are field measurements, and the variance of sensor errors is assumed to be 1 (). The truncation points for the observed pressure are set as , where y denotes the field measurements. The total water demand is equally allocated to each node as the prior node water demand (). The prior variance is assumed to be . The truncation points for the prior PDF are set as . In this case study, only one time-step simulation is performed.
Figure 7

Schematic diagram of Case 2.

Figure 7

Schematic diagram of Case 2.

Close modal
The deviation between the measured pressure values and model output values is used to evaluate the estimation accuracy for the realistic problem. Figure 8(a) shows the comparisons of the measured pressure values and model simulation values for the 61 estimation sensors. For the proposed method, the deviations for 59 estimation sensors are within 1 m, and deviations for 2 estimation sensors are between 1 and 1.5 m. This indicates successful state estimation as the pressure residuals are lower than 2 m for all sensors, which is the criteria specified by the Water Research Centre (1989). The estimation accuracy is further validated by the 16 validation pressure sensors. As shown in Figure 8(b), the majority of the pressure residuals of the 16 validation sensors are within 1 m, with the largest residual being 1.53 m.
Figure 8

Comparisons of measured pressure values and model simulated pressure values: (a) 61 estimation sensors and (b) 16 validation sensors.

Figure 8

Comparisons of measured pressure values and model simulated pressure values: (a) 61 estimation sensors and (b) 16 validation sensors.

Close modal

In contrast, for the method proposed by Shao et al. (2019), the deviations for 54 estimation sensors are within 1 m, while the deviations for 7 sensors are above 1 m and the deviations for 3 sensors are above 2 m, with the largest residual being 3.68 m. In the validation data set, the deviations for 6 sensors are above 1 m, and the deviations for 3 sensors are above 2 m with the largest residual being 3.05 m. Moreover, Shao et al. (2019) estimated 191 negative nodal water demands, while the proposed method does not estimate any negative nodal water demand.

The results of Shao et al. (2019) show that three estimation sensors have deviations greater than 2.5 m, and three validation sensors also have deviations greater than 2 m, indicating excessive estimation errors (Figure 8). As shown in Figure 7, the distribution of sensor locations is highly uneven, with a high density of sensors in urban areas and a low density in rural areas. The sensors with excessive deviations are all located in rural areas with low sensor density. As mentioned earlier, demand estimation with a limited number of field measurements is an ill-conditioned problem. The problem is even more severe in locations with sparse sensor density, which leads to greater estimation errors for these sensors. Additionally, in the Bayesian estimation process, there is a competitive relationship between the sensors, and the estimated demand is a compromise between them. Sensors that are closer together tend to adjust the nodal water demand in a similar direction, making them more competitive. Thus, in areas with a higher sensor density, competitiveness increases, while in areas with a lower sensor density, competitiveness decreases, exacerbating the severity of the ill-conditioning problem in areas with sparse sensors and leading to excessive deviations. The use of truncated likelihood constrains the simulated value to a set range () around the observed value (), solving the ill-conditioning problem by incorporating constraints implied in the truncated likelihood.

The accurate estimate of the nodal water demand is crucial for modeling and managing water distribution networks. Recent advances have made it possible to estimate all nodal water demand in real-time using ubiquitous pressure sensors and limited flow rate measurements (Shao et al. 2019). The nodal demand estimation is often based on the assumed normal PDF for probability modeling in Bayesian inference, allowing the values of random variables to be distributed at infinite intervals. However, engineering practice and field operations commonly confine the state variables and measurement noises in a limited range. This imprecise mathematical representation can lead to an unrealistic estimation of nodal pressure and water demands.

This paper proposed a new nodal water demand estimation approach using truncated normal PDF methods. According to the application results and analysis, the main conclusions can be summarized as follows:

  • (a)

    Existing truncated normal PDF in simulation suffers from the difficulty at the non-differentiable truncated point, a problem that seriously limits the use of analytical methods (e.g., Gaussian approximation). The proposed analytical solutions for truncated normal PDFs method avoid this non-differentiable problem by mathematical approximation in truncated points.

  • (b)

    When using a limited number of sensors to estimate or calibrate network parameters, an ill-conditioning problem arises, making it susceptible to overfitting the noise in the data. However, the proposed method offers a solution to this issue by effectively constraining the fitting range of the noise. This limitation helps to prevent overfitting and ensures that the estimated parameters are more robust and reliable. By effectively managing the noise during the estimation or calibration process, the proposed method mitigates the ill-conditioning problem and improves the accuracy of the parameter estimates.

  • (c)

    When the measurements are biased, the estimated parameters, such as nodal water demand, can exhibit undesirable characteristics, such as negative values or excessively large values. However, the proposed truncated normal PDFs in this paper effectively mitigate this issue. By utilizing truncated normal PDFs, the parameter estimation process is constrained within a specific range, preventing the estimation of unrealistic or extreme values. This approach ensures that the estimated parameters remain within reasonable bounds and avoids the occurrence of negative or excessively large values, enhancing the accuracy and reliability of the estimation results.

  • (d)

    In data assimilation, the truncated normal PDF from the theoretical truncated normal PDF is primarily controlled by the parameters . As a result, the parameter affects the number of iterations and the convergence effect in data assimilation. A fixed value of is used in this study, which may in some cases increase the number of iterations and reduce the computational efficiency. Considering that the objective function (refer to Equation (20)) is similar to the barrier method when dealing with non-linearly constrained optimization problems, updating the parameter in the presence of non-linearities (Nocedal & Wright 2006), can be adopted to improve the algorithm performance in future research.

The proposed method using the truncated normal PDF in the finite region has been applied to simulate nodal pressure and estimate corresponding water demand in a hypothetical simple and a field WDS in eastern China. The results clearly show the advantage of the proposed method in avoiding the artificial negative nodal water demand and unseasonable errors between the estimated and measured values. This improvement is constructive toward WDS simulation and control using real-time network monitoring data. Furthermore, the proposed method can also be utilized to estimate other hydraulic parameters in WDS models. This includes parameters like pipe roughness, which can be accurately represented by a truncated normal PDF.

This work was supported by the National Natural Science Foundation of China (No. 52270095) and the National Natural Science Foundation of China (No. 52200119). As a part of the collaborative research, this publication has been cleared through the U.S. EPA administrative and technical review process. The views expressed in this article are those of the authors and do not necessarily represent the views or the policies of the Agency; therefore, no official endorsement should be inferred.

All relevant data are included in the paper or its Supplementary Information.

The authors declare there is no conflict.

Andersson
L. E.
,
Imsland
L.
,
Brekke
E. F.
&
Scibilia
F.
2019
On Kalman filtering with linear state equality constraints
.
Automatica
101
,
467
470
.
Bar-shalom
Y.
,
Li
X. R.
&
Kirubarajan
T.
2001
Estimation with Applications to Tracking and Navigation
.
A Wiley-Interscience Publication
,
New York
.
Beckers
F.
,
Heredia
A.
,
Noack
M.
,
Nowak
W.
,
Wieprecht
S.
&
Oladyshkin
S.
2020
Bayesian calibration and validation of a large-scale and time-demanding sediment transport model
.
Water Resources Research
56
,
7
.
Bishop
C. M.
2006
Pattern Recognition and Machine Learning
.
Springer
,
New York
.
Burkardt
J.
2014
The Truncated Normal Distribution
. .
Chu
S.
,
Zhang
T.
,
Yu
T.
,
Wang
Q. J.
&
Shao
Y.
2021a
A noise adaptive approach for nodal water demand estimation in water distribution systems
.
Water Research (Oxford)
192
,
116837
.
Chu
S. P.
,
Zhang
T. Q.
,
Xu
C. N.
,
Yu
T. C.
&
Shao
Y.
2021b
Dealing with data missing and outlier to calibrate nodal water demands in water distribution systems
.
Water Resources Management
35
(
9
),
2863
2878
.
Do
N. C.
,
Simpson
A. R.
,
Deuerlein
J. W.
&
Piller
O.
2017
Particle filter-based model for online estimation of demand multipliers in water distribution systems under uncertainty
.
Journal of Water Resources Planning and Management
143
(
11
),
04017065
.
Garcia-Fernandez
A. F.
,
Morelande
M. R.
&
Grajal
J.
2012
Truncated unscented Kalman filtering
.
IEEE Transactions on Signal Processing
60
(
7
),
3372
3386
.
Harmouche
J.
&
Narasimhan
S.
2020
Long-term monitoring for leaks in water distribution networks using association rules mining
.
IEEE Transactions on Industrial Informatics
16
(
1
),
258
266
.
Huang
D. Z.
,
Schneider
T.
&
Stuart
A. M.
2022
Iterated Kalman methodology for inverse problems
.
Journal of Computational Physics
463
,
111262
.
Hutton
C. J.
,
Kapelan
Z.
,
Vamvakeridou-Lyroudia
L.
&
Savić
D. A.
2014
Dealing with uncertainty in water distribution system models: a framework for real-time modeling and data assimilation
.
Journal of Water Resources Planning and Management
140
(
2
),
169
183
.
Lauvernet
C.
,
Brankart
J.-M.
,
Castruccio
F.
,
Broquet
G.
,
Brasseur
P.
&
Verron
J.
2009
A truncated Gaussian filter for data assimilation with inequality constraints: application to the hydrostatic stability condition in ocean models
.
Ocean Modelling
27
(
1–2
),
1
17
.
Law
K.
,
Stuart
A.
&
Zygalakis
K.
2015
Data Assimilation: A Mathematical Introduction
.
Springer International Publishing AG
,
Cham
.
Nocedal
J.
&
Wright
S. J.
2006
Numerical Optimization
.
Springer
,
New York
.
Oikonomou
K.
,
Parvania
M.
&
Khatami
R.
2018
Optimal demand response scheduling for water distribution systems
.
IEEE Transactions on Industrial Informatics
14
(
11
),
5112
5122
.
Robert
C. P.
1995
Simulation of truncated normal variables
.
Statistics and Computing
5
(
2
),
121
125
.
Savic
D. A.
,
Kapelan
Z. S.
&
Jonkergouw
P. M. R.
2009
Quo vadis water distribution model calibration?
Urban Water Journal
6
(
1
),
3
22
.
Scott
S. W.
,
O'Sullivan
J. P.
,
Maclaren
O. J.
,
Nicholson
R.
,
Covell
C.
,
Newson
J.
&
Gudjonsdottir
M. S.
2022
Bayesian calibration of a natural state geothermal reservoir model, Krafla, North Iceland
.
Water Resources Research
58
(
2
)
e2021WR031254
.
Shao
Y.
,
Chu
S.
,
Zhang
T.
,
Yang
Y. J.
&
Yu
T.
2019
Real-time water distribution system hydraulic modeling using prior demand information by formal Bayesian approach
.
Journal of Water Resources Planning and Management
145
(
12
),
04019059
.
Shimada
N.
,
Shirai
Y.
,
Kuno
Y.
&
Miura
J.
1998
Hand Gesture Estimation and Model Refinement Using Monocular Camera-Ambiguity Limitation by Inequality Constraints
.
IEEE
,
Los Alamitos
, pp.
268
273
.
Simon
D.
&
Simon
D. L.
2010
Constrained Kalman filtering via density function truncation for turbofan engine health estimation
.
International Journal of Systems Science
41
(
2
),
159
171
.
Singh
H.
,
Chattopadhyay
A.
&
Mishra
K. V.
2022
Inverse Extended Kalman Filter – Part I: Fundamentals
.
Cornell University Library, arXiv.org
,
Ithaca
.
Tokmachev
M. S.
2018
Modeling of truncated probability distributions
.
IOP Conference Series: Materials Science and Engineering
441
,
012056
.
Vrugt
J. A.
,
Diks
C. G. H.
,
Gupta
H. V.
,
Bouten
W.
&
Verstraten
J. M.
2005
Improved treatment of uncertainty in hydrologic modeling: combining the strengths of global optimization and data assimilation
.
Water Resources Research
41
(
1
),
W01017
.
WRC (Water Research Centre)
1989
Network Analysis – A Code of Practice
.
Water Research Centre
,
Swindon
,
UK
.
Xu
L.
,
Li
X. R.
,
Duan
Z.
&
Lan
J.
2013
Modeling and state estimation for dynamic systems with linear equality constraints
.
IEEE Transactions on Signal Processing
61
(
11
),
2927
2939
.
Yang
C.
,
Zheng
J.
,
Ren
X.
,
Yang
W.
,
Shi
H.
&
Shi
L.
2018
Multi-sensor Kalman filtering with intermittent measurements
.
IEEE Transactions on Automatic Control
63
(
3
),
797
804
.
Yoon
H. N.
,
Marshall
L.
,
Sharma
A.
&
Kim
S.
2022
Bayesian model calibration using surrogate streamflow in ungauged catchments
.
Water Resources Research
58
(
1
),
e2021WR031287
.
Zeng
D. Z.
,
Gu
L.
,
Lian
L.
,
Guo
S.
,
Yao
H.
&
Hu
J. K.
2016
On cost-efficient sensor placement for contaminant detection in water distribution systems
.
IEEE Transactions on Industrial Informatics
12
(
6
),
2177
2185
.
Zhou
X.
,
Xu
W. R.
,
Xin
K. L.
,
Yan
H. X.
&
Tao
T.
2018
Self-adaptive calibration of real-time demand and roughness of water distribution systems
.
Water Resources Research
54
(
8
),
5536
5550
.
Zhou
J. H.
,
Wu
Z. Y.
,
Crow
W. T.
,
Dong
J. Z.
&
He
H.
2020
Improving spatial patterns prior to land surface data assimilation via model calibration using SMAP surface soil moisture data
.
Water Resources Research
56
(
10
),
e2020WR027770
.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY-NC-ND 4.0), which permits copying and redistribution for non-commercial purposes with no derivatives, provided the original work is properly cited (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Supplementary data