The numerical and analytical models used for transient simulations, and hence for the pressurized pipe system diagnosis, require the definition of a rheological component related to the pipe material. The introduction and the following widespread use of polymeric material pipes, characterized by a viscoelastic behavior, increased the complexity and the number of parameters involved in this component with respect to metallic materials. Furthermore, since tests on specimens are not reliable, a calibration procedure based on transient test is required to estimate the viscoelastic parameters. In this paper, the trade-off between viscoelastic component accuracy and simplicity is explored, based on the Akaike criterion. Several aspects of the calibration procedure are also examined, such as the use of a frequency domain numerical model and of different standard optimization algorithms. The procedure is tested on synthetic data and then it is applied to experimental data, acquired during transients on a high density polyethylene pipe. The results show that the best model among those used for the considered system implements the series of a spring with three Kelvin–Voigt elements.