Laboratory investigations on the quality of leak tests and visual inspections of wastewater connection pipes carried out by specialist contractors

In addition to stability and operational safety, tightness is the permanent functional objective of wastewater pipes. Therefore, according to § 61 ‘Self-monitoring for wastewater discharges and wastewater systems’ of the German Water Resources Act (WHG 2009), the operators of wastewater systems are obliged, among other things, to monitor the condition and functionality of their systems themselves. If the pipes are not monitored, any damage may remain undetected and groundwater may enter the sewer system (infiltration) or wastewater may escape (exfiltration), with consequences for the environment. Various test methods are available for checking the tightness of connection pipes: The leak test with water or air according to DIN 1986-30 (2012) or DWA-M 149-6 (2016) for existing pipes and the visual inspection according to DIN 1986-30 (2012). The leak tests are carried out by expert testers and the test is assessed as ‘passed’ or ‘Not passed’ with regard to the criteria ‘leak tightness’.

In Bosseler et al. (2018), the leak tests and visual inspections carried out on rehabilitated wastewater pipes showed results worthy of discussion. On the one hand, despite passing the leak test, leaky areas could certainly be detected during the subsequent visual inspection using camera technology under external water pressure. On the other hand, pipes that exceeded the permissible water addition by a multiple did not show any obvious infiltrations during the visual inspection under external water pressure. Thus, the quality of the leak tests and visual inspection in these cases is more than questionable.

The fact that faulty test results can occur when carrying out tests was also confirmed by expert interviews with sewer network operators, test equipment manufacturers, pipe manufacturers, construction companies and training institutions for obtaining the expertise for condition and function tests (‘expert tester’) (Ulutaş 2017). Accordingly, ‘tight pipes’ can be tested as ‘not passed’ and ‘leaky pipes’ as ‘passed’.

The topics of ‘leak testing’ and ‘visual inspection’ in drainage networks have already been addressed in numerous research projects and scientific papers. Schwebel (1989) developed an air pressure test method for checking the tightness of wastewater pipes. Proof of the correlation between the water pressure and air pressure test was provided by Kaufmann (1997). Künster (2002) developed test criteria for leak testing of individual pipe joints in sewers that cannot be walked inside. In addition, the organisational implementation of leak tests for wastewater pipes was investigated (Pinnekamp et al. 2005). An overview of the methods available on the market for condition assessment and leakage testing of house connection pipes and land pipes is given in Bosseler et al. (2003). Uncertainties due to subjective assessment of an inspector in condition detection (damage detection) on wastewater pipes by means of visual inspections were investigated by Dirksen et al. (2011), Sousa et al. (2014) and Caradot et al. (2017). Furthermore, Roghani et al. (2019) determined the influence of uncertainties in the condition assessment with a prediction model. Van der Steen et al. (2013) investigated the influence of the coding system on the quality of the inspection data.

However, there are currently no well-founded scientific studies on the quality of leak tests and visual inspections to determine the tightness of wastewater pipes. In addition, there is a lack of reliable data on possible test errors (error sources and error rates) and their effects on the test result as well as on the accuracy of the different leak test methods. There are also no reliable findings on the error rates that occur during visual inspections with regard to damage detection and naming as well as damage classification.

The aim of the study summarised below was to determine the quality of leak testing procedures (air overpressure, air underpressure and water pressure) and visual inspection of connecting pipes carried out by specialist contractors.

For the leak test procedures, it was investigated which factors (tester, equipment, test procedure) can influence the test result. In order to identify the influence and their effects, tests by 29 expert testers with their own equipment were accompanied on a transparent laboratory test track on a 1:1 scale with adjustable and quantifiable leakage opening. In this way, test errors (error sources and error rates) and their effects on the test result could be observed and evaluated. In addition, the accuracy (trueness and precision) of the different leak test procedures was identified in order to determine the quality of the test procedures on this basis.

The examinations for the visual inspection were carried out on a connection pipe with four major damages (damage class A) according to the North Rhine-Westphalian image reference catalogue (2014) by expert testers with their own equipment. Subsequently, the extent to which the expert testers fulfilled the requirements with regard to damage detection and naming as well as damage classification was evaluated. In addition, the error rate in the performance of the visual inspection was determined.

The tests for the leak test methods (air overpressure, air underpressure and water pressure) took place in a test hall on an area of approximately 8 m length and 6 m width. On this area, three identical pipe strings DN 150 (L=7.50 m) with two 45° bends and a branch (downpipe) DN 100 (L=3 m) were set up next to each other with a gradient of approximately 1.5% (1.5 cm/m) at a height of 60–70 cm (Figure 1). The downpipe is approximately 3 m long and was laid with a vertical 90° bend. The pipes were placed on Euro pallets and polystyrene plates and secured against shifting.

Two connection lines were constructed from transparent PVC glass (manufacturer: Simona AG) (Figure 1 below, pipes number 1 and 2) in order to visually obtain findings regarding defective shut-off elements, air inclusions during the water pressure test, and so on. A pressure measuring sensor (manufacturer: Sensortechnics GmbH, type: CTE7N01GY6) and a temperature sensor (manufacturer: B+B Thermo-Technik GmbH, type: Pt 1000) were attached to each of these lines, as the physical leak tests were carried out on these lines. The control measurement sensor and the temperature sensor were connected to a notebook (software: LabVIEW) and the time recorded in combination with the pressure as well as the temperature in the connection pipe during the tests. Also the room temperature in the tent was measured and documented.

In addition, a ball valve with a brass cap with a bore of 0.8 mm was installed for each pipe to simulate a leakage. With this procedure, it was possible to simulate the pipe conditions ‘Not Passed/ Leaky’ and ‘Passed/ Tight’ by opening and closing the ball valve.

For the selection of a representative size, preliminary tests were first carried out with different leakage sizes (0.5 mm, 0.6 mm, 0.65 mm, 0.8 mm and 1.0 mm) (Hermeler 2018). The aim was to determine a leakage size with which the pipe condition ‘Not passed/ Leaky’ (leaky test result) can be reliably achieved with all leak test methods. In addition, the same leakage size was to be selected for all test methods in order to be able to compare the results of the different test methods with each other. The results of the preliminary tests showed that only with the 0.8 mm leakage size with all three test methods (air overpressure, air underpressure and water pressure), was the respective permissible limit value exceeded.

The two connection pipes made of PVC glass (Figure 1 below, pipes number 1 and 2) were set up in a tent so that the testers could not visually assess the respective pipe conditions. Only the pipe ends were outside the tent to ensure access to the connection pipes.

The third connection pipe installed was made from opaque PVC pipes (Figure 1 bottom, pipe number 3) as in practice. Information required to carry out the leak tests, such as pipe course and length, was determined by visual inspection. The data was then available, for example, for calculating the permissible water addition for the water pressure test and for positioning the shut-off elements. Accordingly, no physical tests were carried out on this connection pipe. This pipeline was covered with a plastic tarpaulin during the inspection.

On the shown model construction of the PVC glass pipes DN 150 and DN 100 with an adjustable and quantifiable leakage opening (Figure 1 bottom, pipes number 1 and 2), tests were carried out taking into account the test specifications for the leakage test procedures by expert testers with their own test equipment.

The central questions in the investigations were which errors (error sources and error rates) are observed during the test and what influence the test errors have on the test result. In addition, the accuracy (trueness and precision) of the different test methods (air overpressure, air underpressure and water pressure) was investigated under ideal conditions.

To answer the question regarding test failures, 29 expert testers completed an inspection programme with a total of 140 individual tests, in which leak tests were carried out with air overpressure, air underpressure and water pressure for the pipe conditions ‘Not passed/ Leaky’ and ‘Passed/ Tight’ (Figure 2).

In North Rhine-Westphalia (a German state), leak tests and visual inspections of private wastewater pipes may only be carried out by expert testers in order to fulfil the statutory self-monitoring obligations. The list of all expert testers is shown on the website www.sadipa.it.nrw.de/Sadipa/. A total of approximately 1,100 expert testers are listed, all of whom have completed a training course with examination in accordance with the state requirements in order to obtain the expertise for leak tests and visual inspections. The 29 expert testers were selected from this list according to the following criteria:

• 10 expert testers: Criteria ‘performance expectation’ and ‘national market significance’

• 10 expert testers: Criteria ‘local presence’ and ‘positive experience’

• 9 expert testers: Criteria ‘random principle (by lot)’

The tests were accompanied throughout the entire process and the test performance of the expert testers was documented. During the execution of the test work by the expert testers, attention was paid to deviations from the implementation specifications in the test specifications of the corresponding technical standards and regulations. In addition, the test parameters determined by the expert testers could be compared with the actual physical values by means of parallel measurements with calibrated test equipment (control measurement). In this way, it is also possible to detect any anomalies in the measuring sensors of the expert testers' equipment by means of these parallel measurements.

In order to answer the question regarding the accuracy of the test procedures, the test results had to be as error-free as possible. For this reason, parallel measured data (control measurement) were used, as these measured values are more accurate than those obtained by the expert testers. In addition, the measured values were standardised and evaluated in order to derive comparative statements about the accuracy (trueness and precision) of the individual procedures.

Investigations on the visual inspections took place in a test hall with an area of approximately 6.0 m length and approximately 2.0 m width. Within the scope of this study, a DN 150 pipe string (L = approximately 3.50 m) made of the pipe material clay was constructed on this area and connected to a DN 1000 concrete manhole and a DN 300 concrete main sewer (Figure 3, lower pipe). Four large damage patterns (damage class A) according to the North Rhine-Westphalian image reference catalogue (2014) were attached to this connection pipe made of clay pipe material (Figure 4). This pipe was covered with a plastic tarpaulin, so that the damage patterns were not visually recognisable for the expert testers, as in practice.

The task for the expert testers was to inspect the connecting pipeline using camera technology without groundwater influence and to document the results with regard to damage detection and naming as well as damage classification according to the North Rhine-Westphalian image reference catalogue (2014). The inspection results of the expert testers were compared with the correct inspection results and evaluated. Subsequently, the error rate with regard to damage detection and naming as well as damage classification of the four existing damages was determined in order to determine the quality of the visual inspection. When determining the error rate, incorrect damage detection in an intact pipe was also taken into account.

In the following, the results for fault analysis in the leak test procedures (air overpressure, air underpressure and water pressure) are presented according to the two pipe conditions ‘Not passed/ Leaky’ and ‘Passed/ Tight’.

In the air overpressure test for the pipe condition ‘Not passed/ Leaky’, 26 of the 29 tests could be evaluated. No faults were observed in 16 of 26 tests (62%). In contrast, the following faults were found in 10 tests (38%):

• 7× Sensor abnormality detected (equipment)

• 2× Test time not held (tester)

• 1× Test pressure not held (tester)

Thus, 70% of the observed errors were caused by the equipment and 30% by the examiner. Three tests (tester no. 1, 19 and 29) could not be evaluated. In tests no. 1 and 29, there was a fault in the test set-up (blockage of the leakage and thus restricted air flow). In the case of tester no. 19, no comparable conditions were present, as the test was automatically terminated prematurely by software after exceeding the permissible limit value. As a result, the permissible pressure drop was exceeded in all 26 assessable tests, so that even with test errors, the correct test result ‘Not passed’ was determined for all observations (Figure 5).

In the air underpressure test for the pipe condition ‘Not passed/ Leaky’, 14 of the 15 tests could be evaluated. No faults were observed in 10 of 14 tests (71%). In contrast, the following 5 faults (1 test with two faults, tester no. 21) were found in 4 tests (29%):

• 2x Sensor abnormality detected (equipment)

• 2x Test pressure not held (tester)

• 1x Test time not held (tester)

Thus, 60% of the observed errors were caused by the tester and 40% by the equipment. One test (tester no. 19) could not be evaluated because of no comparable conditions (test ended automatically prematurely). The air underpressure tests could only be offered by 15 out of 29 requested expert testers, as this test is comparatively rarely carried out in practice. As a result, the permissible pressure increase was exceeded in all 14 tests that could be evaluated, so that even with test errors, the correct test result ‘Not passed’ was determined for all observations (Figure 6).

In the water pressure test for the pipe condition ‘Not passed/ Leaky’, 26 of the 28 tests could be evaluated. Test errors were observed in 26 tests (100%). In 12 tests, more than one error was identified during the test. The test errors observed are shown below:

• 24x Test pressure not held (tester)

• 10x Incorrect determination of the water addition value (tester)

• 4x Test time not held (tester)

• 1x Leak in the test system (running behind the shut-off element) (tester)

Thus, 100% of the observed errors were caused by the examiner. Two tests (testers no. 1 and 19) could not be evaluated. In test no. 1 there was an error in the test set-up (simulated leakage closed too early) and in test no. 19 there were no comparable conditions (test ended automatically prematurely). As a result, in 22 of 26 assessable tests the water addition value was higher than the permissible water addition value, so that the pipe was correctly tested as ‘Not passed’. In contrast, in four tests (testers 13, 18, 20, 29, Figure 7) the permissible water addition value was not exceeded, so that the leaking pipe was assessed as ‘passed’. In the overall view, the correct test result ‘Not passed’ was determined, albeit with test errors, at 85% (Figure 7).

For the pipe condition ‘Passed/ Tight’, only test errors were observed for the test methods (air overpressure, air underpressure and water pressure) that were already known from the tests for the pipe condition ‘Not passed/ Leaky’. A detailed description is not provided here. In the air tests, no significant deviations from the ‘Not passed/ Leaky’ pipe condition were discernible with regard to the error rates; that is, all pipes were correctly classified here as well. However, better results were achieved in the water pressure tests; all test results were correctly determined; that is, the ‘Tight pipe’ was assessed as ‘Passed’ (in contrast, 15% error rate for pipe condition ‘Not passed/ Leaky’). In the overall view, the permissible limit value was undercut in all assessable tests, so that even with test errors, the correct test result ‘Passed’ was determined for all observations.

In summary, the investigations into the fault analysis for the leak tests showed that various faults were observable for all test methods (air overpressure, air underpressure and water pressure) for the two pipe conditions ‘Not passed/ Leaky’ and ‘Passed/ Tight’. In the air pressure tests (overpressure and underpressure), the test time and the test pressure were not held by the expert testers. Furthermore, measuring sensor anomalies were found (Figures 5 and 6). In the water pressure test, in addition to non-compliance with the test time and test pressure, the water addition value was also incorrectly determined and a leak in the test system (running behind the shut-off element) was not noticed by the expert testers. However, the observed test errors only led to an incorrect test result four times in the water pressure tests for the pipe condition ‘Not passed/ Leaky’ due to serious execution errors by the expert testers (testers 13, 18, 20, 29, Figure 7). In the overall analysis, it could be determined that despite the test errors, the respective correct test result ‘Passed’ or ‘Not passed’ was determined by the expert testers in approximately 97% (136 of 140 individual tests) for the respective pipe conditions.

After the error analysis, the accuracy (precision and trueness) of the different leak test methods was determined. The contents of the methodology for determining accuracy are described in detail in DIN ISO 5725 (1997), DIN 1319-2 (2005) and Kromidas (2016). This methodology was transferred to the present question of an investigation of the quality of leak test procedures.

In order to be able to compare the different leak test methods with each other, the measured values were standardised. Subsequently, the precision and trueness were calculated in order to be able to make a statement about the accuracy of the leak test methods. To determine the relative precision, the coefficient of variation, the normed standard deviation, was identified and on this basis the relative precision for the individual methods was determined. With relative precision, the smaller the value or the shorter the arrow (blue dashed line), the more precise the test procedure (Figure 8). The trueness is determined by finding the distance from the assigned value (green square) to the mean values of the expert values (black circle). For relative accuracy, the smaller the value or the shorter the arrow (red), the more accurate the test procedure (Figure 8).

The investigations on the different leak test methods (Figure 8) showed that the air underpressure test provides the most precise result with 3.2%, followed by the air overpressure test with 3.9%. In contrast, the water pressure test was found to have a relative precision of 8.3%. In terms of trueness, it could be seen that the best result was achieved with the water pressure test with 0.1%. A trueness of 2.5% was determined for the air overpressure test and 3.4% for the air underpressure test.

For leak test methods, there is no definition of the precision and accuracy that is still acceptable. Various limits of up to 15% (Kromidas 2011) or even 16.5% (Eurostat 2013) are discussed for different areas of application. In this case, the value for environmental analysis of 15% is used. Accordingly, a method (test procedure) is generally classified as sufficient if the determined value is a maximum of 15%. As a result, the values determined for all three methods were below 15% in terms of both precision and accuracy. Based on the test results, all leak test methods are therefore assessed as sufficiently accurate.

In addition to the leak test procedures, the test procedure ‘visual inspection’ was also examined with regard to damage detection and naming, damage classification and incorrect damage detection in an intact pipe. In the visual inspection, 27 of the 29 tests could be evaluated.

The results of the damage detection and naming of the four major damages (damage class A) show that 102 of 108 (rate 94%) results are correct. In one of the deficient results, the damage was not detected and in the remaining five deficient results, the damage was detected but incorrectly named. Overall, it could be determined that the damage detection and naming was essentially carried out sufficiently reliably by the expert testers. In comparison, considerable deficits were revealed in the classification of the damage, as only approximately 42% of the classification results were correct. In addition, 10 out of 27 expert testers incorrectly assigned damage to the intact pipe area. In the overall view of all results, there is an overall rate of 67% for correct results.

A method (test procedure) can be classified as reliable if its reproducibility is at least 75% (Schenker-Wicki 1999). The visual inspection is thus assessed as not sufficiently reliable for the present application in order to reliably fulfil all normative requirements with regard to damage detection and naming as well as damage classification.

Dirksen et al. (2011), Sousa, and Caradot et al. (2017) also conducted studies on visual inspection. As a result, error rates of up to 25% were found. Dirksen et al. (2011) analysed the accuracy and reliability of visual inspection in detecting damage in sewer mains. Sousa et al. (2014) investigated the inspector's uncertainty by comparing the periodic inspection reports of three main sewers. Caradot et al. (2017) have investigated the extent to which the actual condition of a pipe is underestimated, overestimated or accurately estimated during a visual inspection. In the present study, damage detection and naming as well as damage classification were investigated. Accordingly, the studies have different objectives.

Within the scope of the present study, the quality of the different test methods for leak testing and visual inspection of connection pipes was determined for the first time on the basis of empirical investigations. The leak tests and visual inspections were carried out on two different test facilities on a 1:1 scale under laboratory conditions. The results were descriptively assessed, analysed and evaluated using statistical methods.

The results of the error analysis showed that various errors were observed in all leak test procedures (air overpressure, air underpressure and water pressure) for the two pipe conditions ‘Not passed/ Leaky’ and ‘Passed/ Tight’. However, the observed test errors only led to a wrong test result four times in the water pressure tests for the pipe condition ‘Not passed/ Leaky’ due to serious execution errors by the expert testers. In the overall analysis, it could be determined that despite the test errors, the respective correct test result ‘Passed’ or ‘Not passed’ was determined by the expert testers for the respective pipe conditions by approximately 97%.

When investigating the accuracy (precision and trueness) of the different leak test methods, it was evident that the air underpressure test provided the most precise result with 3.2%, followed by the air overpressure test with 3.9%. In contrast, a relative precision of 8.3% was determined for the water pressure test. In terms of trueness, it could be seen that the best result was achieved with the water pressure test with 0.1%. A trueness of 2.5% was determined for the air overpressure test and 3.4% for the air underpressure test. Based on the test results obtained, the leak test methods (air overpressure, air underpressure and water pressure) are methodically assessed as sufficiently accurate, as the individual values for precision and accuracy were each below the threshold value of 15%.

In addition to the leak test procedures, the quality of the visual inspection was also analysed. It was found that the visual inspection is not sufficiently reliable to fulfil all normative requirements regarding damage detection and naming as well as damage classification.

In order to be able to increase the quality of the test methods, there is still a need for research. The tests with a leakage size of 0.8 mm have shown that sufficiently accurate results can be achieved. However, the test method does not seem to be sufficiently accurate for leakages lying in the borderline range (<0.8 mm). Assuming that the assigned laboratory value is in the range of the respective limit value, a test result of ‘Pass’ seems to be possible despite a leaking pipe.

The client for the research project was the municipal drainage company Herne. The research project was funded by the Ministry of the Environment, Agriculture, Nature Conservation and Consumer Protection of the State of North Rhine-Westphalia (MULNV NRW) and by a group of network operators. The authors would like to thank all institutions involved in the research project.

All relevant data are included in the paper or its Supplementary Information.