Performance assessment of the operational management of urban drainage systems: a case study of Bejaia City, Algeria

The main function of the operation of Urban Drainage Systems (UDS) is to ensure the integrity and the capacity of pipe materials to support dead and live loads. It is supposed to operate properly during its service life (Ramos-Salgado et al. 2022). However, the UDS inevitably ages and degrades. In fact, managers of the Algerian National Sanitation Office (ONA) have been worrying about the degraded state of their UDS (Igroufa et al. 2020). This infrastructure failure is essentially due to the absence of adequate structured methodology of maintenance (Benzerra et al. 2012; Bedjou et al. 2019). In addition, the installation of appropriate management for the proper UDS operation has been constrained by several aspects such as lack of human and financial resources, lack of motivation and awareness, and limited data collection capacity. Consequently, ONA managers have been managing their assets without estimating quantitatively their performance gaps. As a result, ONA found it very interesting to get a tool that allows them to assess their management of UDS operation.

Performance Indicators (PI) constitute one of the most useful tools for quantifying the efficiency and the effectiveness of actions, and for supporting the monitoring of services over time (Santos et al. 2019). This tool has been successfully applied in the water field, mainly for water supply and wastewater services. In fact, several studies have been published in the field of water service management using performance indicators (Matos et al. 2003; Haider et al. 2015; Silva et al. 2016; Jensen & Wu 2018; Nam et al. 2019). Those authors have developed the PI in several aspects, such as: water resource management, operational and environmental practices, personal training, physical infrastructure, customer service, water quality, public health, and financial issues. Its application would be beneficial in terms of: collection of information, detection of problems, definition of priority objectives, and support of the decision-making process. This tool should be adapted to local specificities and should take into account the available information.

Recently, several authors have applied the assessment framework using a protocol based on PI. Examples are Collivignarelli et al. (2021) and Santos et al. (2022), who aimed to evaluate the wastewater service. Collivignarelli et al. (2021) used and calculated performance indices to analyse the performance of sewer systems and wastewater treatment plants. The tool allows to report the factual state of the system in order to improve the quality of services offered to the public. Santos et al. (2022) built a system of objectives, criteria, and performance metrics for two Portuguese water utilities. The main purpose is to identify the main strengths and weaknesses of the system's functioning, and to support the decision-making. Another research was carried out by Vanegas et al. (2022), who used two performance indicators to evaluate the deterioration model of sediment deposit. The aim is to prioritise inspections of sewer pipes based on self-cleansing criteria.

Within the Algerian context, the research study conducted by Benzerra et al. (2012) was the first that introduced and implemented the notion of PI for UDS. In fact, a hierarchical structure of objectives, criteria and PI was developed to measure the sustainability of Algerian urban drainage systems. Recently, Bedjou et al. (2019) proposed two operating indicators to evaluate the performance of ONA maintenance practices. The PI were calculated and then compared between two Algerian cities during the years 2011–2016. Another research was achieved by Igroufa et al. (2020), who developed an assessment framework for support infrastructure asset management. Two main objectives, six criteria, and 31 PI were constructed with the consultation of experts. The assessment tool takes into account the financial capacities of managers and the local context. The performance assessment of the two identified objectives was performed during the year 2018. Finally, Mezhoud et al. (2022) were interested in the evaluation and the prioritisation of the defects of the UDS using decision support methods. The objective is to support the ONA in its decision-making challenge on priority defects for maintenance work.

In view of the above considerations, this paper aims to answer some specific questions so that managers can take actions relating to the management of the UDS operation. It mainly seeks to achieve two main targets. The first one is to assess the operation of the UDS using PI during the period 2017–2021. The PI used in the assessment tool are defined from the regional works mentioned above. The indicators are then evaluated using performance scales, which are constructed with the help of experts. This performance assessment will help managers to identify the different elements and actions that need improvement. The second target is to analyse the performance evolution of each PI from 2017 to 2021. The interpretation of this evolution will provide answers to two questions: How could managers improve their actions? And what resources will be mobilised to achieve a good operation of the UDS? An example of application is shown on the UDS of Bejaia, a city in north-eastern Algeria.

Performance indicators can be defined as metrics used to quantify the productivity of an activity. They constitute a fundamental component of any performance assessment system. In general, the PI are expressed as ratios between variables, and these ratios may be commensurate (e.g., %) or non-commensurate (e.g., €/m³) (Santos 2021). They provide valuable information on whether the performance of the infrastructure is improving or worsening (Berg 2020). Their main objective is to provide insights into the behaviour of systems in order to make their management easier (Santos 2021).

The definition, the unit, and the processing rule of each PI are detailed in Table 1.

Table 1

Description of performance indicators identified

Performance Indicator .	Unit .	Definition and processing rule .
PI11 Number of black spots per km of the network	Nbr/km	Number of points per km of the sewerage. These spots present sites with structural sensitivities, they are characterised by the repetition of the problem and the need to treat them at least twice per year (e.g., counter slope, root intrusion, … etc.).
PI12 Rate of preventive curing on surface structure	%	Number of manholes and inlets curated preventively divided by the total number of manholes and inlets during the assessment period.
		With:  ⁠: Rate of preventive curing on surface structure;  ⁠: Number of manholes and inlets curated preventively;  ⁠: Total number of manholes and inlets.
PI13 Rate of restorative curing on surface structure	%	Number of manholes and inlets curated restoratively divided by the total number of manholes and inlets during the assessment period.   With:  ⁠: Rate of restorative curing on surface structure;  ⁠: Number of manholes and inlets curated restoratively;  ⁠: Total number of manholes and inlets.
PI21 Rate of replaced pipes	%	Length of defective pipes replaced divided by the total pipe length during the assessment period   With:  ⁠: Rate of replaced pipes;  ⁠: Length of pipes replaced;  ⁠: Total length of pipes.
PI22 Rate of repaired pipes	%	Length of defective pipes repaired divided by the total pipe length during the assessment period   With:  ⁠: Rate of repaired pipes;  ⁠: Length of pipes repaired;  ⁠: Total length of pipes.
PI23 Rate of renovated pipes	%	Length of defective pipes renovated divided by the total pipe length during the assessment period   With:  ⁠: Rate of renovated pipes;  ⁠: Length of pipes renovated;  ⁠: Total length of pipes.

Performance Indicator .	Unit .	Definition and processing rule .
PI11 Number of black spots per km of the network	Nbr/km	Number of points per km of the sewerage. These spots present sites with structural sensitivities, they are characterised by the repetition of the problem and the need to treat them at least twice per year (e.g., counter slope, root intrusion, … etc.).
PI12 Rate of preventive curing on surface structure	%	Number of manholes and inlets curated preventively divided by the total number of manholes and inlets during the assessment period.
		With:  ⁠: Rate of preventive curing on surface structure;  ⁠: Number of manholes and inlets curated preventively;  ⁠: Total number of manholes and inlets.
PI13 Rate of restorative curing on surface structure	%	Number of manholes and inlets curated restoratively divided by the total number of manholes and inlets during the assessment period.   With:  ⁠: Rate of restorative curing on surface structure;  ⁠: Number of manholes and inlets curated restoratively;  ⁠: Total number of manholes and inlets.
PI21 Rate of replaced pipes	%	Length of defective pipes replaced divided by the total pipe length during the assessment period   With:  ⁠: Rate of replaced pipes;  ⁠: Length of pipes replaced;  ⁠: Total length of pipes.
PI22 Rate of repaired pipes	%	Length of defective pipes repaired divided by the total pipe length during the assessment period   With:  ⁠: Rate of repaired pipes;  ⁠: Length of pipes repaired;  ⁠: Total length of pipes.
PI23 Rate of renovated pipes	%	Length of defective pipes renovated divided by the total pipe length during the assessment period   With:  ⁠: Rate of renovated pipes;  ⁠: Length of pipes renovated;  ⁠: Total length of pipes.

View Large

To classify performance, it is necessary to compare PI values against reference values. This may be undertaken by defining performance scales for each PI. In fact, these functions establish a relation between the PI values and a scale of classification (Santos 2021). By comparing the result of the indicators with pre-set reference values, it is possible to assign a judgment to the result, e.g., good, acceptable, or unsatisfactory (Brito et al. 2022). These performance scales can be set from the literature review, regulations standards, available assessment frameworks, benchmarking of a representative sample, or expert opinions (Brito et al. 2022). By monitoring the result of PI over time, it is possible to assess whether the implemented actions are having the expected impact.

In the present work, the construction of performance scales for each PI is based on a participatory approach. In fact, it was necessary to organise several meetings and discussions between the ONA agency experts and the researchers from the Laboratory of Applied Hydraulics and Environment (LRHAE) at the University of Bejaia. Ten experts were consulted to this aim: five from ONA and five from LRHAE. All these experts and researchers have from 10 to 20 years of experience in managing the UDS and conducting research works in the field of asset management and performance assessment. During this process, it was required to take into account the functional requirements of the UDS and the specific local context. Indeed, particular attention was paid to the type of defects appearing in the UDS, and the rate of recorded defects.

The developed assessment tool is applied to the UDS of Bejaia City. The latter is located about 220 km east of Algiers. It has 185.000 inhabitants covering an area of 120.22 km². Based on the report of ONA (2018), 91.9% of pipes have a diameter between 200 and 650 mm, 7.6% have a large diameter varying from 700 to 1,500 mm, and 0.5% have a diameter less than 200 mm. It has been noted that 99.4% of the network operates in gravity mode, of unitary type and of circular shape (Mezhoud et al. 2022). Based on the review of the available documents and the results of the survey, 79.56% of the pipes are made from concrete, 13.52% from PVC, and the rest are not indicated. Approximately 88.32% of the network pipes are aged but without precision about their age, 8.67% were constructed before 1996, 1.78% between 1997 and 1999, and 1.23% from 2000 to now (Igroufa et al. 2020). According to the inspections carried out by ONA, several anomalies have been noticed in the UDS: presence of sand in pipe, clogging, root intrusion into pipe, chemical attacks on pipe, crack, joint displacement, collapse,…etc. The UDS is thus poorly maintained and presents several black spots. In order to save the lifetime of the network, the managers have to make an adequate structured methodology to maintain and rehabilitate the defected pipes. More information about the study area can be found in Igroufa et al. (2020).

The measures collection of the PI from 2017 to 2021 has required several meetings with the managers. These meetings consisted of a deep research in the registers of data. For each year, the following data were collected:

• The length of the network: this length increased from year to year according to the investment projects carried out by the locality;

• The number of inlets and manholes: it also increased over time depending on the length of the extended pipes;

• The number of black spots: it varies from year to year according to the types of problems, and to the frequency of maintenance.

In addition to these data, a survey containing a set of questions was addressed to the staff of the maintenance department of ONA. It aims to determine the number of manholes and inlets curated preventively and restoratively, and the length of pipes repaired, replaced, and renovated. This allowed us to obtain the values of the indicators for all the years 2017, 2018, 2019, 2020, and 2021; they are reported in Table 2.

Improving the performance of these indicators will require:

• More financial and material means;

• More training for the employees;

• More coordination and communication between the managers and the owner of the network;

• More accordance of actions between all stakeholders.

In the present work, the performance of the UDS operation has been assessed using a set of performance indicators. An assessment tool has been established and applied to the case of Bejaia City for a period of years ranging from 2017 to 2021. The performance comparison of each indicator over the five years is thus carried out.

The application of the assessment tool has required scheduling multiple meetings with the experts of the UDS management. The discussions aimed to build the performance scales for each PI already defined. In fact, these functions transform the initial measurement of indicators into scores between 0 and 1. The obtained results showed the need for investment in rehabilitation and maintenance of pipes. It was found that except two indicators over two years, all the measured performances were bad or weak. Based on the results of this study, it appears that there are many challenges facing the operation practice in Algeria. In fact, UDS maintenance and rehabilitation are usually carried out only in the event of malfunctions (no operation plan is followed). Moreover, the lack of adequate technological tools and the weakness of the allocated budget constitute the main obstacles to carrying out all these tasks. This also indicates a large divergence in the actions of the stakeholders. Gathering these actors for strategic partnership becomes thus an urgent priority.

Regarding the performance evolution of each indicator, it differs from one PI to another. Three of them, PI₂₁, PI₂₂, and PI₂₃, displayed performance improvement over the considered years. This motivates the ONA's managers to reinforce their actions to achieve their goals. The remaining indicators, PI_11, PI₁₂, and PI₁₃, show an increased performance in the first years. Then, the performance relapses for one year, and rises again in the following year. This performance analysis will help ONA's managers to understand the reasons for the performance drop. Consequently, they will manage better the UDS operation by improving their actions.

The assessment tool based on performance indicators provided interesting and useful information to the managers. They can use the results of this study to help them to formulate policy and implement plans which are relevant to the problems that have been exposed. Analysis of indicators' performances gives more details about possible failures and guides ONA's managers to act in order to improve the overall performance. In a future perspective, for the completion of the analysis of the performance of UDS operation, the tool could be extended to identify further criteria and PI applicable to the context of Bejaia City.

The authors would like to express their gratitude to the ONA, which facilitated this research by providing the required data.

All relevant data are included in the paper or its Supplementary Information.

The authors declare there is no conflict.