Internal audit, certification and maturity assessment based on ISO55001 in Sendai wastewater utility, Japan

The Sendai Wastewater Utility asset management (AM) system had been developed according to the ‘Asset Management Implementation Strategy’, including business process management, to introduce and improve the principles of ‘objective management ’, ‘risk assessment frameworks', ‘decision- making criteria for investment’, etc., since 2009.

During development the utility recognized that it was necessary to have an internal audit system to check and improve the AM system continuously. Because of this an effort was made to establish an internal audit program based on ISO55001.

This paper summarizes experience of the internal audit, certification exam and maturity assessment, with discussion of their characteristics and differences, and Sendai's future developments.

The AM Implementation Strategy established in 2009 included process benchmarking with Australian water utilities using ‘Aquamark’, to understand potential improvements to the AM system. Aquamark is an AM process benchmarking framework developed by the Water Service Association of Australia and has been implemented every 4 years since 2004. The results indicated a strong need to improve Sendai's objective management and risk management framework. In addition, they also indicated that an internal audit system should be developed after improvement of the AM system. Experience of Aquamark provided a good opportunity to get to know the concepts and approaches needed to assess and improve the AM system directly, through an audit and maturity assessment.

When Sendai began using its AM system in 2013, improvements to the objective management framework, risk management framework and business processes were largely complete, and performance assessment also started, using indicators. The internal audit system for assessing the effectiveness of the established AM system hadn't then been developed, however. Development of the AM internal audit program, based on ISO55001, started with collaborative research with ABeam Consulting Ltd. (ABeam), who had participated in ISO/PC251, which was the committee to develop the ISO55000 series, and had experience of maturity assessment based on PAS55.

Research was started by assessing the maturity of the AM system. The assessment check items were developed according to the requirements of ISO55001, and their contents for evaluating the levels were determined in reference to ISO55001, ISO55002, PAS55 and Aquamark. Three parameters defined in Aquamark were adopted to assess the AM system – process development, process documentation and process coverage – and the level of each was scored between 1 and 5. Subdividing the parameters and assessing them from each point of view enabled a simple and easy evaluation. Each check item evaluated, for example, whether process flows and guidelines existed, and covered the item contents and how the AM system penetrated into the businesses. ABeam carried out the assessment by questioning those staff members who had developed and administrated the audit program.

The internal audit program was established using ABeam's internal audit program for ISO9001 and ISO27001 as reference. It included the audit criteria, the audit plan and the checklist.

The internal audit criteria are the basic rules of the internal audit, including audit policy, the audit implementation system, the requirements of the auditor, and the audit procedure. The audit plan describes the audit's purpose, and its scope, objectives and schedule each year. For example, the scope of the 2013 audit included all sections of the utility, and focused on staff awareness and the system's penetration. The audit checklist was developed to clarify the auditors' interview points and to provide examples of questions to examinees. The check items, points and questions were described for every requirement of ISO55001, based on the results of the maturity assessment. An example of the 2013 audit checklist is given in Table 1.

The first internal audit was executed on the basis of the established audit program. The auditors were selected from directors and young employees who were expected to be core members of AM system operation. They were given one day's training to gain basic knowledge of the audit and AM system, the contents of ISO55001, and questions and evaluation skills.

The results of the internal audit are shown in Table 2, which revealed both hidden problems and good practices. The ideas from and requests concerning the AM system made by examinees were also picked up. These results were directly reported to and reviewed by the ‘Wastewater Utility Coordination Committee’, which had been established for management review. The nonconformities found in the audit were dealt through corrective actions agreed with each section. The results of these were confirmed in the ‘follow up audit’, and their effectiveness and improvements arising will be reviewed in future audits.

When publication of ISO55001 was approaching in July 2013, MLIT recognized its importance and set up the ISO55001 Application Guideline Research Committee. This committee aimed to summarize the guideline (finally called the ‘user's guide’) to explain ISO55001 application for utilities by conducting trial certification in selected public utilities and private companies. Sendai Wastewater Utility decided to participate in the trial to check the validity and improve the effectiveness of its audit by conducting the certification procedure simultaneously.

The gap analysis against ISO55001 was conducted over 3 months. It assessed the AM system contents – e.g., business processes, criteria, manuals, etc. – according to the ISO55001 requirements. The actual analysis was executed in reference to the method in the ISO9001 and 14001 certification, because most of the requirements are common to them. The check points for the certification exam were grasped through the analysis. At the same time, the AM system issues found in the exam analysis were modified.

The certification exam was conducted after final publication of ISO55001 in January 2014. No nonconformities were found other than minor issues like lack of awareness of the AM system by operators in the certification exam.

The wastewater network part of the AM system was certified on March 11, 2014, as the first case in Japan. The scope of certification was also expanded to all parts of the wastewater business in March 2015.

While the performance of a utility's business can be measured by performance indicators, an internal audit, certification and maturity assessment are best for assessing the AM system itself. Internal audits, ISO certification and maturity assessment each have strengths and weaknesses, and benefits from each method arise from their strengths. The benefits, strengths and weaknesses are shown in the Table 3.

An internal audit is a systematic method of managing the validity and effectiveness of business processes, criteria, manuals, etc., in an AM system. A utility with an AM system should also have an internal audit system to maintain and improve it. In particular, internal audit is more effective when an existing management system is changed comprehensively – e.g., implementation of an AM – because an audit facilitates understanding of the system, and communications between top management and staff members.

On the other hand, an internal audit has the weaknesses shown in Table 3:

• It is difficult for young auditors to ask severe questions, especially in conventional Japanese work environments. The questionnaires tend to be platitudes and examinee preparation becomes easy.

• It is difficult to cover all areas of the AM system in a limited time.

• The level of the AM system cannot be evaluated effectively in the audit.

These issues may lead the audit to lose substance. They can be compensated for, however, by certification and maturity assessment. Thus, the first issue can be covered by the strength of certification. Assessors can be objective and ask questions based strictly on their knowledge and experience. Gap analysis in certification ensures that the utility can be assessed through the continuous viewpoints based on the utility's character.

The second and third problems can be resolved by maturity assessment of an AM system. This can be used to evaluate the system's operational level quantitatively, and requires examinees to answer all areas and questions. The utility then can understand where and by how much its AM system should be improved.

AM systems can be improved using these evaluation methods if it is understood that they complement each other.

The requirements of ISO55001 represent common practice in AM and not best practice in any specific sector. Therefore, it is difficult to say whether a utility with ISO certification operates to the best-level management. In such cases it is effective to implement process benchmarking based on best practice in a specific sector – e.g., Aquamark. The Aquamark inquiries are more detailed and specific than the requirements of ISO55001, so it may be more suitable for utilities aiming for higher levels of AM improvement.

Sendai Wastewater Utility implemented internal audits, a certification exam and maturity assessment on the basis of ISO55001. The results suggest that those three methods improved conformity to the standard and the effectiveness of the AM system, while complimenting each others' weaknesses.

Experience of the internal audit and ISO certification in Sendai Wastewater Utility were reflected in the user guide published by MLIT and could contribute to the spread of ISO55001 in Japan.

Sendai Wastewater Utility will continue to improve its AM system by developing these activities further.

This paper used the results of collaborative research with ABeam Consulting Ltd. in 2013. Especially we would like to show our appreciation to Mr. Enomoto.