Sendai Wastewater Utility acquired the first certification of ISO55001 in Japan at March 2014. ISO55001 is the international standard for asset management systems and was published in January 2014. During the process towards certification, the maturity assessment, internal audit and certification exam based on ISO55001 were all executed. It was possible, therefore, to compare the three methods, and recognize their strengths and weaknesses. The methods complement each other when implemented simultaneously.

INTRODUCTION

The Sendai Wastewater Utility asset management (AM) system had been developed according to the ‘Asset Management Implementation Strategy’, including business process management, to introduce and improve the principles of ‘objective management ’, ‘risk assessment frameworks', ‘decision- making criteria for investment’, etc., since 2009.

During development the utility recognized that it was necessary to have an internal audit system to check and improve the AM system continuously. Because of this an effort was made to establish an internal audit program based on ISO55001.

At the same time, the Sewerage Department of the Ministry of Land, Infrastructure, Transport and Tourism (MLIT) formed a committee to draft ISO55001 application guidelines in preparation for its publication. The committee chose utility to execute the certification exam on a trial basis. After the gap analysis between the requirements of ISO55001 and Sendai's AM system had been implemented to find check points, the ISO55001 certification examination was conducted for the first time in Japan. The user's guide for application was developed on the basis of this experience. The schedule of the internal audit and the research committee is shown in Figure 1.
Figure 1

Schedule of internal audit and certification in fiscal year 2013/14.

Figure 1

Schedule of internal audit and certification in fiscal year 2013/14.

This paper summarizes experience of the internal audit, certification exam and maturity assessment, with discussion of their characteristics and differences, and Sendai's future developments.

ESTABLISHMENT OF THE INTERNAL AUDIT PROGRAM

Approach to internal audit

The AM Implementation Strategy established in 2009 included process benchmarking with Australian water utilities using ‘Aquamark’, to understand potential improvements to the AM system. Aquamark is an AM process benchmarking framework developed by the Water Service Association of Australia and has been implemented every 4 years since 2004. The results indicated a strong need to improve Sendai's objective management and risk management framework. In addition, they also indicated that an internal audit system should be developed after improvement of the AM system. Experience of Aquamark provided a good opportunity to get to know the concepts and approaches needed to assess and improve the AM system directly, through an audit and maturity assessment.

When Sendai began using its AM system in 2013, improvements to the objective management framework, risk management framework and business processes were largely complete, and performance assessment also started, using indicators. The internal audit system for assessing the effectiveness of the established AM system hadn't then been developed, however. Development of the AM internal audit program, based on ISO55001, started with collaborative research with ABeam Consulting Ltd. (ABeam), who had participated in ISO/PC251, which was the committee to develop the ISO55000 series, and had experience of maturity assessment based on PAS55.

Maturity assessment

Research was started by assessing the maturity of the AM system. The assessment check items were developed according to the requirements of ISO55001, and their contents for evaluating the levels were determined in reference to ISO55001, ISO55002, PAS55 and Aquamark. Three parameters defined in Aquamark were adopted to assess the AM system – process development, process documentation and process coverage – and the level of each was scored between 1 and 5. Subdividing the parameters and assessing them from each point of view enabled a simple and easy evaluation. Each check item evaluated, for example, whether process flows and guidelines existed, and covered the item contents and how the AM system penetrated into the businesses. ABeam carried out the assessment by questioning those staff members who had developed and administrated the audit program.

The assessment results are shown in Figure 2. They made it possible not only to grasp the present situation to construct the program but also to specify the issues in the AM system. For example, the roles and responsibilities of top management, the poor definition of competency, the insufficient documentation, etc., were all indicated by their low maturity levels in the assessment. These problems were improved before the audit and the certification exam.
Figure 2

Results of the maturity assessment in July 2013.

Figure 2

Results of the maturity assessment in July 2013.

Drafting the internal audit program

The internal audit program was established using ABeam's internal audit program for ISO9001 and ISO27001 as reference. It included the audit criteria, the audit plan and the checklist.

The internal audit criteria are the basic rules of the internal audit, including audit policy, the audit implementation system, the requirements of the auditor, and the audit procedure. The audit plan describes the audit's purpose, and its scope, objectives and schedule each year. For example, the scope of the 2013 audit included all sections of the utility, and focused on staff awareness and the system's penetration. The audit checklist was developed to clarify the auditors' interview points and to provide examples of questions to examinees. The check items, points and questions were described for every requirement of ISO55001, based on the results of the maturity assessment. An example of the 2013 audit checklist is given in Table 1.

Table 1

Example of internal audit checklist

  Requirement Example of question Check item 
Outsourcing Control of processes and activities to be outsourced Explain the process of work orders.
Do you implement your job according to the defined processes?
Show the job records 
Examinees understand the defined business processes and their documentation.
The jobs are executed according to the processes.
Necessary information is recorded. 
  Requirement Example of question Check item 
Outsourcing Control of processes and activities to be outsourced Explain the process of work orders.
Do you implement your job according to the defined processes?
Show the job records 
Examinees understand the defined business processes and their documentation.
The jobs are executed according to the processes.
Necessary information is recorded. 

Conducting the internal audit

The first internal audit was executed on the basis of the established audit program. The auditors were selected from directors and young employees who were expected to be core members of AM system operation. They were given one day's training to gain basic knowledge of the audit and AM system, the contents of ISO55001, and questions and evaluation skills.

The results of the internal audit are shown in Table 2, which revealed both hidden problems and good practices. The ideas from and requests concerning the AM system made by examinees were also picked up. These results were directly reported to and reviewed by the ‘Wastewater Utility Coordination Committee’, which had been established for management review. The nonconformities found in the audit were dealt through corrective actions agreed with each section. The results of these were confirmed in the ‘follow up audit’, and their effectiveness and improvements arising will be reviewed in future audits.

Table 2

Results of the 2013 internal audit

Type of indication Number Main contents 
Minor nonconformities 
  • Lack of understanding of AM guidelines

  • Defined business processes had not been implemented

  • Training and seminars had not been implemented

 
Recommendations 
  • Revision of manuals

  • Tacit knowledge held by few staff should be converted to explicit knowledge

 
Good points 
  • Preparing manuals of routine works independently

  • The staff members of sewerage administration center were studied in the treatment plant to improve their works

 
Opinions and requests 12 
  • Clarification of procedures and roles

  • Revision of AM guidelines

 
Type of indication Number Main contents 
Minor nonconformities 
  • Lack of understanding of AM guidelines

  • Defined business processes had not been implemented

  • Training and seminars had not been implemented

 
Recommendations 
  • Revision of manuals

  • Tacit knowledge held by few staff should be converted to explicit knowledge

 
Good points 
  • Preparing manuals of routine works independently

  • The staff members of sewerage administration center were studied in the treatment plant to improve their works

 
Opinions and requests 12 
  • Clarification of procedures and roles

  • Revision of AM guidelines

 

IMPLEMENTATION OF CERTIFICATION EXAM AND MLIT

Research committee

When publication of ISO55001 was approaching in July 2013, MLIT recognized its importance and set up the ISO55001 Application Guideline Research Committee. This committee aimed to summarize the guideline (finally called the ‘user's guide’) to explain ISO55001 application for utilities by conducting trial certification in selected public utilities and private companies. Sendai Wastewater Utility decided to participate in the trial to check the validity and improve the effectiveness of its audit by conducting the certification procedure simultaneously.

Gap analysis and certification exam

The gap analysis against ISO55001 was conducted over 3 months. It assessed the AM system contents – e.g., business processes, criteria, manuals, etc. – according to the ISO55001 requirements. The actual analysis was executed in reference to the method in the ISO9001 and 14001 certification, because most of the requirements are common to them. The check points for the certification exam were grasped through the analysis. At the same time, the AM system issues found in the exam analysis were modified.

The certification exam was conducted after final publication of ISO55001 in January 2014. No nonconformities were found other than minor issues like lack of awareness of the AM system by operators in the certification exam.

The wastewater network part of the AM system was certified on March 11, 2014, as the first case in Japan. The scope of certification was also expanded to all parts of the wastewater business in March 2015.

BENEFITS AND DISCUSSION

Audit, certification and maturity assessment benefits

While the performance of a utility's business can be measured by performance indicators, an internal audit, certification and maturity assessment are best for assessing the AM system itself. Internal audits, ISO certification and maturity assessment each have strengths and weaknesses, and benefits from each method arise from their strengths. The benefits, strengths and weaknesses are shown in the Table 3.

Table 3

Benefits, strengths and weaknesses of internal audits, ISO certification and maturity assessment

  Benefits Strengths Weaknesses 
Internal audit 
  • Reinforcing PDCA cycles in the utility

  • Facilitating communications between field operators and top management

  • Improving business using auditor advice

  • Improving the AM system because the auditors acted like core staff

 
  • Periodic assessment

  • Improving auditor's knowledge

  • Easy to agree the nonconformities through the dialogues between auditors and examinees

  • Setting an audit theme timely and appropriately

 
  • Difficulty for auditors to ask severe questions in conventional work environments

  • Sampling survey carried out in limited time

  • Qualitative evaluation

  • Uneven auditor skills

 
ISO certification 
  • Grasping potential improvements differing from conventional ideas, by external assessment

  • Facilitating examinees’ understanding of the AM system through their explanations to assessors

  • Promoting corrective actions found in the audit

 
  • Certification based on an international standard

  • Highly objective assessment

  • Getting credibility from stakeholders

  • Incentive for internal business processes such as an internal audit

 
  • Sampling survey in a limited time

  • Checkpoints do not affect the highest level management of specific sectors

  • Difficult to get support to improve businesses because consultation is prohibited in certification

 
Maturity assessment 
  • Recording the situation more accurately and quantitatively

  • Grasping points to improve, by quantitative and visualized evaluation

  • Comparing the AM system with other utilities if assessment criteria are shared

 
  • Quantitative evaluation

  • Comprehensive assessment by checking all requirements

  • Indirect assessment without interviews based on the results of other audits and meetings

 
  • Difficult to improve issues when assessment conducted without examinee consultation

  • Insufficient objectivity if evaluated by few staff member

 
  Benefits Strengths Weaknesses 
Internal audit 
  • Reinforcing PDCA cycles in the utility

  • Facilitating communications between field operators and top management

  • Improving business using auditor advice

  • Improving the AM system because the auditors acted like core staff

 
  • Periodic assessment

  • Improving auditor's knowledge

  • Easy to agree the nonconformities through the dialogues between auditors and examinees

  • Setting an audit theme timely and appropriately

 
  • Difficulty for auditors to ask severe questions in conventional work environments

  • Sampling survey carried out in limited time

  • Qualitative evaluation

  • Uneven auditor skills

 
ISO certification 
  • Grasping potential improvements differing from conventional ideas, by external assessment

  • Facilitating examinees’ understanding of the AM system through their explanations to assessors

  • Promoting corrective actions found in the audit

 
  • Certification based on an international standard

  • Highly objective assessment

  • Getting credibility from stakeholders

  • Incentive for internal business processes such as an internal audit

 
  • Sampling survey in a limited time

  • Checkpoints do not affect the highest level management of specific sectors

  • Difficult to get support to improve businesses because consultation is prohibited in certification

 
Maturity assessment 
  • Recording the situation more accurately and quantitatively

  • Grasping points to improve, by quantitative and visualized evaluation

  • Comparing the AM system with other utilities if assessment criteria are shared

 
  • Quantitative evaluation

  • Comprehensive assessment by checking all requirements

  • Indirect assessment without interviews based on the results of other audits and meetings

 
  • Difficult to improve issues when assessment conducted without examinee consultation

  • Insufficient objectivity if evaluated by few staff member

 

Relationship between internal audit, certification and maturity assessment

An internal audit is a systematic method of managing the validity and effectiveness of business processes, criteria, manuals, etc., in an AM system. A utility with an AM system should also have an internal audit system to maintain and improve it. In particular, internal audit is more effective when an existing management system is changed comprehensively – e.g., implementation of an AM – because an audit facilitates understanding of the system, and communications between top management and staff members.

On the other hand, an internal audit has the weaknesses shown in Table 3:

  • It is difficult for young auditors to ask severe questions, especially in conventional Japanese work environments. The questionnaires tend to be platitudes and examinee preparation becomes easy.

  • It is difficult to cover all areas of the AM system in a limited time.

  • The level of the AM system cannot be evaluated effectively in the audit.

These issues may lead the audit to lose substance. They can be compensated for, however, by certification and maturity assessment. Thus, the first issue can be covered by the strength of certification. Assessors can be objective and ask questions based strictly on their knowledge and experience. Gap analysis in certification ensures that the utility can be assessed through the continuous viewpoints based on the utility's character.

The second and third problems can be resolved by maturity assessment of an AM system. This can be used to evaluate the system's operational level quantitatively, and requires examinees to answer all areas and questions. The utility then can understand where and by how much its AM system should be improved.

AM systems can be improved using these evaluation methods if it is understood that they complement each other.

Following up maturity assessment

After the first assessment, Sendai Wastewater Utility calculated the expected maturity score for the end of 2015, when maturity was assessed again. The results are shown in Figure 3. The steady AM implementation and two audits led to improvements in the AM system's maturity score. In particular, operation of the risk management framework, documented information and the internal audit improved dramatically. In contrast, the AM planning and resource scores were worse than expected because AM plan drafting was delayed more than expected. It is possible to select and concentrate on those parts needing development, using these results. For example, it might be difficult to improve the scores of 7.1 Resource and 7.2 Competency, because human resource management based on manifest staff competency in relation to job description is not popular in traditional Japanese working environments, especially in municipal government. This is one of the cultural differences between industry in Japan and other countries, and it is hard for individual utilities to overcome.
Figure 3

Results of the March 2015 follow up maturity assessment.

Figure 3

Results of the March 2015 follow up maturity assessment.

Future improvement beyond ISO55001

The requirements of ISO55001 represent common practice in AM and not best practice in any specific sector. Therefore, it is difficult to say whether a utility with ISO certification operates to the best-level management. In such cases it is effective to implement process benchmarking based on best practice in a specific sector – e.g., Aquamark. The Aquamark inquiries are more detailed and specific than the requirements of ISO55001, so it may be more suitable for utilities aiming for higher levels of AM improvement.

CONCLUSION

Sendai Wastewater Utility implemented internal audits, a certification exam and maturity assessment on the basis of ISO55001. The results suggest that those three methods improved conformity to the standard and the effectiveness of the AM system, while complimenting each others' weaknesses.

Experience of the internal audit and ISO certification in Sendai Wastewater Utility were reflected in the user guide published by MLIT and could contribute to the spread of ISO55001 in Japan.

Sendai Wastewater Utility will continue to improve its AM system by developing these activities further.

ACKNOWLEDGEMENTS

This paper used the results of collaborative research with ABeam Consulting Ltd. in 2013. Especially we would like to show our appreciation to Mr. Enomoto.

REFERENCE

REFERENCE
Abe
Y.
Enomoto
Y.
2014
Establishment of asset management internal audit based on ISO55001
. In:
Abstract book of the 51st Annual Technical Conference on Sewerage Osaka)
.
Japan Sewage Works Association
,
Tokyo
,
Japan
, pp.
85
87
(in Japanese).