Developing a water safety plan (WSP) is now a requirement for all service providers of drinking water in the Philippines. To assist compliance with the Philippine Department of Health (DOH), this study develops an index model that the DOH can use for evaluating WSPs and covers the WSPs of 14 water districts and 11 health care facilities. The WSP Index model was developed using a nine-step process and was tested in 25 WSPs to determine the robustness of its weights and benchmark. Approximately 21 WSPs received a passing mark when the 60% benchmark was used but only nine WSPs passed when the benchmark was raised to 74%. This Philippine model may be utilized by countries in evaluating the WSPs, and further adapted to their local context and considerations.

INTRODUCTION

Water safety plan (WSP) development has been a key recommendation of the World Health Organization (WHO) Guidelines for Drinking-water Quality, 3rd (WHO 2004) and 4th (WHO 2011) editions. Some countries have embarked on studies to measure the beneficial effects of WSPs on water quality, operations of water utilities, and public health (Gunnarsdottir 2012; Gunnarsdottir et al. 2012). With the popularity of WSP initiatives, approximately 90 countries around the world have implemented WSPs and about 40 countries have developed policies or regulations that promote or require WSPs, including the Philippines (WHO 2016).

In the Philippines, the water safety planning approach was introduced in 2006 and was first applied in Maynilad Water Services a Incorporated with support from the Australian government and the WHO. Lessons learned from piloting were replicated by other types of drinking-water service providers in the country. As of June 2016, more than 300 water districts (a local corporate entity that operates and maintains a water supply system in one or more provincial cities or municipalities), ten local-government-managed water systems (a drinking-water service provider operated by a city, or municipal or provincial government), eight water refilling stations (a drinking-water service provider operated by a private entrepreneur that uses a packaged-type water treatment technology and sells or delivers treated water to households or establishments by filling 20-litre plastic containers either owned by the station or by the customers), 11 health care facilities (a place where patients visit to avail themselves of health care services such as hospitals and rural health units), and ten community-based water systems (a drinking-water service provider operated by a community organization) received training on water safety planning. These activities saw the completion of about 65 WSPs.

A nationwide roll-out of WSPs is envisaged by the Department of Health (DOH) as a strategy to curb waterborne diseases. To achieve this, on 4 September 2014, the DOH issued the ‘National Policy on WSP’ (Administrative Order No. 2014-0027) which requires all drinking-water service providers (the Listahang Tubig or Water Register of the Philippines recorded 23,070 drinking-water supply providers as of May 2015 (NWRB 2015)) to develop and implement WSPs within three years. To support the DOH, the Local Water Utilities Administration (LWUA) issued Memorandum Circular No. 010.14 on 1 December 2014 reinforcing the requirement for all water districts to develop and implement WSPs. Given this, the demand for WSP development is expected to increase in the coming years, including its review and approval as specified in the Administrative Order.

The DOH is still in the process of developing guidelines for reviewing and approving WSPs. The completed WSPs have not undergone a formal evaluation to determine if they comply with the DOH requirements of a satisfactory understanding of WSP concepts and processes. To assist the DOH with the issuance of guidelines for the WSP review and approval process, this research attempts to develop an index model for evaluating the acceptability of WSPs. This can serve as the basis for approval of a WSP before its implementation. Considering the 11 steps and five stages of WSP development, it would be helpful for the WSP evaluator to have a tool that guides the review process, summarizing the performance of a drinking-water service provider in developing its WSP. The index value expressed as a single number will facilitate the decision of the approving entity on whether to accept the WSP or not.

MATERIALS AND METHODS

Concepts of WSP review and approval

The WHO WSP guidelines (Davison 2005), which discuss the items to be reviewed and approved in a WSP and the WHO WSP Quality Assurance tool (WHO 2012), which helps appraise the stage of WSP processes including WSP development and implementation, were used in developing the framework of this study. Following a review of the literature, the research found a gap in available models that can serve as a reference for evaluating any type of WSP. This lack of information prompted the study and development of a model for the WSP Index.

Concepts of an index model

The use of indices is not new in the development realm. Jacobs et al. (2004) defined ‘index’ as an aggregation of indicators of individual performance that integrates a large amount of information in a format that is easily understood and is therefore a valuable tool for conveying a summary assessment of performance in priority areas. According to Sharpe & Smith (2005), it is an aggregation of individual indicators using a certain weighting scheme. Metge et al. (2009) said that it is a mathematical combination of several indicators or measures which can be used to describe an entire set of indicators, and allows for an examination of differences between places and across time.

Steps in developing the WSP Index

In formulating a WSP Index, the author considered the steps recommended by OECD (2008) and Jacobs et al. (2004). The following procedures were adopted.

Establish a theoretical framework

Figure 1 shows the theoretical framework used in the study, which was derived from the WSP Manual (WHO 2009). It contains WSP stage indicators (i.e., categories of the different steps in developing a WSP); WSP step indicators (i.e., procedures for how to prepare a WSP); and WSP output indicators (i.e. required items to be completed at every step of WSP development). Output indicators are evaluated with a score based on requirements of the WSP steps. The raw scores are consolidated per WSP stage and multiplied by established weights per WSP stage to determine the WSP Index.
Figure 1

Theoretical framework of the WSP Index.

Figure 1

Theoretical framework of the WSP Index.

Select indicators

As reflected in the framework, the WSP stage indicators are: (1) Preparation; (2) System Assessment; (3) Operational Monitoring; (4) Management and Communication; and (5) Feedback and Improvement (Figure 1). The WSP step indicators include: (1) Assemble the WSP team; (2) Describe the water supply system; (3) Identify the hazardous events and assess the risk; (4) Determine and validate control measures, reassess and prioritize risks; (5) Develop, implement and maintain an improvement/upgrade plan; (6) Define monitoring of the control measures; (7) Verify the effectiveness of the WSP; (8) Prepare management procedures; (9) Develop supporting programs; (10) Plan and carry out a periodic review of the WSP; and (11) Revise the WSP following an incident. Twenty-eight output indicators (Table 1) were identified as the required observable results of each WSP step. These indicators were derived from the WSP Quality Assurance tool (WHO 2012), WSP Manual (WHO 2009), and WSP Manual for Distribution Systems (WHO 2014) using the criteria of Davison et al. (2005) and WHO (2011).

Table 1

WSP output indicators and maximum possible scoresa per output and per WSP stage

StageStepOutput indicatorMaximum possible score per output indicatorTotal score per stage
1. Management support  
2. WSP team  
3. Stakeholders  
   15 
4. Users and uses  
5. Drinking water quality standards  
6. Schematic/flow diagrams  
7. System description  
8. Risk assessment methodology  
9. Hazardous events  
10. Raw risks  
11. Control measures  
12. Residual risks  
13. Significant risks  
14. Improvement plan link to risks  
15. Improvement plan details  
   60 
16. Operational monitoring plan  
17. Corrective action  
18. Compliance monitoring plan  
19. Internal auditing plan  
20. External auditing plan  
21. Consumer satisfaction monitoring plan  
   30 
22. SOPs – Normal  
23. SOPs – Incident  
24. SOPs – Emergency  
25. Supporting program description  
26. Supporting program plan  
   25 
10 27. Periodic review plan  
11 28. Revision plan  
   10 
Total   140 140 
StageStepOutput indicatorMaximum possible score per output indicatorTotal score per stage
1. Management support  
2. WSP team  
3. Stakeholders  
   15 
4. Users and uses  
5. Drinking water quality standards  
6. Schematic/flow diagrams  
7. System description  
8. Risk assessment methodology  
9. Hazardous events  
10. Raw risks  
11. Control measures  
12. Residual risks  
13. Significant risks  
14. Improvement plan link to risks  
15. Improvement plan details  
   60 
16. Operational monitoring plan  
17. Corrective action  
18. Compliance monitoring plan  
19. Internal auditing plan  
20. External auditing plan  
21. Consumer satisfaction monitoring plan  
   30 
22. SOPs – Normal  
23. SOPs – Incident  
24. SOPs – Emergency  
25. Supporting program description  
26. Supporting program plan  
   25 
10 27. Periodic review plan  
11 28. Revision plan  
   10 
Total   140 140 

aMaximum possible score of 5 is used per output indicator to determine the maximum possible total score per stage to be used as denominator in the WSP Index formula.

Determine the weights

The analytic hierarchy process (AHP) (Saaty 2008) using pairwise comparison was used in determining the corresponding weights of the five WSP stages (Table 2). The weights were calculated using an eigenvector technique and were normalized using a consistency ratio. AHP is considered over other weighting methods due to the involvement of local expert opinion and higher level of transparency of indicators that are inherent features of index development (OECD 2008). The inputs from the local WSP experts made the established weights unique to the Philippines. These inputs are opinions on which WSP stage is considered more important than others. Other countries may adopt the process of establishing the weights from the AHP method using the opinions of their local experts.

Table 2

Determination of weights per WSP stage using pairwise comparison of AHP

WSP stageEigenvector
Computed weightsSuggested weights by local WSP experts
12345TotalRow average
0.0769 0.0833 0.0541 0.0870 0.0769 0.3782 0.0756 0.08 0.10 
0.3846 0.4167 0.5405 0.4348 0.3846 2.1612 0.4322 0.43 0.40 
0.3846 0.2083 0.2703 0.3478 0.3077 1.5187 0.3037 0.30 0.30 
0.0769 0.2083 0.0676 0.0870 0.1538 0.5936 0.1187 0.12 0.15 
0.0769 0.0833 0.0676 0.0435 0.0769 0.3482 0.0696 0.07 0.05 
WSP stageEigenvector
Computed weightsSuggested weights by local WSP experts
12345TotalRow average
0.0769 0.0833 0.0541 0.0870 0.0769 0.3782 0.0756 0.08 0.10 
0.3846 0.4167 0.5405 0.4348 0.3846 2.1612 0.4322 0.43 0.40 
0.3846 0.2083 0.2703 0.3478 0.3077 1.5187 0.3037 0.30 0.30 
0.0769 0.2083 0.0676 0.0870 0.1538 0.5936 0.1187 0.12 0.15 
0.0769 0.0833 0.0676 0.0435 0.0769 0.3482 0.0696 0.07 0.05 

CI = 0.0937; CR = 0.084.

Develop a scoring system

The scoring system has a rating that interprets the scores to evaluate the contents and quality of WSP output indicators. The scores and ratings used are: 1 – Poor, if the output did not comply with major requirements and with errors in understanding major concepts; 2 – Fair, if the output has completed major requirements but with errors in understanding major concepts; 3 – Good, if the output has completed major requirements but with errors in understanding minor concepts; 4 – Very Good, if the output has completed all requirements with minor corrections in specific items; and 5 – Excellent, if the indicator has completed all requirements and all concepts are well understood.

Aggregate the indicators to formulate the WSP Index

The WSP Index formula included the weights of the five WSP stages and scores of output indicators. Additive aggregation was used to compute the value of the WSP Index. This method is preferred due to its simplicity, independence from outliers, and compensatory ability for low scores, which is needed in a benchmarking exercise. According to OECD (2008), a linear aggregation method is useful when all individual indicators have the same units of measurement and with preferred independence. However, the absolute value of the indicators has a tendency to be lost (OECD 2008). 
formula
where:
  • S1 = s1 = Stage 1: Preparation scores

  • S2 = s2 + s3 + s4 + s5 = Stage 2: System assessment scores

  • S3 = s6 + s7 = Stage 3: Operational monitoring scores

  • S4 = s8 + s9 = Stage 4: Management and communication scores

  • S5 = s10 + s11 = Stage 5: Feedback and improvement scores

  • wi (i=1 to 5) = weight per WSP stage (in percent)

  • xi(i=1 to 5) = total maximum possible score per WSP stage

  • si (i=1 to 5) = scores per WSP step

Set a benchmark for WSP acceptability

To determine the passing mark for WSPs, a minimum score of 3 in all of the 28 output indicators was considered. The score of 3 means that the drinking-water service provider has completed the major requirements, but with errors in understanding minor concepts in any of the output indicators. This computation resulted in 60% as the minimum WSP Index needed for WSP acceptance. The robustness of this benchmark was tested using sensitivity analysis and it was found that a minimum benchmark of 74% for WSP acceptability is more realistic (Table 3).

Table 3

Simulating WSP Index by adjusting the scores of WSP steps

StageStepOutput indicatorMax scoreAssumption
12
1. Management support 
2. WSP team 
3. Stakeholders 
4. Users and uses 
5. Drinking water quality standards 
6. Schematic/flow diagrams 
7. System description 
8. Risk assessment methodology 
9. Hazardous events 
10. Raw risks 
11. Control measures 
12. Residual risks 
13. Significant risks 
14. Improvement plan link to risks 
15. Improvement plan details 
16. Operational monitoring plan 
17. Corrective action 
18. Compliance monitoring plan 
19. Internal auditing plan 
20. External auditing plan 
21. Consumer satisfaction monitoring plan 
22. SOPs – Normal 
23. SOPs – Incident 
24. SOPs – Emergency 
25. Supporting program description 
26. Supporting program plan 
10 27. Periodic review plan 
11 28. Revision plan 
WSP Index 60% 74% 
StageStepOutput indicatorMax scoreAssumption
12
1. Management support 
2. WSP team 
3. Stakeholders 
4. Users and uses 
5. Drinking water quality standards 
6. Schematic/flow diagrams 
7. System description 
8. Risk assessment methodology 
9. Hazardous events 
10. Raw risks 
11. Control measures 
12. Residual risks 
13. Significant risks 
14. Improvement plan link to risks 
15. Improvement plan details 
16. Operational monitoring plan 
17. Corrective action 
18. Compliance monitoring plan 
19. Internal auditing plan 
20. External auditing plan 
21. Consumer satisfaction monitoring plan 
22. SOPs – Normal 
23. SOPs – Incident 
24. SOPs – Emergency 
25. Supporting program description 
26. Supporting program plan 
10 27. Periodic review plan 
11 28. Revision plan 
WSP Index 60% 74% 

Note: Assumption 1 – All output indicators have a minimum score of 3.

Assumption 2 – All output indicators with readily available information were given a maximum possible score of 5.

Test the WSP Index model

Purposive sampling was used in selecting types of WSPs from the existing WHO project that recently evaluated WSPs. This includes 14 water districts and 11 health care facilities. The author tested the WSP Index formula in 25 WSPs to determine its applicability (Table 4). An Excel spreadsheet was utilized to facilitate computation.

Table 4

Summary of WSP Index of drinking-water supply providers

 WSP Index (%)
benchmark = 60%
Equal weightsSimple weightsAHP weights
a. Water Districts 
 1. Water District A 61 69 68 
 2. Water District B 76 80 81 
 3. Water District C 64 63 64 
 4. Water District D 66 70 71 
 5. Water District E 79 79 79 
 6. Water District F 71 72 71 
 7. Water District G 75 71 70 
 8. Water District H 63 66 68 
 9. Water District I 69 72 72 
 10. Water District J 91 89 87 
 11. Water District K 89 90 90 
 12. Water District L 70 76 74 
 13. Water District M 70 71 73 
 14. Water District N 52 57 57 
b. Health Care Facilities 
 1. Health Care Facility A 72 73 71 
 2. Health Care Facility B 34 39 37 
 3. Health Care Facility C 82 79 78 
 4. Health Care Facility D 75 71 70 
 5. Health Care Facility E 63 69 69 
 6. Health Care Facility F 80 82 83 
 7. Health Care Facility G 65 61 58 
 8. Health Care Facility H 64 56 54 
 9. Health Care Facility I 89 88 88 
 10. Health Care Facility J 63 63 63 
 11. Health Care Facility K 88 88 89 
 WSP Index (%)
benchmark = 60%
Equal weightsSimple weightsAHP weights
a. Water Districts 
 1. Water District A 61 69 68 
 2. Water District B 76 80 81 
 3. Water District C 64 63 64 
 4. Water District D 66 70 71 
 5. Water District E 79 79 79 
 6. Water District F 71 72 71 
 7. Water District G 75 71 70 
 8. Water District H 63 66 68 
 9. Water District I 69 72 72 
 10. Water District J 91 89 87 
 11. Water District K 89 90 90 
 12. Water District L 70 76 74 
 13. Water District M 70 71 73 
 14. Water District N 52 57 57 
b. Health Care Facilities 
 1. Health Care Facility A 72 73 71 
 2. Health Care Facility B 34 39 37 
 3. Health Care Facility C 82 79 78 
 4. Health Care Facility D 75 71 70 
 5. Health Care Facility E 63 69 69 
 6. Health Care Facility F 80 82 83 
 7. Health Care Facility G 65 61 58 
 8. Health Care Facility H 64 56 54 
 9. Health Care Facility I 89 88 88 
 10. Health Care Facility J 63 63 63 
 11. Health Care Facility K 88 88 89 

Conduct uncertainty and sensitivity analysis

Potential sources of uncertainty were addressed by conducting the following activities: selection (inclusion or exclusion) of individual indicators; normalization – use of ranking for output indicators; weighting – use of analytical hierarchy process; and aggregation of indicators – use of linear additive aggregation.

Sensitivity analysis is used to test the robustness of the AHP weights and the WSP Index benchmark of 60%. Table 4 shows that AHP weights are more stringent than equal weights (100% divided by five categories = 20%) and simple weights (maximum possible score of a WSP category divided by the total maximum possible score of 140) in the sense that, for example, Health Care Facility G WSP, with a failing mark of 58% using AHP weights, got a passing mark of 65% and 61%, respectively, in consideration of the initial benchmark of 60%. Sample computations are shown in Tables 57 for the WSP Index using different weighting systems. To test the 60% benchmark, scores of output indicators were manipulated using two assumptions:

  • (1)

    all output indicators were given a minimum score of 3; and

  • (2)

    all output indicators with readily available information were given a maximum score of 5. This test resulted in a 74% benchmark.

Table 5

Sample computation of WSP Index for Water District B using equal weights

StageStepOutput indicatorScore per output indicatorTotal score per stage (A)Maximum possible score per stage (B)Equal weights (%) (C)WSP Index (%) (D)
1. Management support     
2. WSP team     
3. Stakeholders     
  15 15 20 20 
4. Users and uses     
5. Drinking water quality standards     
6. Schematic/flow diagrams     
7. System description     
8. Risk assessment methodology     
9. Hazardous events     
10. Raw risks     
11. Control measures     
12. Residual risks     
13. Significant risks     
14. Improvement plan link to risks     
15. Improvement plan details     
   41 60 20 14 
16. Operational monitoring plan     
17. Corrective action     
18. Compliance monitoring plan     
19. Internal auditing plan     
20. External auditing plan     
21. Consumer satisfaction monitoring plan     
   28 30 20 18 
22. SOPs – Normal     
23. SOPs – Incident     
24. SOPs – Emergency     
25. Supporting program description     
26. Supporting program plan     
   25 25 20 20 
10 27. Periodic review plan     
11 28. Revision plan     
   10 20 
Total   111 111 140 100 76 
StageStepOutput indicatorScore per output indicatorTotal score per stage (A)Maximum possible score per stage (B)Equal weights (%) (C)WSP Index (%) (D)
1. Management support     
2. WSP team     
3. Stakeholders     
  15 15 20 20 
4. Users and uses     
5. Drinking water quality standards     
6. Schematic/flow diagrams     
7. System description     
8. Risk assessment methodology     
9. Hazardous events     
10. Raw risks     
11. Control measures     
12. Residual risks     
13. Significant risks     
14. Improvement plan link to risks     
15. Improvement plan details     
   41 60 20 14 
16. Operational monitoring plan     
17. Corrective action     
18. Compliance monitoring plan     
19. Internal auditing plan     
20. External auditing plan     
21. Consumer satisfaction monitoring plan     
   28 30 20 18 
22. SOPs – Normal     
23. SOPs – Incident     
24. SOPs – Emergency     
25. Supporting program description     
26. Supporting program plan     
   25 25 20 20 
10 27. Periodic review plan     
11 28. Revision plan     
   10 20 
Total   111 111 140 100 76 

Sample computation for Stage 1:

Sample computation for WSP Index:

.

Table 6

Sample computation of WSP Index for Water District B using simple weights

StageStepOutput indicatorScore per output indicatorTotal score per stage (A)Maximum possible score per stage (B)Simple weights (%) (C)WSP Index (%) (D)
1. Management support     
2. WSP team     
3. Stakeholders     
   15 15 11 11 
4. Users and uses     
5. Drinking water quality standards     
6. Schematic/flow diagrams     
7. System description     
8. Risk assessment methodology     
9. Hazardous events     
10. Raw risks     
11. Control measures     
12. Residual risks     
13. Significant risks     
14. Improvement plan link to risks     
15. Improvement plan details     
   41 60 43 30 
16. Operational monitoring plan     
17. Corrective action     
18. Compliance monitoring plan     
19. Internal auditing plan     
20. External auditing plan     
21. Consumer satisfaction monitoring plan     
   28 30 21 20 
22. SOPs – Normal     
23. SOPs – Incident     
24. SOPs – Emergency     
25. Supporting program description     
26. Supporting program plan     
   25 25 18 18 
10 27. Periodic review plan     
11 28. Revision plan     
   10 
Total   111 111 140 100 80 
StageStepOutput indicatorScore per output indicatorTotal score per stage (A)Maximum possible score per stage (B)Simple weights (%) (C)WSP Index (%) (D)
1. Management support     
2. WSP team     
3. Stakeholders     
   15 15 11 11 
4. Users and uses     
5. Drinking water quality standards     
6. Schematic/flow diagrams     
7. System description     
8. Risk assessment methodology     
9. Hazardous events     
10. Raw risks     
11. Control measures     
12. Residual risks     
13. Significant risks     
14. Improvement plan link to risks     
15. Improvement plan details     
   41 60 43 30 
16. Operational monitoring plan     
17. Corrective action     
18. Compliance monitoring plan     
19. Internal auditing plan     
20. External auditing plan     
21. Consumer satisfaction monitoring plan     
   28 30 21 20 
22. SOPs – Normal     
23. SOPs – Incident     
24. SOPs – Emergency     
25. Supporting program description     
26. Supporting program plan     
   25 25 18 18 
10 27. Periodic review plan     
11 28. Revision plan     
   10 
Total   111 111 140 100 80 

Sample computation for Stage 1:

Sample computation for WSP Index:

.

Table 7

Sample computation of WSP Index for Water District B using AHP weights

StageStepOutput indicatorScore per output indicatorTotal score per stage (A)Maximum possible score per stage (B)AHP weights (%) (C)WSP Index (%) (D)
1. Management support     
2. WSP team     
3. Stakeholders     
   15 15 10 10 
4. Users and uses     
5. Drinking water quality standards     
6. Schematic/flow diagrams     
7. System description     
8. Risk assessment methodology     
9. Hazardous events     
10. Raw risks     
11. Control measures     
12. Residual risks     
13. Significant risks     
14. Improvement plan link to risks     
15. Improvement plan details     
   41 60 40 27 
16. Operational monitoring plan     
17. Corrective action     
18. Compliance monitoring plan     
19. Internal auditing plan     
20. External auditing plan     
21. Consumer satisfaction monitoring plan     
   28 30 30 28 
22. SOPs – Normal     
23. SOPs – Incident     
24. SOPs – Emergency     
25. Supporting program description     
26. Supporting program plan     
   25 25 15 15 
10 27. Periodic review plan     
11 28. Revision plan     
   10 
Total   111 111 140 100 81 
StageStepOutput indicatorScore per output indicatorTotal score per stage (A)Maximum possible score per stage (B)AHP weights (%) (C)WSP Index (%) (D)
1. Management support     
2. WSP team     
3. Stakeholders     
   15 15 10 10 
4. Users and uses     
5. Drinking water quality standards     
6. Schematic/flow diagrams     
7. System description     
8. Risk assessment methodology     
9. Hazardous events     
10. Raw risks     
11. Control measures     
12. Residual risks     
13. Significant risks     
14. Improvement plan link to risks     
15. Improvement plan details     
   41 60 40 27 
16. Operational monitoring plan     
17. Corrective action     
18. Compliance monitoring plan     
19. Internal auditing plan     
20. External auditing plan     
21. Consumer satisfaction monitoring plan     
   28 30 30 28 
22. SOPs – Normal     
23. SOPs – Incident     
24. SOPs – Emergency     
25. Supporting program description     
26. Supporting program plan     
   25 25 15 15 
10 27. Periodic review plan     
11 28. Revision plan     
   10 
Total   111 111 140 100 81 

Sample computation for Stage 1:

Sample computation for WSP Index:

Consult WSP experts and stakeholders

Filipino WSP experts from the DOH, LWUA, and the University of the Philippines and other stakeholders of selected water utilities and agencies were consulted to gather their insights and comments on the selected indicators, weights of WSP stages, formula of the WSP Index, and the minimum benchmark of the index for WSP acceptability. All elements of the WSP Index were accepted, except for the weights wherein adjustment to rounded figures was suggested (Table 2). This concept of WSP Index can be further enhanced even for global application by getting the inputs of international WSP experts on similar WSP elements considered in this study.

RESULTS AND CONCLUSION

WSP Index formula

The derived WSP Index formula is composed of WSP output indicators, WSP step indicators, WSP stage indicators, and the established weights using AHP. After determining the weights per WSP stage (Table 2) and the maximum possible scores for WSP output indicators (Table 1), the final WSP Index formula is 
formula
This equation was developed using local experts' input on weights and number of output indicators which may vary in other countries depending on the decision of their local WSP experts.

WSP Index computation

The 25 WSPs were evaluated using the WSP Index formula. About 21 of 25 WSPs were found to meet the benchmark of 60%. However, when the benchmark was raised to 74% after the sensitivity analysis, only nine WSPs passed. Specific WSP steps that received low scores were steps 1, 4, 5, 7, 8, 9, 10 and 11. Reasons for low scores are due to the incomplete data submitted and poor understanding of the risk assessment methodology. Incomplete data can be addressed by actual visits to drinking-water service providers. Some available information missed in the WSP submission could be gathered and validated on site to improve the scores.

CONCLUSION

The development and testing of the WSP Index was able to determine the overall status of the selected WSPs and provide a basis for DOH approval upon meeting the established benchmark. The scoring system can serve as a checklist to determine the completion of DOH requirements per WSP step and identify the gaps in understanding WSP concepts. Scores may improve if actual visits to drinking-water service providers could be conducted to gather and validate the readily available information. This consideration will justify the WSP acceptance benchmark of 74% instead of 60%. Other countries may adopt the Philippine model of developing the WSP Index which may have variations in the index equation due to local experts' inputs on weights and indicators. The index model can be further enhanced by its application in other countries provided there is sustained collaboration and input from international WSP experts.

REFERENCES

REFERENCES
Davison
A.
Howard
G.
Stevens
M.
Callan
P.
Fewtrell
L.
Deere
D.
Bartram
J.
2005
Water Safety Plans: Managing Drinking-Water Quality from Catchment to Consumer
.
World Health Organization
,
Geneva, Switzerland
.
Gunnarsdottir
M. J.
2012
Safe Drinking Water: Experience with Water Safety Plans and Risk Factors
.
PhD dissertation
,
Faculty of Civil and Environmental Engineering, University of Iceland
.
Gunnarsdottir
M. J.
Gardarsson
S. M.
Elliott
M.
Sigmundsdottir
G.
Bartram
J.
2012
Benefits of water safety plans: microbiology, compliance, and public health
.
Environmental Science & Technology
46
,
7782
7789
.
Jacobs
R.
Smith
P.
Goddard
M.
2004
Measuring Performance: An Examination of Composite Performance Indicators
.
A report for the Department of Health, CHE Technical Paper Series 29
,
Centre for Health Economics, University of York
,
United Kingdom
.
Metge
C.
Chateau
D.
Prior
H.
Soodeen
R.
De Coster
C.
Barré
L.
2009
Composite Measures/Indices of Health and Health System Performance
.
University of Manitoba
,
Canada
.
National Water Resources Board
2015
Listahang Tubig; A National Water Survey
.
http://listahangtubig.cloudapp.net/ (accessed November 2016)
.
OECD
2008
Handbook on Constructing Composite Indicators: Methodology and User Guide
.
Organisation for Economic Co-Operation and Development, OECD Publishing
,
Paris, France
.
Saaty
T. L.
2008
Decision Making for Leaders: The Analytic Hierarchy Process for Decisions in a Complex World
.
RWS Publications
,
Pittsburgh, Pennsylvania, USA
.
Sharpe
A.
Smith
J.
2005
Measuring the Impact of Research on Wellbeing
.
Centre for the Study of Learning Standards
,
Ottawa, Canada
.
WHO
2004
Guidelines on Drinking Water Quality
,
Third Edition, Volume 1 Recommendations
.
World Health Organization
,
Geneva, Switzerland
.
WHO
2009
Water Safety Plans: Managing Drinking-Water Quality from Catchment to Consumer
.
World Health Organization
,
Geneva, Switzerland
.
WHO
2011
Guidelines on Drinking Water Quality, Fourth Edition, Volume 1 Recommendations
.
World Health Organization
,
Geneva, Switzerland
.
WHO
2012
Water Safety Plan Quality Assurance Tool
.
World Health Organization
,
Geneva, Switzerland
.
WHO
2014
Water Safety in Distribution Systems
.
World Health Organization
,
Geneva, Switzerland
.
WHO
2016
A Practical Guide to Auditing Water Safety Plans
.
World Health Organization
,
Geneva, Switzerland
.