Strengthening country-led water and sanitation services monitoring and data use for decision-making: lessons from WaterAid experience in four countries

There is a growing recognition that universal access to water, sanitation, and hygiene (WASH) services will only be realised through a system-wide approach (Hollander et al. 2020; Valcourt et al. 2020) to drive progress in WASH systems and ensure a sustainable realisation of the human rights to water and sanitation. The human rights to water and sanitation state that these services should be available and accessible to all, be safe, of acceptable quality and affordable, and offer privacy and dignity to users (UN 2010). Underpinning their achievement are the principles of non-discrimination and equality, access to information and transparency, participation, accountability, and sustainability. Governments have an obligation to progressively realise these rights, underpinned by human rights principles. Monitoring processes play a key role in the progressive realisation of rights by ensuring the availability of information and data use for effective and evidence-based planning, budgeting, service improvement, and accountability. Timely availability of reliable WASH data is essential to proper service planning and decision-making; ‘with accurate data on who has access to water and sanitation, and at what level of service, States can prioritise the provision of services to the people who need them the most’ (de Albuquerque 2014, p. 22). At the same time, rights holder participation in monitoring and data use is key to improving accountability between service users and service providers, and cycles of continuous learning, adaptation, and improvement (Da Silva Wells et al. 2013; Kempster 2020).

In WASH, as well as the provision of many other public services, there is a trend in developing countries towards greater decentralisation of power and responsibilities for planning, budgeting, and service provision (Faguet 2014; Gadenne & Singhal 2014). In theory, making decisions about services closer to the local level increases accountability and greater responsiveness (Faguet 2014; Gadenne & Singhal 2014). While the literature on decentralisation and governance highlights the political nature of decision-making (Giné Garriga et al. 2015; Kempster 2020), it seldom refers to the availability and use of data to ensure well-informed and evidence-based decision-making by local leaders or for rights holders and civil society to hold local leaders to account for their decisions.

Efforts and investments to improve and strengthen national WASH monitoring systems and data availability have become increasingly prevalent in the sector (Schouten & Smits 2015), accelerated by the increasing availability and use of Information Communication Technologies (ICTs) and the increased possibility of data generation at all levels (World Bank 2018). Much of the literature on WASH monitoring systems focus on such technical aspects of data production as robustness, timeliness, scale, communication, and participation (Da Silva Wells et al. 2013; Cronk et al. 2015; Giné Garriga et al. 2015). One practical framework which sets out guidance for WASH agencies to assess WASH Monitoring and Evaluation (M&E) systems and design interventions to strengthen them is the ‘12 components of the National WASH M&E system’ (Dickinson 2016). This framework outlines factors and conditions required for an effective WASH monitoring system, based on UNAIDS organising framework for a functional national HIV M&E system (UNAIDS 2008). The 12 components represent a useful checklist of components, and helpful framing of desirable performance results in a well-functioning national WASH M&E system. Less analysis exists of factors that lead to better data use in the sector. Research reviewing the factors for successful ICT-based water supply services reporting (Welle et al. 2015; Williams et al. 2016) identified some of the key conditions necessary for the successful use of data, including service providers’ leadership of the initiative, clarity of service provider responsibility, and available financing to respond to water supply data; and overall, the need to integrate data generation initiatives as part of larger service delivery reforms to improve service providers accountability. Kempster (2020) presents a WASH data-informed decision-making framework guidance that takes into consideration local political economy and behavioural factors.

Despite the more extensive literature on the technical aspects of WASH monitoring systems, there is limited documented evidence of success in WASH data generation and use to support decision-making, and consequently, there is limited analysis and literature of the main driving factors/conditions that can lead to successful improvements to monitoring processes and data use. This paper aims to respond to this literature gap, drawing on experiences from WaterAid's approach and work to strengthen WASH M&E systems in four countries: Uganda, Myanmar, Papua New Guinea, and Cambodia, and focusing on the analysis of driving factors and activities that gradually lead to transformations in WASH monitoring and data use. While the ultimate goal of WaterAid's work to strengthen government-led WASH M&E systems is to achieve strong, transparent, and data-driven decision-making, it should be acknowledged that the strengthening process is not linear and the interventions are based on different starting points for each country case study.

WaterAid's approach to strengthening sector monitoring acknowledges the need to go beyond the production of data and its analysis and instead work within the wider system to strengthen institutions, processes, behaviours (Hallsworth et al. 2018), factors, and actors, including the complex interactions required to enhance the use of data for decision-making (Kempster 2020). To identify key driving factors impacting improvements to data availability and use, WaterAid established a review team that had worked on strengthening WASH sector monitoring in systems in four countries. The country case studies were selected to present geographic and intervention type diversity. A summary of the background, context, and activities is shown in Table 1.

To conduct the review of each country's case study, the WaterAid review team developed a structured analytical framework based on the ‘From Data to Decisions’ policy brief and data use planning guide developed by ODI (Overseas Development Institute) and WaterAid (Kempster 2020) and the 12 components framework (Dickinson 2016). The structured analytical framework (Figure 1) was designed to review two main aspects of the monitoring system interventions:

• • Intervention activities for each factor: Key strengthening activities were identified for each factor (see Supplementary Material, Appendix 1), drawing on areas and activities included in the 12-component framework (Dickinson 2016).

• • Contribution of factors towards continuous government monitoring and data use outcomes: The perceived contribution of each factor in driving success towards the following four key outcomes of successful WASH monitoring and data use at a national level:

• • 1. Coordinated monitoring processes and tools are used in the sector.

  • 2. Relevant, timely, and quality data are available.

  • 3. Decision-makers use data for tracking progress, planning, and budgeting.

  • 4. Informed decision-making impacts WASH services provision, sustainability.

Within the framework, the driving factors were grouped into four categories, in line with categories from Kempster (2020), and defined as follows:

• a.

  Data needs (Data)

• b.

  Processes and technology (Processes)

• c.

  Institutional settings (Context)

• d.

  People and Drivers (People)

A rating scale assessed the extent to which each factor was perceived to contribute to the successful achievement of each of the outcomes of continuous monitoring and data use listed above (including four scorings: no contribution to change observed (0), factor provides a small contribution to realisation of change (1), factor strongly contributes to realisation of change (2), and essential (primary) contributing factor to realisation of change (3).

Quantitative responses to each of these rating scales have been complemented by qualitative explanations to justify the rating selected. While it is acknowledged that other systematic factors also influence realisation of outcomes, particularly for data use, the scoring developed aimed at identifying which strengthening activity has had a larger impact on different outcomes within each country's context, to inform future intervention design and prioritisation.

The method to review the four case study interventions included:

• • Development of the analytical framework (summarised in Figure 1 and more detail in Supplementary Material, Appendix 1).

• • Testing and refinement of the analytical framework based on review team's own experiences and existing documentation of the monitoring and data use interventions.

• • Population of the framework analysis templates through in-depth interviews in July 2021 with at least one government counterpart from each of the four-country contexts who were selected based on their role as focal point or coordinator for the government WASH monitoring system. These interviews involved:

• • 1. The review team explains the analytical framework, and terms and definitions.

  • 2. Discussion of each of the activities completed for each factor as part of the monitoring system intervention; quantitative assessment using a rating scale of the extent to which each activity was fully, partially, or not at all included in the intervention to strengthen the WASH M&E system; and a qualitative explanation of the rating.

  • 3. Discussion of each of the four monitoring and data use outcomes; quantitative assessment using a rating scale of the extent to which each factor was perceived to contribute to the successful achievement of each of the outcomes; and a qualitative explanation of each rating.

• • Normalisation of the ratings across the four case studies to facilitate cross-country comparison and facilitate cross-country comparison.

The results from the analysis in each of the four case studies are as follows:

Table 2 presents the relative contribution of each of the factors (from the sum of the contribution of three factor activities) towards the overall four outcomes. This supports identifying the key factors and associated strengthening activities that had a larger impact on the overall outcome of improved data production and use.

Table 3 presents the cumulative assessment of all factors towards the monitoring and data use outcomes for each of the case studies. This supports the identification of what outcome was mostly impacted by the strengthening activities analysed.

In Cambodia, standardisation of monitoring processes and availability of data through the Management Information System (MIS) has facilitated a culture of regular upwards reporting and aggregation of data which has significantly improved progress tracking against sector plans and enabled inter-province comparison. However, there is still limited evidence of the MIS data's use for provincial or district WASH service planning and budgeting.

The following factors have contributed significantly to the successful establishment of the Cambodia MIS, and were among the highest scores in Table 2.

• Institutionalisation: The Cambodian government, through the Ministry of Rural Development (MRD), demonstrated strong ownership and leadership of the MIS process from the beginning. The leadership of the MIS development was consultative with the wider sector but was embedded within existing government roles and responsibilities at national and provincial levels through an official Prakas (Regulation) which encouraged upward accountability. In addition to formalised responsibilities, the MIS working group developed clearly defined annual work plans, flow charts, and procedures to operationalise the MIS within the mandate of the MRD.

• Data analysis and dissemination: Annual analysis of data, sharing of summaries, and creation of dashboards for sub-national government to extract relevant indicators helped to deepen the understanding and engagement of MIS stakeholders and increased the utility of the data for progress monitoring at national and sub-national levels. Annual reflection workshops were held to disseminate the results and collectively reflect on the sector's progress, as well as gather feedback to refine the MIS itself.

• Monitoring framework development: The MIS is the system for monitoring progress against the indicators already identified in the National Action Plan (NAP) for Rural WASH. The need to report progress against the NAP's monitoring framework was a strong motivation for establishing the MIS. The second phase of the NAP was developed concurrently with the MIS, and lessons learned from using the MIS to monitor NAP progress informed the selection and short-listing of indicators in the second NAP.

• Indicators: The process of harmonising indicators helped bring together government and non-government actors to build sector coordination and consensus. Prioritising a manageable list of indicators also helped to encourage timeliness in the data gathering processes and build trust in the data quality.

• Data culture: The Minister of Rural Development's support for the MIS and creation of a government core group to lead the MIS contributed to prioritisation of this intervention and gave weight to coordination and tool development. Annual training and orientation on indicators, data collection, and interpretation of results increased understanding of data management and use at national and sub-national levels and helped planners and implementers to see how the MIS could help monitor progress against the NAP.

Three factors/activities received particularly low scores in Table 2, and represent significant risks to the continued sustainability of the MIS in Cambodia:

• Process financing: While government staff, employed by MRD, form the backbone of the MIS, the costs associated with travel, workshops, consultants for developing ICT dashboards and printing of documents have been largely borne by non-governmental organisations (NGOs) and development partners, and advocacy for allocation of government funding to cover these costs has been limited.

• Understanding of decision-making: the absence of structured or systematic assessment of fiscal, administrative and political data use is likely a key factor for the continued limited use of MIS data for (service) planning and budgeting. Provincial WASH Working Groups, established at the same time as the MIS and led by provincial political leaders, are not yet routinely drawing on MIS data for their annual planning and budgeting, likely because of the service level data they need for such processes is not yet available within the MIS. Significantly, the decentralisation of responsibilities for rural WASH in Cambodia occurred during the establishment of the MIS. District administrations, who have not been involved in the development or use of the MIS to date, now have additional decision-making responsibilities for WASH and the MIS may not be meeting their data needs.

• Community involvement: The MIS development has initially focused on how data can be made available for WASH planning and performance monitoring but with minimal consideration or involvement of community and service users in the process. This represents a risk by not considering the data needs of users or data use for accountability from the system's inception.

The understanding of the decision-making: factor mentioned above is one reason for the relatively limited progress observed towards the outcome of data use for WASH service improvement among the four outcomes. Without a strong understanding of how data are, and could be, used for decision-making and integrated with planning and budgeting, several other factors which would be expected to contribute strongly to data use also rated poorly on contribution to this outcome such as understanding the individual motivations and incentives for data use, and allocation of funding and technical resources to the monitoring focal points. In addition, the indicators being collected through the Cambodia MIS until now have mostly focused on performance measurement (e.g. number of villages receiving Community-Led Total Sanitation (CLTS), number of MRD staff participating in training) and while they provide a good summary at national level of progress across the NAP indicators, data on service access or quality indicators which would be most useful for service provision and sustainability decision-making (e.g. household access to sanitation, or water supply functionality) are still being integrated in the current phase of the MIS.

As in the other case studies, the strongest outcome progress in the Cambodia case study was towards the coordination of sector monitoring tools and processes and relevant, timely availability of quality data (Table 3). The factor ratings (Table 2) indicate this is predominantly for the following two reasons:

• (a)

  The MIS was primarily established to fill a gap identified in the review of the first National Action Plan for rural WASH, in consistent processes and mechanisms for data collection, analysis, and reporting between different provinces or at the national level.

• (b)

  The mechanism through which the MIS was developed and refined, a government-led working group with the active participation and buy-in of key non-government sector actors had the effect of strengthening coordination and sharing knowledge between sector actors.

The Department of Rural Development from the Government of Myanmar identified the need to strengthen the national rural water supply monitoring as a key step toward progress monitoring and informed planning towards the achievement of Sustainable Development Goal (SDG)6. Key drivers for change included the department leadership and clear progress vision, a keen technical savvy government team driven by career progression, and a goal to adopt technology innovation to improve data availability, access, and sharing. The process included co-designing with the government at the local and national level, and adapting regularly to feedback and changes required.

Through the analysis of the interventions that led to the strengthening of the monitoring system and data use (Table 2 and Supplementary Material, Appendix 2), we can identify the key factors and associated strengthening activities leading to progress and successful outcomes, in particular, the outcomes related to coordinated monitoring and availability of relevant, timely and quality data;

• Indicators definition and harmonisation: the collaborative process of introducing, testing, and adapting new sector best practice indicators for rural water supply monitoring, including Joint Monitoring Programme (JMP) SGD6 monitoring indicators, and World Bank rural water supply sustainability indicators (World Bank 2017) led to a step change into governmental and sectoral shifting from infrastructure and service delivery reporting to water supply service levels monitoring in line with SDGs. Furthermore, the sector-wide consultation for indicators reviews and harmonisation with other WASH stakeholders was key to engaging stakeholders in the use of a common monitoring indicators framework (for example, UNICEF fully adopted the government led rural water supply indicators and data collection tool).

• Data analysis and dissemination: WaterAid co-designed with government staff the data analysis approach through the selection of appropriate data visualisations (maps, charts, tables) summarising key indicators data to respond to different information needs and capacity for data interpretation at local and national levels.

• Introduction, piloting, and adapting of ICT and locally appropriate technology: in Myanmar, the adoption of ICT-based technology had progressed rapidly, however mostly for private use rather than to support government-led processes. By introducing and piloting a freely available ICT platform that was accessible and usable with local government's own phones without any modification or investment, it was possible to demonstrate to national and local governments how to leverage existing technology to accelerate data collection and real-time analysis. Access to real-time data and its analysis was seen as an essential step change, particularly by the national government.

• Mapping data needs at different levels and for different uses at the beginning of strengthening activities supporting the generation of a data culture (through regular trainings, demonstrations) were also a key driver for achievement of outcomes.

It is instead observed that some factors and strengthening activities, such as the creation of roles and responsibilities and routine monitoring framework, which we would expect to strongly contribute to outcomes such as ‘decision-makers use data for tracking progress, planning and budgeting’ did not yet (in the period analysed) contribute to the outcomes, particularly 3 and 4. This is however due to the short period of intervention analysed (2 years) and progress slowed down the process due to political changes.

Furthermore, the scale-up and adoption of the indicators and ICT-based data collection approach introduced was facilitated by the top-down political model, while the limited outcomes related to data used to support local level planning and budgeting were also limited by the weak decentralisation of WASH planning and budgeting processes in Myanmar. This is due to the historical politically centralised management and weak decentralisation reforms. Quality data access improvement was acting as an entry point for a wider reform towards decentralised planning and budgeting processes. While a data culture has been growing within the DRD teams, additional work is required to shift the culture of ‘reporting upwards’ to data use for improvements to planning and budgeting particularly at subnational level.

Identification of both political champions and technical experts within government and the co-designing and adaptive approach (including regular reviews and changes to the system proposed) together with the generation of a common vision, with government ownership but also encouraging sector coordination, discussion and engagement in the process have been key to ensure suitability of processes and development of a system that responded to data needs at local and national level.

The lack of effective government monitoring in Papua New Guinea (PNG) is a significant barrier to the WASH sector. Until recently, the Department of National Planning and Monitoring (DNPM) did not have nationally agreed to WASH indicators with definitions. Both at the national and subnational level the roles and responsibilities for data collection, processing, management, and use were poorly defined. Recently, progress towards establishing an effective national WASH monitoring system has been driven by national level leadership and the requirement for tracking process towards the National WASH Policy (DNPM 2015) and Medium-Term Development Plan 3 (MTDP3) (DNPM 2018) targets. A national level technical working group on WASH monitoring was established. Leading a co-design and consultation process to map and agree with data and decision-making needs of government, development partners and NGOs. A WaterAid staff member was formally engaged through a consultant contract as a National M&E advisor within DNPM to document data needs, indicators, and definitions through the development of the DNPM National WASH M&E framework, monitoring manual, development and enhancement of the WASH MIS and supporting subnational data use through District planning processes and the development of 5-year WASH investment plans.

Through the analysis of the interventions that led to the strengthening of the PNG monitoring system and data use (Tables 2 and 3), we can identify key factors and associated strengthening activities that led to progress towards all four outcomes areas. In PNG, several factors were the key to the enhancement of WASH M&E systems. These gains were driven primarily by five key areas:

• Sector coordination groups: the formation of a national level technical working group chaired by DNPM and brought together the key government departments, development partners, and NGOs to map data needs and decision-making processes of government and non-government stakeholders. While experiences of the working group at a subnational level were considered in the process consultation on data needs with the District government was only possible in eight pilot districts out of 89 total districts across the country.

• Indicator definitions, harmonisation: WaterAid with the Government of PNG and the TWG undertook a collaborative process to document key and non-key indicators, definitions, and their intended uses. DNPM adopted SDG indicators and extended indicators for subnational planning and finance processes and formalised the indicators with the National WASH M&E framework (DNPM 2020).

• Monitoring framework development: a collaborative process was led by WaterAid to establish PNG's first national WASH M&E framework, outlining key indications, definitions and standard data collection forms. The development of the M&E framework led to enhancement of sector coordination, notably key WASH sector stakeholders adopting standardised data collection formats aligned to national WASH policy, MTDP3 and providing effective data for subnational planning and budgetary processes.

• Demonstration of appropriate ICT systems to establish national level WASH MIS: the PNG WASH policy provides a framework for action at the District level, through key development partner programmes, several INGOs partnered with District Development Authorities across the country to systematically gather District-wide WASH data. Three key development partner programmes (World Bank, DFAT, and EU/UNICEF) were used to pilot national data collection forms. Data were collected by government and INGOs via the mWater system from 12 districts across the country with data and indicators visualised on a common dashboard at the national level. Based on the pilot, DNPM formally adopted the system and assigned staff to progressively administer and manage the system. The DNPM National WASH MIS has driven progress towards the availability of relevant quality and timely data.

• Data Analysis and Dissemination: data analysis and dissemination can be problematic in low resource settings such as rural PNG. The WASH MIS automatically processes analysis based on the national indicators and has been key to achieving data use by District level government. The WASH MIS data have been used by decision makers at District level to formulate 5-year WASH investment plans highlighting targets, required financing and roles and responsibilities for service improvement.

The analysis of the outcomes (Table 3) indicates that the intervention of strengthening WASH monitoring in PNG resulted in notable gains in all four outcome areas while noting that most support has been provided in outcome areas 1 and 2. While ‘coordinated monitoring processes and tools’ has the most impact, it should be noted that the most energy and resources were focused on this area. Efforts to enhance coordination for development partners to engage in collective monitoring processes, develop and formalise indicators and data collection forms and use a common system were found to be a viable pathway to accelerate progress.

While efforts have resulted in progressive improvement to relevant, timely and quality data and the formation of an MIS to visualise and use incoming data, the approaches have drawn largely on 12 districts where INGOs are closely partnered with District Development Authorities. While DNPM have plans to scale the approaches across the country, analysis is based on experiences from the 12 districts and significant efforts are needed to institutionalise monitoring processes, particularly in districts with less development partner support.

The PNG National WASH policy provides a framework for the development of 5-year WASH investment plans at the district level. The key to success has been linking data collection processes within the National M&E system to District level planning and budgeting processes. The resulting approach has encouraged subnational leadership and fostered transparent decision-making through the formation of district WASH committees for planning and budgeting processes. This has been observed in eight of the 12 districts that have contributed data to the WASH MIS.

It is noted that WASH monitoring finance has scored poorly in PNG. It should be noted that within the phasing for M&E system rollout DNPM is working to strengthen WASH MIS systems and processes and rollout the system throughout the country. This includes supporting the subnational government to allocate financial and human resources for recurrent monitoring.

To date, the government of PNG has not established community or citizen reporting mechanisms within the national WASH MIS. Due to limited resources, existing efforts have focused on institutionalising the system within the public service. Access to WASH services is low in many parts of the country and service providers such as District Development Authorities are poorly resourced to respond to community demands, therefore there can be a reluctance to prioritise community reporting. This may be prioritised in the future as the MIS continues to evolve and strong systems and processes are built.

In Uganda, the coordinated monitoring processes and tools are used in the sector and relevant, timely and quality data are available outcomes were achieved more consistently across different interventions but also contribute to Informed decision-making. WaterAid, in partnership with the Ministry of Health (MoH), Ministry of Water and Environment, WHO, UNICEF, Makerere University, and Emory University among other stakeholders, facilitated the development of a national measurement framework for WASH in healthcare facilities (HCFs) and in communities. JMP service level indicators were localised for communities and HCFs and these have been integrated in the MoH-HMIS revised in 2018. A national data collection tool was also developed by contextualising the Emory University's WASH-Con Tool which included systems strengthening indicators around planning and budgeting for WASH in HCFs. The standardised indicators have since been widely adopted nationally and used by MoH, UNICEF, WaterAid, Kampala Capital City Authority (KCCA), and the Ministry of Water and Environment to collect national data on the status of WASH in HCFs. These data have then resulted in the development of national guidelines for WASH in HCFs. Furthermore, WaterAid Uganda has continued to support the KCCA indicators harmonisation process for WASH in schools, by integrating JMP WASH in school core questions and supporting updating the city-wide online database for WASH in public primary and secondary schools covering 79 primary schools and 22 secondary schools in Kampala. This capacity support initiative has helped KCCA map out schools with different WASH needs and is informing financial investment from both public funds and allocation of development partner support including NGOs. It also has improved information sharing and use among the different stakeholders in terms of timely data generation and reporting.

Based on Table 2, in Uganda, the factors that have had largest impact on outcomes included:

• Sector coordination groups: The MoH has continued to convene the national sanitation working group that provides a platform for discussing cross-sectoral coordination and integration of WASH data for planning and decision-making. By partnering with the Water Resources Institute of the Ministry of Water and Environment WaterAid has drafted the development of a Training manual on SDG6 to among other objectives define roles and responsibilities in monitoring SDG6 targets and indicators.

• Roles and responsibilities: the definition of clear roles at different levels has been key for the routine generation of data and its use. From data collection at community level, data management and analysis at health centre level and aggregation at national level where data are used to inform policies and planning processes.

• Data analysis and dissemination: the support towards improved data analysis and dissemination led to increased data access and use outcomes. WaterAid supported the building capacity of civil society organisations coordinated under the Uganda Water and Sanitation NGO Network to proactively contribute to data availability compiled and disseminated during the annual Joint Sector Reviews (JSR) as part of the Sector Performance Report. The JSR platform has been used to disseminate data and inform data-based cross-sectoral policy discourse and decision-making on priority undertakings for government and non-state actors to focus on.

• Technical expertise: the technical support in introducing and demonstrating best practice indicators in line with SDG service levels indicators has led to an increased understanding of WASH service levels and the use of this data to inform planning processes, moving from coverage data use only to consideration for service levels (reliability, accessibility, quality, etc). This was also complemented by support on use of digital tools for data collection and analysis which were rapidly adopted by government staff.

• Indicators definition and harmonisation: There process of sector wide indicators harmonisation across line ministries aimed at alignment with the global SDG commitment on universal access and JMP indicators, particular with HMIS indicators on health care facilities and sanitation, has led to increased coordination across the sector and increased availability of consistent WASH data.

• Each case study presented had different starting points, in terms of existing monitoring processes and frameworks, drivers, data needs, and data use scope limiting the full comparability. In Myanmar, there was no previous presence of a rural water supply service levels monitoring system with only upwards donor-led or large investment reporting processes in place. Understanding of service level monitoring beyond functionality monitoring was low. In Cambodia, a clear monitoring framework had been established through the sector-agreed National Action Plan for rural WASH however there was no consistent monitoring system prior to the MIS; each province had developed its own templates, procedures, and indicators for monitoring, often aligned with the data requests of NGOs and development agencies. This variety of existing monitoring and data culture and processes leads to different approaches and impacts of strengthening activities and factors in the different case studies. For example, PNG MIS development was focused in 12 pilot districts while Cambodia was nation-wide, but the data gathered were more limited in scope compared to PNG.

• The analysis is limited to assessing the strengthening interventions and their impact. While the analysis is grounded in the political economy context of each of the case studies, it does not aim to analyse other possible processes and activities occurring outside the interventions which might have influenced the outcomes.

From the analysis of the four WaterAid's country experiences of strengthening government national monitoring systems, it was possible to identify activities (and associated factors) that have the largest impact across all case studies. These included the following (in order of score):

• Indicators: introduction of best practice indicators, including SDG6 JMP indicators, clear definition setting and support for the sector wide indicators harmonisation process.

• Data analysis and dissemination: development of targeted and locally contextualised and appropriate data analysis generation and sharing at different government levels and sector.

• Technical expertise: provision of technical training, and expertise on WASH services monitoring to national and subnational government staff.

• Sector coordination groups: developing and facilitating sector working group/committee to harmonise indicators and coordinate WASH monitoring processes.

• Technology (ICT) demonstration, piloting, and adapting to local context needs for data collection, management, and analysis.

• Data use culture: understanding existing data use, data needs at a different level and regular demonstration and advocacy on data use for planning, progress tracking, and policy adaptations.

• Mapping data needs: Mapping pre-intervention/previous data use, mapping data needs and gaps at different levels, identifying and aligning with sector priorities, policies, targets, and related indicators.

• Monitoring framework development: development of government WASH monitoring frameworks at the national level help to formalise and embed indicators within the public service. This provides consistency and lends support to the subnational rollout and institutionalisation of the monitoring system.

This leads to the interpretation that the above factors are viable and strategic entry points that contribute to accelerating progress towards strengthening and institutionalising coordinated government-led WASH monitoring processes. Efforts of development agencies like WaterAid to adapt and improve suitable data collection and analysis processes, introduce best practice indicators and support indicators harmonisation, map data needs, and demonstrate ICT systems through co-design processes with the government can have a direct impact and influence progressive improvement of government monitoring at scale. In particular, it can lead to improvements to sector coordination processes and data availability.

The WASH sector places a high emphasis on putting decision-makers at the forefront of data needs (Da Silave Wells et al. 2013; Dickinson 2016; Kempster 2020). Logically decision makers need quality, and timely data to effect evidence-based decisions, however, data use and decision-making processes are not always clear cut: data needs, decision-making processes, and responsibilities by government and service providers are diverse and evolve based on changes in context, political influence, personal incentives, and overall political economy. Priorities and data needs of national level stakeholders may not align with their subnational counterparts and therefore data analysis and visualisation need to reflect these different needs, even if based on the same dataset. In PNG, district-level government staff were found to have low policy awareness with very minimal awareness of SDG indicators and service levels. District level decision-makers such as local level government managers do not have formal processes or criteria for how a WASH project should be selected and to what standard of services should be delivered. Lacking this, priorities commonly align with the provision of basic services (for example, if a community had an existing rainwater tank or not). While the PNG WASH MIS displays SDG service levels, useful for national level policy tracking and reporting, this had little bearing on decision-making processes at the subnational level. Similarly, in Myanmar, national decision-makers identified SDG indicators and water service levels tracking as a key data need for progress comparability at an international level, while local government staff only identified the need to track key sub-indicators mostly associated with coverage and functionality of water supply, as this information aligned more clearly with their core responsibilities and targets.

From the analysis, we have observed that activities associated with strengthening institutional settings and people factors, including understanding and leveraging current data use, identification of stakeholders’ drivers, development of monitoring frameworks, have had a limited direct or immediate impact the four outcomes. On one hand, this is unsurprising as these represent the factors that are intrinsically linked to government functioning and therefore less likely to be influenced over short-term intervention by external agencies like WaterAid. However, these had an indirect impact on developing data and data use culture, for example, in the PNG case study, which made significant progress towards the intended outcome of data impacting WASH service provision and sustainability. Despite the limited direct impact, the case study narratives demonstrate that the successful strengthening of indicators, data collection, and analysis could only be effective if developed with an understanding and recognition of the wider system, its factors, formal and informal processes and with regular reviews and adaptations of approaches propose to the regularly changing political economy change. This is in line with sector recognition of the need to work within the wider system to strengthen institutions, processes, behaviours (Hallsworth et al. 2018) and that improvements to monitoring systems or introduction of ICT can only support improved services and accountability if coupled with wider reforms (Williams et al. 2016) linked to regulatory processes, management models, sector financing that impact how data can be used.

The culture and attitudes around data among WASH system actors, including community and civil society, are central to the use of data for improved service planning and budgeting, but also in making the step to fulfilling the human rights principle of accountability. In an effort to shift from a ‘reporting culture’, activities that aim to strengthen monitoring processes represent entry points towards the first step of the creation of a ‘data use culture’ in which data are not just reported but used for evidence-based decision-making, course correction and reducing inequality in the provision of services (Figure 2). As such, a sector culture of joint reflections, formal and informal networking and social learning encourage continuous learning, and use of data to manage WASH services adaptively (Da Silva Wells et al. 2013). The review team observed some progress in moving towards a ‘data use culture’ in the WASH systems in each of the four case studies. The progressive realisation from a ‘data use culture’ to a ‘data-based accountability culture’ in WASH is a longer process, but an essential shift through which monitoring and data availability contribute to the accountability mechanisms through which rights holders hold duty bearers to account for fulfilment of the rights to water and sanitation (de Albuquerque 2014). The shift from a ‘data use culture’ to a ‘data-based accountability culture’ is often dependent on the realisation of other institutional reforms, such as the implementation of regulatory processes, policy targets and requirements, leadership, clarity on management models and responsibilities, and decentralisation of planning and budgeting processes to the local level. It also requires strengthening of citizen capacity to understand monitoring indicators and their rights in relation to WASH, as well as avenues to integrate monitoring systems with mechanisms for accountability such as involving civil society rights groups in the development and strengthening of the monitoring system and facilitating data analysis and interpretation (Da Silva Wells et al. 2013; de Albuquerque 2014). Nevertheless, improved monitoring processes, quality and timely data and capacity to interpret and use data are necessary steps towards the greater accountability assumed by WASH decentralisation processes (Faguet 2014; Gadenne& Singhal 2014). Without improved monitoring and availability of data for both decision-makers, civil society, and rights holders, local leaders may use any conveniently available WASH data which may serve their political purposes rather than being accountable for service users’ satisfaction and realisation of human rights to water, sanitation, and hygiene without discrimination on the grounds of wealth, race, class, sex, religion or geography.

Often WASH monitoring literature advocates for the creation of a uniform national monitoring system, (Schouten & Smits 2015; Dickinson 2016). Through our case studies, we have found that the creation of uniform national monitoring systems needs to be balanced with co-designing appropriate approaches and phasing to strengthening WASH Monitoring with government stakeholders, reviewing indicator appropriateness, and understanding and responding to different data needs and feedback processes at the local level in order to avoid reinforcing a ‘reporting culture’ with top-down centralised monitoring processes reducing local level data ownership. In Cambodia, we observed that the creation of a national MIS to standardise and aggregate data for progress monitoring reinforced reporting culture from the provincial level to the national level. This system adequately met the progress-tracking needs of the national level (and likely provincial level), however, it did not necessarily provide the WASH data needed by district decision-makers to plan, prioritise, and budget for local WASH services once they received these responsibilities through decentralisation. By contrast, in Uganda the collaborative, inter-ministerial co-design of the national monitoring framework and data collection tool for WASH in HCFs prioritised application and data use for local-level planning and budgeting of WASH in HCFs, and influenced the updating of new standards, leading to its increased coordinated adoption and utilisation at all levels. In contrast, we observed that who collects data impacts how the data are used and interpreted; therefore, it is important to involve decision-makers (including from the national level) during data collection and validation processes, rather than fully delegating to local level staff. In Uganda, without government participation in data collection, data analysis findings cannot be officially reported or referenced.

Identification of political and technical champions within government staff at the national and sub-national levels has been key to driving change and progress towards improvements towards harmonisation of indicators and data availability in line the with literature (Da Silva Wells et al. 2013; Kempster 2020). In Myanmar, high level political leadership with a clear vision and top-down culture, combined with the presence of technical savvy young engineers driven by career progress and technical expertise, accelerated progress and scale-up of the newly introduced monitoring system (Figure 3). Likewise in Cambodia, strong government ownership and institutionalisation of the MIS benefited from the active engagement of political champions and their official allocation of responsibility for the MIS through a government regulation, combined with dedicated technical staff resources within the MRD.

• Community involvement has generally had a low impact, probably because all country case studies represented early stages of establishing Monitoring systems. As these are government owned monitoring systems, there needs to be a fully established data use by the government and sufficient resourcing and capacity to respond to low services and community demands. Seeking community review and feedback without government or service providers’ ability to respond can undermine the trust and people's willingness to continue supporting and operationalising the monitoring system.

• Financing: Again, as the case studies analysed present evidence from early-stage monitoring system development, effective long-term financing of monitoring systems was yet to be fully established. While in all contexts it was recognised as an essential step for the sustainability of the monitoring system, there was a recognition that the value of the monitoring system as an essential support to evidence-based decision-making still needed to be demonstrated in order to attract budget allocation funding.

Based on the experiences from the four case studies presented, we suggest the following implications and lessons learned for future efforts aiming to strengthen WASH monitoring systems:

• 1.

  Approach monitoring systems as part of the broader political economy: accurate, relevant and timely data are critical to effective decision-making and the performance of the WASH sector, however other factors influence and drive decision-making and data use. Decision-making processes are inherently political and vulnerable to political change; power dynamics, interests, and counterproductive incentives can trigger over/under reporting, while biased and selective data use, in the absence of rigorous data validation or standardised indicators, can undermine monitoring processes by eroding confidence and trust in the quality of data and its representativeness. Wider institutional arrangements, such as regulatory settings, decentralisation, and cross-ministerial coordination, can either promote or inhibit data-informed decision-making. Regular political economy analysis and mapping of informal and formal decision-making processes before and during activities to strengthen monitoring systems contribute to the ongoing evolution and long-term sustainability.

• 2.

  Co-designing with data users: user-centred co-design monitoring systems support meeting the data needs and technical capacity of formal and informal decision-makers, sector stakeholders, and rights holders themselves. Upfront identification of technical and political government champions is key to supporting this while adapting to continuous changes.

• 3.

  Engage with monitoring processes strategically: based on the analysis of the four case studies, strategic key entry points for agencies/actors aiming to strengthen government-led WASH monitoring processes includes efforts to strengthen:

• • • Mapping data needs: Mapping pre-intervention/previous data use, mapping data needs at different levels, identifying and aligning with sector priorities and related indicators.

  • • Indicators: introduction of best practice indicators, clear definition setting, and harmonisation process.

  • • Data analysis: development of targeted and locally contextualised data analysis generation and sharing at different government levels and sector.

  • • Technical expertise: provision of technical training and expertise to national and subnational government staff.

  • • Technology (ICT) demonstration, piloting, and adapting to local context need for data collection, management, and analysis.

  • • Sector coordination groups: Developing and facilitating sector working group/committee to coordinate WASH monitoring processes.

• 4.

  Data availability and use for planning and budgeting, while critical, will not by itself guarantee evidence-based accountability: the availability of relevant and timely data access can lead to improved WASH sector coordination processes (through forums, sector working group who use data to increase investment coordination, progress monitoring, etc) and lead to improved ‘data use culture’; however, it is not sufficient to achieve a ‘data-based accountability culture’ which also requires realisation of other institutional reforms in the sector, particularly related to regulatory processes and clear responsibilities.

• 5.

  Strengthen a sector culture of reflection, learning and adaptation: to ensure an effective shift of ‘data use culture’ into WASH accountability, activities to strengthen monitoring need to respond and adapt to changes in the dynamic political and institutional contexts. Fostering a culture of continuous learning, reflection and adaptation can help stakeholders respond to the inevitable changes in political drivers that define sector targets (and therefore indicators), institutional settings such as decentralisation processes (therefore responsibilities for decision-making) and key actors (e.g. turnover in political or technical government staff who are acting as catalysers). Engaging civil society rights holder groups early in the monitoring discussions might ensure data culture does not stop evolving once data are available for service providers’ use but continue to build momentum towards a culture of data-based accountability and rights holders using monitoring systems to hold duty bearers to account.

• 6.

  Frameworks developed to support planning and targeting of WASH monitoring strengthening activities can be useful to identify weaknesses and entry points for WASH monitoring strengthening: however, these would need to be complemented with political economy analysis that include analysis of roles and responsibility, drivers for data use, champions, etc. Furthermore, co-designing with government is key to develop effective monitoring processes.

• 7.

  Ensure recurrent financing for monitoring: financing of monitoring processes by government is still limited, with high dependency on supporting agencies. Sustainability of stronger WASH monitoring systems requires sectoral monitoring data to be integrated in core government public financial management functions. Significant advocacy is needed, based on demonstrated value of the monitoring system, to promote recurrent financial resource allocations by government.

Funding for WaterAid to support the monitoring systems interventions described in this paper was provided by the Water Supply and Sanitation Development Programme (a World Bank concession loan) and EU/UNICEF-funded KlinpelaKomunitiProjek (PNG), WaterAid UK (Myanmar, Uganda), and the H&M Foundation through the SusWASH Programme (Cambodia, Uganda).

All relevant data are included in the paper or its Supplementary Information.