E-learning for WASH systems strengthening: lessons from a capacity-building platform

Gaps in human resource capacity in the water and sanitation sector have been highlighted for at least the last three decades (Suter & Lüthi 2021) hindering progress in providing adequate water and sanitation services. A 2014 study by the International Water Association (IWA) highlighted that there can be no sustained progress in achieving Sustainable Development Goal 6 (SDG 6) – ensure the availability and sustainable management of water and sanitation for all – without effective capacity building and training of an adequate number of people (IWA 2014). The United Nations GLAAS report of 2019 (UN-Water 2019) showed that even where most countries report having policies for drinking water (94%), sanitation (94%), and hygiene (79%), less than 14% have sufficient human resources to implement these policies.

Water, sanitation, and hygiene (WASH) service delivery are provided through a series of interactions between social and ecological systems (Da Silva Wells et al. 2013). Given the complexity and socio-technical nature of WASH service delivery, systems thinking is a core competency (Gabrielsson et al. 2020). The capacity to understand such systems in a holistic manner is fundamental for engaging with the actors involved (Hollander et al. 2020; IHE-Delft 2020). Systems thinking helps to understand water, hygiene, and sanitation as a system, however, conventional training usually focuses on specific disciplines rather than multidisciplines, hence a holistic approach is required (Wiek et al. 2011). Enrolments in Massive Open Online Courses (MOOCs) globally have recently increased, making these an increasingly popular vehicle for building capacities.¹ As in other sectors, the COVID-19 pandemic has led to increased interest in online training in the water and sanitation sectors. MOOCs have the potential to make a substantial contribution to capacity development in the water and sanitation sectors, reaching larger audiences than other training modalities (Otto et al. 2019). They provide an unrestricted educational platform that offers free knowledge, tools, and experiences to learners globally with internet access (Johnson 2021; Siemens 2013).

This paper aims to evaluate the performance and application of a MOOC with the core topic of the Water, Sanitation, and Hygiene (WASH)² systems strengthening³ in three different learning arrangements. It explores performance in terms of user reactions and learning, and actual application (behaviour change) and follow-up actions that might contribute to the strengthening of water and sanitation systems. The three cases are:

• Case 1: MOOC as self-paced and self-guided,

• Case 2: MOOC with structured support (India), and

• Case 3: MOOC with blended learning (Ethiopia).

The MOOC used was ‘WASH systems strengthening: The basics’⁴, which forms the basis of all three cases. This MOOC has been accessible for free since September 2019 via the e-learning platform the WASH Systems Academy⁵, developed by IRC WASH (IRC 2021). The study was conducted between October 2020 and July 2021.

A modified Kirkpatrick model (Kirkpatrick 1994) was used to evaluate the performance of the MOOC and explore the application of its content at three levels, namely reaction, learning, and behaviour change. The research is based on quantitative and qualitative data analysis using data extracted from the e-learning platform, surveys among participants, individual interviews, and focus group discussions with practitioners in the field who have used the platform and to some extent applied its content. Additionally, an Outcome Harvesting methodology (Wilson-Grau 2018) was used to analyse the behaviour changes.

IRC designed the MOOC ‘WASH systems strengthening: The basics’ as the basis for its free e-learning platform the WASH Systems Academy. The Academy's ambition is to target a large audience by offering high-quality and strategic content for promoting sustainable, universal water and sanitation, and hygiene service delivery through continuous systems strengthening towards SDG 6. The Academy was launched in September 2019 with the course ‘WASH systems strengthening: The basics’. After 1 year, it reached 2,000+ users from more than 40 different countries with two additional MOOCs on offer. By July 2021, the Academy had 3,100+ users from 110 countries with seven MOOCs available.

The parameters of success of a MOOC to potentially contribute to systems strengthening are reflected in the ability of MOOCs to attract a large audience from diverse backgrounds who share a joint interest to gain knowledge and skills on water and sanitation service delivery. Other parameters of success are positive user evaluations and high course completion rates. Acknowledging that such parameters could be a first step in the long process related to recognising the complexity and intertwined nature of the different components (or building blocks) of the WASH systems.

The WASH Systems Academy's Theory of Change (ToC) 2019–2030 (Figure 1) describes the vision and actions for contributing to achieving SDG 6 and highlights how WASH education interacts with and potentially contributes to systems strengthening. The ToC maps out the intermediate outcomes considered most critical for the sector to deliver universal and sustainable WASH services and the WASH Systems Academy's proposed contribution to those outcomes.

This MOOC course covers the basic concepts of WASH systems strengthening, with an estimated average duration of 16 h. The MOOC aims to explain the systems strengthening approach, how and why it was developed, and how to start applying it. Its content is divided into nine sessions that need to be completed in consecutive order with three multiple choice tests:

• Session 1: Introducing the WASH Systems Academy

• Session 2: From broken pumps to lasting systems

• Session 3: What is WASH systems strengthening?

• Multiple choice test on sessions 1–3

• Session 4: WASH is a service

• Session 5: Sanitation systems strengthening

• Session 6: Hygiene promotion for WASH systems strengthening

• Multiple choice test on sessions 4–6

• Session 7: WASH systems strengthening away from home

• Session 8: Leaving no one behind

• Session 9: The roadmap to sustainable WASH services

• Multiple choice test on sessions 7–9

Each session contains a combination of activities including podcasts with WASH system experts, animations, forum discussions, quizzes, and exercises to create one's own materials to promote WASH systems strengthening.

The research compared three different learning arrangements within particular contexts and selected through an emergent sampling, particularly suitable when the research being conducted is exploratory in nature (Cohen & Crabtree 2006). The three cases with their key features are shown in Figure 2.

A modified Kirkpatrick model (Kirkpatrick 1994) was used to evaluate the performance of the MOOC and explore the application of its content (see Figure 3). The Kirkpatrick model consists of four levels: reaction, learning, behaviour change, and results (Kirkpatrick 1994). To categorise the findings, we have defined the performance as reaction and learning (levels 1 and 2) and the application as behaviour change (level 3). Part of the performance is also a comparison of costs per user for the three learning arrangements in terms of capital expenditure (CapEx) and Operational and minor maintenance expenditure (OpEx).

This study did not consider the fourth level results, as this requires organisational change (i.e., changes of more than one person), and users of Case 1 (i.e., the MOOC) participate as individuals. Hence, this level does not apply. Achieving results in level 4 of systems change is also a long-term process of multiple years and the scope of the study is short-term (October 2020 and July 2021). It is, therefore, too early to draw conclusions regarding changes at the results level.

Level 1 ‘Reaction’ was assessed in terms of completion rates and user experience, and level 2 ‘Learning’ was evaluated in terms of knowledge and skills. Level 1 ‘Reaction’, level 2 ‘Learning’, and partly level 3 ‘behaviour change’ were assessed with quantitative and qualitative data analysis on user engagement on the E-learning Platform and through anonymous user surveys. The user survey includes four open-ended questions and seven statements on content, navigation, and user experience that users rate with a Likert Scale of 1–5, with 1 being ‘strongly disagree’ and a score of 5 ‘strongly agree’. Level 3 ‘behaviour change’ on applying elements of WASH systems strengthening was explored through in-depth interviews with selected users, through open-ended questions in the user survey and open response sections of the MOOC, and in Cases 2 and 3 through group discussions. Reported changes were categorised using the Outcome Harvesting methodology (Wilson-Grau 2018) in terms of changes in relations, practices, actions, and activities. For an overview of data collection methods, see Table 1.

Outcome Harvesting is a method that enables monitoring and evaluation of specific initiatives. It has been widely used by networks, non-governmental organisations, and research centres among others (Wilson-Grau & Britt 2012). The method contributes to the collection of evidence on what has been achieved (the outcome) whether good or bad, planned, or unplanned, and works in a reverse direction to verify and explore how the initiative contributed to that specific change by using specific questions. Unlike common evaluation methods that measure progress against pre-set outcomes, this method highlights how a given initiative has contributed to outcomes, in the sense of behavioural changes in relationships, actions, activities, practices, or policies. That is why it was chosen as it can be used to collect evidence of achieved effects and systems change, (Wilson-Grau 2018; Bori & Huston 2020). The methodology facilitates rigorous investigation into whether and how the MOOC contributed to behavioural changes in relationships, actions, activities, or practices. No policy changes were expected to be found as they require organisational changes that go beyond an individual's sphere of influence.

Although the Kirkpatrick model has been criticised (Bates 2004), it has been used for decades and remains one of the most widely used tools for the evaluation of workplace training due to its systematic approach (Paula et al. 2016). The modified Kirkpatrick model helps to gain an insight from a user's perspective. It also gives ideas for improvements in user knowledge and skills after completing the course and on the user's application of knowledge which makes clear if the course was successful or not.

To examine the costs per user associated with the three cases, linked to Performance, the life-cycle cost approach (LCCA) was applied (see Table 2). An LCCA is a methodology to comprehensively identify and analyse the full costs of delivering WASH services, including infrastructure and both direct and indirect support. Of the six life-cycle cost categories, only CapEx and OpEx are applicable to the three cases. The first Capital maintenance expenditure (CapManEx) for the studied MOOC is planned for 2022 as it is expected that every MOOC needs a major overhaul or a replacement every 3–4 years.

The study was conducted between October 2020 and July 2021 with:

• Case 1: MOOC was open to all free of charge between September 2019 and October 2020.

• Case 2: MOOC with structured support to selected users from India in February 2021.

• Case 3: MOOC with blended learning for selected users from Ethiopia in three pilots between April and June 2021.

For Level 1, Reactions, two user surveys were sent out to participants (July 2020, n = 201, and July 2021 n = 423) to solicit feedback on the overall performance and user experience. For Level 2, Learning, each user answered a feedback survey after completing the course. For Level 3, Behaviour Change, exploratory interviews with users were held. The interviewees were randomly selected from a user survey sent out to users who had taken or completed the MOOC. Individuals who mentioned they had or were planning to apply the knowledge in their work were identified and those who gave their e-mail addresses were contacted.⁶ With those who responded, interviews were conducted to find out more about how the Academy has helped them strengthen WASH systems. Additionally, some users who could not be part of the interviews self-reported, via e-mail, on the behavioural changes that occurred after taking the course. This was incorporated into the data analysis by looking at the topic and actions of the behaviour changes.

For Levels 1 and 2 ‘Reactions and Learning’, participants took part in a survey after finishing the training. Furthermore, facilitator feedback was collected through interviews, and participants' grades from the online assessments (three tests) taken during the course.

For Levels 1 and 2 ‘Reactions and Learning’, participants took part in a survey after finishing the training. Furthermore, informal interviews with the governmental agencies that participated were held. For Level 3, Behaviour Change, focus group discussions with participants and key governmental staff were held.

With respect to the limitations of this study, we acknowledge a potential bias in our user surveys since the options offered were positively oriented. The responses were simplified to encourage participants to answer them. However, a Likert scale was used, so that participants could express the extent to which they agree or disagree with the statements while also giving space for more feedback.

Moreover, a self-completion survey itself can be considered as biased as users were willing to take time out to report back, and you may assume that they only do that if they enjoyed the course. Participants with opposing views are not likely to take part, so no negative feedback could be gathered this way.

Another limitation is that the user surveys were answered by a very specific audience, 65% of the users who had completed the survey in June 2020 were from NGOs. This type of audience may have already been familiar with the content. To create long-lasting systems change, stakeholders with different backgrounds need to take the courses and not just NGOs. Actors such as governments (national and subnational), private sector, donors, civil society organisations (CSOs), and community-based organisations (CBOs) are all key stakeholders in systems strengthening.

This section presents the findings on the performance and application for all three cases – Case 1: MOOC, Case 2: MOOC with structured support, and Case 3: MOOC with blended learning. Together the three cases reached a global audience of 1,594 users from 47 countries as shown in Figure 4. The largest representation of users was in Africa followed by Asia, Europe, and North America.

As Table 3 shows, the audience reached by Case 1: MOOC is different from Cases 2 and 3. Users of the stand-alone MOOC have relatively fewer years of work experience related to WASH and are predominantly working for an NGO. In Cases 2 and 3, users are predominantly male and have more than 5 years of work experience related to WASH. For Case 1, data on gender was only systematically collected from June 2021 and not for the period of this study (i.e., September 2019 to October 2020). However, the user data from June 2021 to September 2021 for Case 1: MOOC also shows that 94% of the users are male (N = 308).

Table 4 shows that the performance of all three cases in terms of user learning and reaction is comparably positive. In all three cases, users strongly recommend the course, scoring the statement with weighted average scores of 4.6, 4.7, and 4.6 (for MOOC, MOOC with structured support, and MOOC with blended learning, respectively) on a scale of 1–5 with 1 being ‘strongly disagree’ and a score of 5 ‘strongly agree’.

In total 1,512 users registered for ‘WASH systems strengthening: The basics’ between September 2019 and October 2020. Figure 5 indicates the 47 countries where users were located who complete the user survey sent out in June 2020 (N = 202). The two regions with the highest representation were Africa (48%) and Asia (22%), followed by Europe (16%) and North America (12%).

The users of the MOOC who had completed the survey in June 2020 (N = 203) were predominantly from NGOs (i.e., 65%) and government (national) at just over 7%, closely followed by research and education (6%), consultants (6%), private sector (5%), and government (subnational) at only 3%. At the bottom are donors (2%) and CBOs or CSOs (1%).

The costs associated with Case 1: Self-paced MOOC range between 1,500 and 3,000 Euro per month (Table 5). In Case 1, no CapEx is incurred as the same MOOC is continuously made available for free between September 2019 and October 2020. The operational expenditure (OpEx), given per month, mainly relates to time spent by IRC to answer questions from users via e-mail on the content of the MOOC or technical questions on the platform and costs to host the platform (and all MOOCs) on the IRC server. When new materials such as guidelines, tools, etc. became available, this was added to the MOOC, and time spent by IRC to update the MOOC is included in the OpEx.

The cost range is given per month as the MOOC has continuously run since September 2019.

In the user survey sent out in June 2020, 59% indicated (N = 195) that they found opportunities to apply content from the MOOC in their work on elements of WASH systems of learning and adaptation, planning and budgeting, monitoring, and water resource management. In the open-ended section of the survey, interviewees described ways in which the topics of the course had been applied in their work, below are two examples.

Interviewees mentioned examples (Box 1) of how they have applied knowledge and in particular how they have changed the way they work, also with others. As well as their priorities in different elements of their work, for example, the idea of working and implementing change from the local level up to the national level. Most changes occurred in actions and activities (see Figure 6).

Performance of the MOOC is measured by the ‘reaction’ of users in terms of completion rates and user experience. The completion rate of the MOOC between September 2019 and October 2020 was 21% (N = 341 users).

In terms of user experience, the MOOC scored an average of 4.5 (July 2021, n = 423) on a scale of 1–5 with 1 being ‘strongly disagree’ and 5 ‘strongly agree’. Users learned something new (Box 2) and would recommend the course, and the average scores were 4.6 and 4.7 (see Table 4).

In February 2021, the IRC India programme was approached by Tata Trusts to develop a structured and supported 1-week online learning programme for 17 staff members of Tata Trusts' implementing partner organisations (17–24 February). Due to the COVID-19 pandemic, using traditional training methods (classroom) was not possible in India. Therefore, a structured and supported online learning programme was chosen. The approach included:

• A self-paced online MOOC (i.e., ‘WASH systems strengthening: The basics’)

• Three live webinars of 2 h each

• Group work in each webinar

This was complemented by a WhatsApp group for participants and course facilitators, where participants could ask questions and connect outside of the live webinars.

In terms of the user profile, most participants were male implementing partners of Tata Trusts with more than 5 years of work experience related to WASH. One participant worked for subnational government and one for a consultancy company.

The costs associated with Case 2: MOOC with structured support range between 3,000 and 8,000 Euros (see Table 6). For Case 2, CapEx to customise the MOOC, the webinars, and group work for the target audience is minimal. CapEx includes mainly providing the MOOC that is branded with the logo of the client to a closed group of selected users and adding examples related to the target audience to the webinars. The OpEx relates to running the course, facilitating the webinars, and supporting users via e-mail and WhatsApp. Monitoring the behaviour of users on the MOOC and reporting to the client are also included in OpEx.

Performance in terms of completion rates was 65% (11 out of 17 users). Performance in terms of reaction and learning scored well with users rating the course with an average score of 4.5 (n = 10) on a scale of 1–5 (see Table 4).

In open-ended response sections of the user survey and in group discussions of the webinars, users described the course as ‘interesting’, ‘relevant for their context’, and ‘very informative’. They thought that the webinars, group work, and WhatsApp group were useful additions to the course. Also, participants mentioned that without the facilitated face-to-face group sessions, they would never have started an online course or completed one.

Some participants offered suggestions for improving the format of the MOOC, webinars, and coursework. These suggestions included: conducting the training in local languages and translating the MOOC into local languages; increasing one-to-one interaction between participants and facilitators; providing more books, articles, and other supplementary reference materials; and accommodating more diverse settings and contexts of participants.

The participants' learning was evaluated by assessing the three multiple choice tests in the MOOC. The course grades showed promising results. The average course grade for participants who completed all five of the assessments was 94%, indicating a good understanding of the course content. The group work and discussions that were part of the live webinars also reflected this.

Although the performance is looking promising, it is still too early to see behaviour changes. The application of new knowledge and skills by participants will be further investigated in a new study from 2022.

Between April and June 2021, the MOOC ‘WASH systems strengthening: The basics’ was used for a blended learning approach in three pilots in Ethiopia on request of the Ethiopian Water Technology Institute (EWTI).⁷ The blended learning pilots combined the MOOC with group work and discussions held in an in-person training setting. The pilots took place in Bishoftu town and Bahir Dar, Ethiopia:

• Pilot 1 (26 April to 8 May): Training of Trainers (ToT) of 26 EWTI trainers.

• Pilot 2 (7–9 June): Regional training in the Amhara region of 18 participants.

• Pilot 3 (17–19 June): Regional training in the Oromia region of 19 participants.

In terms of user profile, Case 3 was aimed at targeting government staff (see Table 7) as governments are ultimately responsible for service provision in their country. Pilot 1 reached government staff at the national level, while pilots 2 and 3 predominantly reached subnational governments at regional and district levels and utilities.

The costs associated with Case 3: MOOC with blended learning range between 5,500 and 12,000 Euros (see Table 8). In Case 3, CapEx consists of time to customise the MOOC and the 3-day training workshop for the target audience. The CapEx includes providing the MOOC that is branded with the logo of the client to a closed group of selected users. The OpEx mainly relates to running the workshop. Monitoring the behaviour of users on the MOOC and reporting to the client are also included in the OpEx.

The performance of the MOOC with a blended learning approach proved successful in terms of completion rates. All 65 people, 28 at the national and 37 at the regional level (i.e., Amhara and Oromia regions), completed the course. Performance in terms of user satisfaction and learning indicated that users were positive about the approach, rating it with an average score of 4.4 (see Table 4).

In the group discussions part of the course, users mentioned that the concept of WASH systems strengthening and partaking in a MOOC was completely new to them. Before the COVID pandemic, EWTI provided only face-to-face courses and seminars. Users indicated in the discussions not to have had much interest in MOOCs because the options offered by EWTI were more appealing. The issue of the English language was another reason why some users did not use MOOCs. In the evaluation group discussion and the user survey, they indicated that they enjoyed the online e-learning platform and quickly felt comfortable using it (see Table 2). Participants wrote in the open-ended questions in the survey that the course made them feel ‘happy and confident’.

In terms of learning, through the group discussions and reflections, participants showed a good understanding of the key concepts underlying WASH systems thinking, i.e., how all actors and factors contribute to systems strengthening. This was reflected in the three multiple choice tests in the MOOC which had a total average score of 96% correct (n = 40).

It is too early to assess signs of behaviour change on WASH systems strengthening in Case 3. However, in the preliminary results, it is possible to identify some changes in terms of actions and activities. For instance, based on the initial pilot in April, EWTI decided to (1) include ‘WASH systems strengthening: The basics’ in its training package as a mandatory course for all its trainees and (2) cascade and pilot the course in the Amhara and Oromia regions in June. Based on feedback received in a group discussion with participants and key EWTI staff after the initial pilot, IRC and EWTI identified necessary adaptations for the course to be cascaded to the regional level.

The initial pilot ToT resulted in a new course titled WASH Systems Strengthening for Ethiopia. The adaptation focused on relating all the key concepts, such as the SDGs, service delivery models, WASH service levels, and the right to water and sanitation, to the context of Ethiopia. For example, after discussing the right to water and sanitation, information was added on how these rights are embedded in the Ethiopian constitution with a link to the constitution as part of the course resources. Key sector documents were also added to the course resources such as Ethiopia's 2017 Voluntary National Review of the SDGs and Ethiopia's Growth and Transformation Plan II (GTP II). The course resources were adapted to the local context, including images, to improve further uptake. Terms and definitions for Ethiopia were added to the course glossary on service authorities (i.e., woreda water office) and service providers (i.e., WASH Committees or WASHCOs, rural water board, rural utility, and urban utility).

The new course ‘WASH Systems Strengthening for Ethiopia’ was piloted in two regions (i.e., Amhara and Oromia regions) in June. In the group discussion part of the course, participants indicated that they wanted to provide the training to their district stakeholders. They felt confident that they could take this on themselves. However, for this to be successful, the MOOC would first need to be translated into the local language(s). The face-to-face discussions in the pilots were conducted in three local languages and the MOOC in English.

Other first indications of application were found in the open-ended part of the user survey (June 2021, n = 51) where users indicated to be ‘confident and striving to teach or give trainings about WASH systems to appropriate audiences throughout the country’.

MOOCs are often criticised for their high incompletion rates.⁸ Large MOOC providers generally have completion rates of less than 10% (Diver & Martinez 2015). Davis et al. (2017) also report low course completion rates ranging between 5 and 10%, while Jordan (2015) informs a median value of 12.6%. In all our three cases, completion rates were above average ranging from 21% in Case 1 MOOC to 65% in Case 2 MOOC with structured support and 100% in Case 3 MOOC with blended learning.

However, Case 2, MOOC with structured support, also showed the difficulty of sustaining participation among working professionals who have limited time for additional activities outside of their work. This is compared to Case 1 where working professionals have the flexibility of 1-year free access to the WASH Systems Academy to complete the MOOC. It took users of the stand-alone MOOC (i.e., Case 1) on average 40 days to complete the MOOC spread over 1–8 weeks. This falls in the range provided in industry reports that find that MOOCs range in length from 1 to 16 weeks, although there may be significant variations (Bowden 2020).

In general, MOOCs tend to dive into topics that only specialists in the field are familiar with and are not effective in promoting hands-on skills (Hall & Villareal 2015; Cundell & Sheepy 2018). To overcome these common obstacles faced by MOOCs, next to high incompletion rates, blended instruction is reported to be more effective than purely online learning or purely face-to-face (Bornstein 2012). Blended learning can result in high levels of student achievement and can be more effective in creating sustainable behaviour changes than just face-to-face learning (Saritepeci et al. 2015).

In the case of blended learning in all three pilots in Ethiopia, users felt that this approach created more active user engagement than a traditional face-to-face training. They found the self-study through the online course to be motivating because it visualised their progress. The self-study and activities offered on the MOOC – such as exercises, quizzes, and forum discussions – enabled participants to self-assess their understanding of the content. The contextualisation of the MOOC through group discussions and face-to-face interaction with an instructor, which was part of Case 3, created a learning environment where users could better relate to and understand the content. However, to further scale up the MOOC with blended learning at the subnational level, the MOOC needs to be translated into local languages to utilise the full potential of this approach.

The case of the MOOC with blended learning in Ethiopia was also successful at reaching government actors at national and subnational levels (i.e., regional and district) and to some extent service providers (i.e., utility). Users of the MOOC and the MOOC with structured support (i.e., Cases 1 and 2) were predominantly working at NGOs. The stand-alone MOOC (i.e., Case 1) attracted an audience with relatively few years of work experience related to WASH than the structured support and blended learning (i.e., Cases 1 and 2). Together the three approaches were able to reach a varied audience in terms of work experience related to WASH and in terms of stakeholders. However, the audience in all cases was predominantly male. More investigation is planned to identify ways to attract more female trainees.

The parameters of success that potentially contribute to systems strengthening are not just shown in the large audience that was attracted, the positive user evaluations and high completion rates (i.e., performance of the MOOC in terms of user reaction and learning). First indications of intermediate outcomes in application regarding modifications in observable events by users in changes in actions, activities, relations, and practices have also been found (i.e., behaviour change). Users reported opportunities to apply content from the MOOC in their work on elements of WASH systems of learning and adaptation, planning and budgeting, monitoring, and water resource management.

In this regard, the MOOC showcases to some extent that the expected intermediate outcomes from the ToC (Figure 2) are feasible, hence, moving a step closer to reaching impact. At the same time, the MOOC helps as a recognition that capacity development could be a first step to realising behavioural changes and results that feed into strengthening the WASH systems.

The experience of the WASH Systems Academy shows that MOOCs can be a versatile and effective capacity development tool. The performance of all three cases was successful in terms of completion rates and user satisfaction. Users in all three cases also strongly recommend the course and have found opportunities to apply content from the MOOC in their work.

From this short assessment, we can conclude that the first arrangement (Case 1: MOOC) is best suited for reaching a large number of people via an e-learning platform and has the lowest cost of all three cases. The second set-up (Case 2: MOOC with structured support) is good for encouraging the completion rate by providing live support and contextualisation of the content in webinars and via WhatsApp. However, this was at a much higher cost than Case 1. The third arrangement (Case 3: MOOC with blended learning) is successful regarding completion rates, attracting a different audience of national and subnational government actors and service providers, and creates more active user engagement but with significant higher costs than Cases 1 and 2.

The authors would like to acknowledge the following people for their contributions to this paper; Dechan Dalrymple for designing all images, Gezahegn Lemecha Boru and Zewdu Assefa for the case of Ethiopia, Ruchika Shiva and Shiny Saha for the case of India, Samantha Wind for the section on methods and the case of India, Angela Huston, John Butterworth, and Camilo Benitez for their support in reviewing this paper, and Tettje van Daalen for editing this paper. The WASH Systems Academy and the MOOC ‘WASH Systems Strengthening: The basics’ discussed in this paper were developed with funding from the Conard N. Hilton Foundation and the Ministry of Foreign Affairs of the Netherlands (DGIS).

All relevant data are included in the paper or its Supplementary Information.