To extend water, sanitation and hygiene services to all, technological innovations are required which take into account a diverse range of stakeholder perspectives. We report the experiences of an intervention which sought to build capacity in the assessment and introduction of technologies in Uganda, Ghana and Burkina Faso by developing the Technology Applicability Framework (TAF), a tool which culminates in a multi-stakeholder scoring workshop. The project also used Learning Alliances to build capacity around technology introduction. This paper explores how stakeholder attitudes changed through the project and evaluates the Learning Alliance approach. It finds that whilst the intervention did manage to connect stakeholders in a novel way, uptake of the TAF may be hampered by a lack of government involvement in the earliest stages of the project.

Introduction

The role of technology in providing water, sanitation and hygiene services

Progress has been made towards improving access to water, sanitation and hygiene promotion (WASH) services over the last two and a half decades. However, huge challenges lie ahead. Figures from 2015 estimated that 2.4 billion (109) people lacked access to adequate sanitation and 663 million people were still without access to an improved water source. There are also large disparities in access to WASH services across the globe. Most notably, sub-Saharan Africa has the lowest levels of coverage for water and sanitation in the world and many countries within the region did not meet the Millennium Development Goal (MDG) targets (World Health Organization/United Nations Children's Fund (WHO/UNICEF), 2015). Sub-Saharan Africa's increasingly expanding population, which is predicted to grow faster than anywhere else in the world over the next 50 years (United Nations, 2004), together with the impacts of climate change and declining freshwater availability (Röling, 2002), are likely to compound the situation further.

WASH technology in the sub-Saharan African region has a history of poor performance. To provide a clear example, breakdown figures show that 30–40% of decentralised water systems are non-functional at any one time, a statistic which has hardly changed over the last two decades (Evans, 1992; Rural Water Supply Network (RWSN), 2009; Lockwood & Smits, 2011). This phenomenon could be attributed to the extreme poverty in the region. In 2010, 48% of the population of sub-Saharan Africa lived on less than $1.25 per day, nearly twice that of any other region (Olinto & Uematsu, 2013). This means there are few resources available to provide and maintain water supplies. The breakdown figures represent a huge waste of resources and, most importantly, an interruption in services for communities, with obvious implications for the health and social well-being of those concerned. Participatory management, involving all the stakeholders concerned with a problem, is becoming increasingly accepted as the desirable means of addressing service delivery issues. Consequently, development organisations are shaping their interventions and looking to bring innovative practice in service delivery through innovative participatory approaches.

A key factor in addressing challenges in service delivery across sub-Saharan Africa is the sector's ability to move new technologies from the pilot phase to more widespread usage (scale-up) to ensure the sector as a whole can benefit. Technologies are not ubiquitously applicable and what may work well in one location may not work well in another location, potentially not even in a neighbouring village. Historically, such participatory interventions (particularly in the rural WASH sector), have worked at the community level to identify problems and develop locally appropriate innovative solutions. However, Moriarty et al. (2005) suggest such community level interventions typically focus on the particular project in hand rather than considering whether the innovation could be scaled up and contribute to the wider development of the sector. Such interventions rarely acknowledge the role of ‘intermediate organisations’, like local government, non-governmental organisations (NGOs) and the private sector, in supporting the uptake and implementation of new approaches (Moriarty et al., 2005; Smits et al., 2007). This is in spite of the fact that these organisations are thought to be crucial in the scaling process, as they provide critical technical assistance, access to information and secure supply chains for spare parts (Schouten & Moriarty, 2002). The role of national level stakeholders is also ignored by many participatory research projects.

This wide spectrum of organisational actors is vital in policy formation for sector-wide approaches, disseminating new practice and financing new initiatives. Bypassing these organisations often results in the implementation of innovations which are inappropriate or impractical at a local level because they fail to take into account institutional capacity or legislation (Moriarty et al., 2005, Smits et al., 2007). There is also no creation of ownership for the intervention at the national or intermediate levels or capacity to support its implementation. Further, a project-based approach to implementation has often meant an over-emphasis on developing innovative solutions to the detriment of investment in raising awareness of the innovation within the sector (Moriarty et al., 2005). All of these factors mean that innovation is unlikely to become more widespread and reach scale, the necessary changes in sector policy and practice are not achieved and the sector-wide adoption, crucial for cohesive service delivery, fails to materialise (Smits et al., 2007; Lockwood & Smits, 2011).

The Learning Alliance approach

The frustrations surrounding an apparent inability to scale up innovation in WASH service delivery had spurred the development organisations in the sector to pursue a range of progressive scaling-up methods. One of the most promising approaches to emerge in recent years is that of Learning Alliances. Learning Alliances aim to address some of the aforementioned barriers to scale-up, by working within the sector's institutional framework with an emphasis on multidisciplinary stakeholder involvement. Learning Alliances can be thought of as providing a facilitated interconnected platform for engaging different stakeholders within a sector at different institutional levels and from different disciplines (Röling, 2002). Together they strive to optimise the prospects of ‘scaling up’ appropriate innovations and embed them within sector practices and procedures and develop capacity in the use of an innovation along with the ability to further adapt and reproduce it in different contexts (Smits et al., 2007). However, Sijbesma et al. (2007) identify that this is likely to be somewhat dependent on the ability of individuals to institutionalise new innovative practices developed in Learning Alliance activities within their respective organisations. Capacity is also expected to be built at an institutional level and in many circumstances this may be the most important type of capacity, particularly where the area of service delivery being addressed requires a sector wide adoption of an innovation. Through the creation of platforms for different organisations to come together and exchange ideas, it is suggested that relations will be built or strengthened between these organisations. Sijbesma et al. (2007) argue that improved organisational relations mainly evolve through the mutual benefits realised through improved sharing of information and the negotiation of joint activities. Smits et al. (2007) suggest additional benefits of accountability, coordination and cooperation may also be observed.

The Learning Alliance approach has its roots in the business world where the term was used to describe inter-organisational knowledge sharing aimed at the mutual enhancement of performance of those businesses involved (Inkpen, 1996; Khanna et al., 1998). In the development world, Learning Alliances have been taken up as a means of devising and subsequently institutionalising innovations in technology, praxis and governance, most notably within the agriculture and water and sanitation sectors. One of the first examples of the application of Learning Alliances within the development sector was the TRANSCOL project taking place in Columbia between 1989 and 1996, although it lacked the national level platforms characteristic of current Learning Alliance initiatives (Smits et al. 2007; Visscher et al., 2007). This project sought to scale up the use of multi-stage filtration water purification technology for use in different regions in Columbia. However, the first example of a development project explicitly adopting and referring to a Learning Alliance approach to programme management came in the form of an agro-enterprise implemented by a Columbia-based research organisation called Centro International de Agricultura Tropical (Lundy & Gottret, 2007). Since these early examples, the Learning Alliance approach has been gaining popularity, most notably in WASH projects seeking to institutionalise innovation in governance.

The Learning Alliance approach has derived its theoretical underpinnings from the two closely related concepts of social learning and innovation systems (Lundy & Gottret, 2007; Proost & Lewis, 2007). However, these are still relatively underdeveloped and it is not yet clear how many of the proposed benefits are realised in practice (Sijbesma et al., 2007; Muro & Jeffrey, 2008; Reed et al., 2010). Thus far, the Learning Alliance literature generally takes the form of commentaries provided by facilitating or management organisations involved in Learning Alliance projects. In these examples, the perspectives of different stakeholders taking part in Learning Alliances are rarely portrayed. For example, the role of national level stakeholders and how they support, in particular, the institutionalisation process, but also the diffusion of innovation within the sector is largely overlooked. Lockwood & Smits (2011) explain how in many developing countries the process of decentralisation is incomplete. As such, devolution of funding and service authority has not actually materialised in practice and these intermediate organisations are lacking the capacity to implement changes. Moreover, in many developing countries there is a shift towards using sector-wide regulatory frameworks and basket funding mechanisms in a bid to increase coordination of service delivery (Visscher et al., 2007; Lockwood & Smits, 2011, da Silva Wells et al., 2013). Learning Alliances are now also being applied to solve water management challenges in Europe. The Dutch Water Sector Intelligence platform, launched in 2008, is a horizon-scanning platform addressing the common future challenges of the Dutch water sector. It examines and reports on social, economic, political, technological, ecological and demographic trends, and translates national reports to the sectoral level (Frjins et al., 2013). Learning Alliances have also been implemented in a number of cities (Accra, Alexandria, Beijing, Belo Horizonte, Birmingham, Cali, Hanburg, Lima, Lodz, Tel Aviv and Zaragoza) through a project funded by the European Commission with the intent of speeding up the identification, adaptation and uptake of new innovations (van der Steen & Howe, 2009).

Study context

An intervention was designed to overcome the various impediments to scale-up noted in the foregoing sections. This was achieved through the development of tools that can be used to assess technologies and aid the introduction of appropriate WASH technologies. Principal amongst these tools was a Technology Applicability Framework (TAF) tool. This was a framework used to assess the applicability of a technology to the local context (for a detailed overview, see Kimera et al. (2013)). The TAF tool requires the collection of field data around the six different sustainability dimensions: economic; technological; social; skills and know-how; environmental; and institutional and legal.

In employing the TAF tool a field team of researchers and facilitators conducted surveys with questions relating to the sustainability dimensions identified above. Data collection explicitly sought the perspectives of three main groups of actors: the users of the technology, the producers or providers of the technology and the regulators of the sector (Kimera et al., 2013). Once the field team had collated the data, a meeting was called comprising stakeholders from different disciplines usually operating at the intermediate and community levels. These stakeholders were encouraged to discuss the performance of the technology in the six sustainability dimensions. With the help of a facilitator, the group were urged to reach consensus on a score (positive, negative, neutral or unknown) to attribute to the technology within each dimension as illustrated in Figure 1. During the project, there were three rounds of testing of the TAF in each of three countries – Ghana, Uganda and Burkina Faso. After each round there was a review which was used to inform changes to the TAF for the next round. The findings from the scoring workshops were presented as technology recommendations for the sector, presenting each technology as an individual case study. The technologies assessed during the TAF testing are shown in Table 1. They were selected by staff based in each country to provide a range of technologies that they deemed to be successful, promising, new or failed and also to provide a balance between water and sanitation technologies. The final report from each test, including the background on each study site is available at WASHTech (2016).
Table 1.

Technologies assessed during tool development by country.

Country Technology assessed 
Burkina Faso Rope pump 
Ventilated Improved Pit (VIP) latrine 
India Mark II 
UDDT 
Sand Dam 
Water Harvesting Tank 
Ghana Rope pump 
Poor flush 
Enviroloo 
Ghana modified India Mark II 
Biofil Toilet 
Slow Sand Filter 
Uganda Rope pump 
UDDT 
Solar Water Pump 
U2 Pump 
Ferro Cement Tank 
Tippy Tap 
Country Technology assessed 
Burkina Faso Rope pump 
Ventilated Improved Pit (VIP) latrine 
India Mark II 
UDDT 
Sand Dam 
Water Harvesting Tank 
Ghana Rope pump 
Poor flush 
Enviroloo 
Ghana modified India Mark II 
Biofil Toilet 
Slow Sand Filter 
Uganda Rope pump 
UDDT 
Solar Water Pump 
U2 Pump 
Ferro Cement Tank 
Tippy Tap 
Fig. 1.

Example of scoring produced from the TAF.

Fig. 1.

Example of scoring produced from the TAF.

Project members from African organisations in charge of programme facilitation were responsible for the formation of national Learning Alliances. These organisations formed core working groups of key stakeholders operating at the national level. Core working groups generally consisted of representatives from government ministries, national utility companies, NGOs, research institutes and private sector organisations. Facilitating organisations made use of existing sector platforms in order to recruit stakeholders for involvement in core working groups. Information about technology performance and the performance of the TAF tool was fed back to core working group members after the scoring workshops took place. The Learning Alliance members could then share the new knowledge and practice throughout their organisation.

Research objectives

This paper presents the experiences of stakeholders involved in the project described above. A qualitative study explores how stakeholder attitudes changed through the project. The paper also evaluates the Learning Alliance approach and whether it was able to help change stakeholder attitudes.

Methodology

Study design and concept

In executing a qualitative study, we adopt a realist evaluation approach to the investigation of the Learning Alliances (Pawson & Tiley, 1997; Pawson, 2006). Within the context of programme evaluation, realist evaluations are increasingly seen as an adjunct to experimental designs, particularly in the evaluation of complex interventions addressing issues such as governance and capacity building (Holma & Kontinen, 2011). In such interventions, it is often difficult to define clearly measurable indicators for the realisation of project outcomes. Moreover, it is argued that the emphasis on producing indicators that can be easily measured is at odds with a learning agenda which is obviously important in the improvement of development interventions (Holma & Kontinen, 2011).

A realist evaluation begins with theory building, i.e. a process of constructing understanding around the interventions being implemented and how they are proposed to work. For this study, a comprehensive review of project documentation, including sources such as funding proposals, project monitoring documents and the project's theory of change, informed the initial identification of project interventions. A subsequent review of the literature surrounding the various interventions including that of: Learning Alliances, social learning, innovation systems, diffusion of innovations and participatory research was necessary to develop the theory behind the project interventions. Context, mechanism, outcome, and configurations were proposed as key factors which describe and shape the potential efficacy of interventions (Pawson & Tilley, 1997). This process informed the development of the interview questions used for the study. Questions were developed and underwent two rounds of drafting before finalisation. Interviews elicited respondent perceptions of five thematic issues: (i) changes in policy, practice and procedure relating to technology assessment and introduction, (ii) changes in knowledge and attitudes towards a technology and technology assessment procedures, (iii) levels of control and coordination within the sector, (iv) perception of project innovations, and (v) level of institutionalisation realised by project innovations and suggested actions for future institutionalisation and scale-up.

Sampling and data collection

A purposive sampling strategy was adopted with participants being selected as potential interviewees if they had previous contact with the intervention team. The sample in each included: (i) Learning Alliance members involved in core working groups operating at the national level, (ii) those involved in the field testing of technologies assessed by the project, (iii) technology developers and manufacturers that worked with the technologies being assessed, (iv) participants of scoring workshops established to evaluate the performance of the technologies tested during the project, and (v) representatives from the organisations facilitating the project.

Forty semi-structured key informant interviews (average duration 38 minutes) were conducted over an 8-week period in June and July of 2013; 11 in Ghana, 12 in Uganda and 17 in Burkina Faso. Additionally, a focus group discussion was conducted with 15 employees that worked for the general directorate of water resources of the Ministry of Agriculture, Water and Fisheries in Burkina Faso. With respect to the sample's organisational profile, 10 (25%) interviews were conducted with representatives from national government, two (5%) with national utility companies, eight (20%) with local government employees, two (5%) with staff from the regional municipal offices of utility companies, 12 (30%) with NGO employees, six (15%) with staff from private sector organisations, and one (2.5%) with a representative of a research organisation. All had been part of a Learning Alliance except the representatives of technology companies and small-scale manufacturers. Prior to the start of the interview respondents were briefed about the purpose of the study and signed consent was obtained from all participants. Ethics approval for the fieldwork was secured from Cranfield University's Science and Engineering Research Ethics Committee prior to initiation of data collection. Interviews taking place in Burkina Faso were conducted in French through an interpreter with the exception of three interviews, which were conducted in Mòoré and translated into English.

Supplementary data

The study also made use of data generated from interviews conducted as part of the impact monitoring component of the project, which used the Most Significant Change (MSC) methodology (Davies & Dart, 2005). This entailed encouraging WASH stakeholders to provide a short story about a change they had observed as a result of the project. Facilitating organisations in each country gathered stories from stakeholders who either had direct involvement in the Learning Alliances or who had encountered the project through the project's dissemination activities. Thirty MSC stories were used from the participant countries. However, these were not subjected to the same rigorous coding and analysis procedures (described below) as the key informant interviews and focus group discussions. Instead, the MSC stories were used to verify and corroborate the interview findings as it became apparent the analysis had reached a point of saturation with regard to the findings.

Analysis

All interviews were recorded and transcribed verbatim. NVivo 10 (NVivo 2012) was used for the purposes of data management and analysis. An initial list of codes corresponding to the context-mechanism-outcome configurations model, developed during the theory building phase of the study, was generated. This list of codes was not exhaustive and was open to adaptation throughout the coding process. The lists of codes were entered into NVivo 10 as nodes and grouped into five themes related to the interview question subject areas. A nodal hierarchy was formed so that reported outcomes formed the main nodes, with mechanisms behind outcomes and important contextual information forming sub-nodes. Interview transcripts were subjected to several rounds of coding. In order to limit the bias introduced during coding, both affirmation and refutation of proposed outcomes and mechanisms were coded.

Response validation was carried out by emailing participants and asking for clarification on any answers which were not clear. A process of triangulation took place whereby reported outcomes, mechanisms and contextual factors were verified by other sources such as project documentation, project baseline reports, MSC stories and government and development partner reports. In the following sections, the notation used to identify the source of direct quotes has the following coding: U – Uganda, BF – Burkina Faso, G – Ghana, UG – Uganda; NG – national level government or utility, LG – local level government or utility, NGO – NGO, PS – private sector. Each respondent in each coding category has a unique number.

Results

Knowledge about technology performance

Twenty-six respondents explained how the knowledge they had acquired about technology performance had led to changes in their attitude towards certain technologies. All those reporting these outcomes had either attended the scoring workshops or had been involved in the field testing. The recognition of the technology user's perspective was frequently cited as an important contributor to this understanding of the ‘field reality’. ‘So the WASHTech project brought that good change of assessing technologies involving the grassroots population. So himself, he got that new knowledge of assessing technologies before introduction’ (BF/LG_2). Technologies for which respondents' attitudes had changed for the better included the rope pump in Burkina Faso, the Biofil toilet in Ghana and the urine diverting dry toilet (UDDT) in Uganda.

Coordination amongst stakeholders in the sector

Coordination between different stakeholders involved in the Learning Alliances was another important theme emerging from the data. Private sector organisations in each country provided examples of gaining awareness of the official bodies involved in technology assessment and introduction. Stakeholders from the private sector were put in touch with governmental bodies during the scoring workshops. Respondents from the private sector, as well as an NGO in Ghana, commented that interaction with government authorities had reinforced the need to engage with sector regulators when they are introducing a technology. ‘So for us, having been beneficiaries of the project, we will put in place measures to make sure that no such technology is introduced without passing through the formal assessment’ (G/NGO_2).

In Ghana, the involvement of a private technology developer in the multidisciplinary scoring workshops had allowed them to identify the relevant authorities involved in technology assessment and introduction. This company's sanitation technology had been identified as being a promising prospect for scale-up within the sector. The company has now been engaging with the Environmental Health and Sanitation Directorate of the Ministry of Local Government and Rural Development in order to have the technology officially assessed with a view to scaling up the technology for wider use within the sector, as a representative from the company explains:

Because without the TAF you don't even know the framework for introducing your technology and so you have your idea but you don't even know where to start or who to go to for help and with the TAF, it brings all the stakeholders together, so you are able to know who to go to and for what. Especially for the ministry of local government, now we are in some form of negotiation with them to ensure that district assemblies are put on board with regard to decentralising our technology to homes and so in a way it has been helpful to us’ (G/PS_1).

The involvement of government authorities in scoring workshops had also allowed them to take account of the frustrations of non-governmental actors around the lack of clear processes for technology assessment and introduction. As one respondent noted, ‘we may say that we have got a positive impact – during the evaluation of the products we realised that it is not always obvious, actors are not known, sometimes there is misinformation and sometimes the sender of the product does not know all the processes to approve the product’ (BF/LG_2).

In Burkina Faso and Uganda, the involvement of rope pump manufacturers in field testing activities provided them with the opportunity to engage with the users of their technology which is usually installed by NGOs. The rope pump manufacturer explained how they were now making efforts to ensure communities were able to contact them if they had operation and maintenance issues. The respondent representing the technology company in Uganda also mentioned that they had even trained some community members in operation and maintenance of a particular spare part. In Burkina Faso, one rope pump manufacturer suggested his attendance at multidisciplinary scoring workshops had been the motivating factor in improving community engagement. ‘So he said he needs to talk more to people to visit them more, to better understand if there are any difficulties at the community level. So like back and forth, he hears things during workshops and that sort of encourages him to go back and talk to the communities’ (BF/PS_1).

The interviews in Burkina Faso also revealed that the testing and scoring workshops had brought rope pump manufacturers working in different parts of the country together, and they realised how they need to coordinate their work. For example, the scoring workshops highlighted that it can take some time after pumping starts for the water to emerge. The stakeholders have now worked together to improve the mechanism. ‘They were intending to really improve the design of the rope pump and he even requested WaterAid, namely [Facilitator], the one who was with us, to help them organise with all the actors to see how best to redesign the rope pump’ (BF/PS_2).

Attitudes towards technology introduction and assessment

All 41 respondents were unanimous in their opinion that there was a need for a tool like the TAF in the sector. Respondents in each country frequently described the project as an ‘eye opener’ with regard to technology issues in the sector. They believed that by applying a standardised methodology and criteria for assessment, the TAF had the ability to address many challenges relating to technology introduction. They also thought that having standardised criteria would provide grounds for accepting or rejecting technologies and this would enable government organisations, rather than powerful non-state funded actors, to dictate the technologies being implemented. One respondent observed that ‘I have seen cases where technologies have been brought into the country introduced and then they become a disaster but now getting rid of them becomes a problem but this process that has been introduced will definitely help with making sure that whatever is being introduced is appropriate in the country’ (UG/LG_3).

In Ghana, the project had allowed stakeholders within the Community Water and Sanitation Agency to change their perceptions around the introduction of decentralised water technologies. In the early 1990s, the agency passed a decree to standardise the hand pumps used in the country. This policy was introduced to ensure spare parts were available for the hand pumps being used in the country. Stakeholders from this organisation reported that their involvement in the Learning Alliance had allowed them to reconsider their position. They commented that they now believed there was scope for introducing new technologies, as long as it was carried out in a controlled manner. Respondents from this organisation also thought the TAF tool presented the opportunity to do this.

However, there was some evidence, in Ghana and Burkina Faso in particular, which suggested that the Learning Alliances had not promoted the dissemination of new knowledge and practice. For example, a focus group discussion involving a stakeholder organisation in Burkina Faso revealed that knowledge had not passed from their representative in the Learning Alliance to other members of the organisation. Those involved in the focus group were unaware of the progress with the project innovations and needed updating on the project activities. A similar finding was demonstrated in the WASH utility company in Burkina Faso, as well as the private sector organisation taking part in the Learning Alliance in Ghana.

It is possible to identify several factors which may have hindered the dissemination of information. Firstly, the majority of the dissemination activities took place at national level. For example, aside from the regional scoring workshops and a few other workshops in Ghana and Uganda, the majority of presentations appear to have taken place at national level events, one interviewee complained that ‘I think the documents have not been shared; they have just been tested in the field with some ministry officials, with the technical support units, with the district water engineers; so I think that is how far they are in circulation’ (UG/NGO_1).

Secondly, the fact that a tool was being developed during the project, meant that consortium members felt reluctant to share the unfinished versions of the tool. In Burkina Faso, for example, the Learning Alliance members reported how they were waiting for the finished versions of the TAF tool before sharing it at their main sector event. This also applied at the international level – the finalised tools and technology recommendations were not shared until 4 months before the project ended. Despite the concerted efforts to disseminate project information, government and NGO representatives were almost unanimous in their opinions that there was a need to increase awareness of the tools in the sector, especially at the decentralised level.

Stakeholder perceptions of project innovations

Respondents reported a process whereby tools were made more user-friendly throughout the course of the project. In particular, the Learning Alliance members' interaction with the users of technologies during the testing of the tools had allowed the members to adapt the questions in the TAF tool and make them more suitable for the intended audience. A frequent observation was that initially the data collection was very slow and some of the wording of the questions was difficult to understand but after several rounds of testing, the process became faster as questions were cut out and improved. ‘At the beginning it was hard because some of the questions are really not easy to understand but then from criticism and feedback from different people, many questions were simplified, they were rephrased, reformulated and more you know easier to understand’ (BF/LG_1).

However, a common theme to emerge, particularly in Burkina Faso and Uganda, was that the TAF tool was still too complex at the time at which the interviews took place. Respondents reported that the tool was ‘heavy’ or ‘bulky’ and required questions to be removed or simplified. Many respondents felt there was a need for further adaptation before it could be scaled up in the sector and used by those actors who would ultimately be responsible for its use. Hence, ‘Now there are too many questions, it is a bit bulky, so some of the ideas could have been aggregated’ (BF/LG_5).

Again, particularly in Burkina Faso and Uganda, respondents also had concerns about the actual implementation of the tool. The main concerns reported were costs associated with carrying out technology assessment in this manner. These respondents characterised cost in terms of the economic costs, the human resources and the time associated with applying the tool. Taking these factors into consideration, respondents felt there may not be the capacity to implement these tools in the way they had been during the project, at the district level, which is where these procedures would need to be applied. In Burkina Faso, government stakeholders were also aware of the need for financial support around capacity building amongst intended users and were not sure where this money would come from. The project had proposed that it would be from those seeking to introduce technologies. However, there were questions as to whether some of these stakeholders would be willing or able to cover these costs. ‘But one thing I need to point out that is critical is that the testing is a little bit expensive. You need to call so many people and we have been discussing on how we make it a little bit less expensive so that it can also be carried out by the district themselves and even lower level government’ (UG/NG1).

The effectiveness of the Learning Alliances

A challenge identified in managing the Learning Alliances was the ability of facilitating organisations to ensure that regular core group meetings took place and were well attended. Ghana was the only country where the project managed to achieve the expected quarterly core group meetings. Consortium members in each country reported that high level government officials were often too busy to attend core group meetings, or they moved on to different organisations during the course of the project. This is a difficult problem to overcome, particularly in a relatively small project where large sector events and meetings will invariably take precedence over Learning Alliance activities. However, a representative from the Ministry of Water and Environment in Uganda proposed a possible strategy to overcome this problem: encouraging government officials to sign memorandums of understanding at the beginning of a project, thus allowing for the necessary time and budget allocations to be made. The same interviewee explained how this could also help to improve accountability among these individuals and stimulate information sharing within organisations.

Institutional embedding and scale-up

In Ghana and Uganda, government officials explicitly stated that they had plans to develop more formalised procedures of technology assessment and introduction:

With the development of the TAF we are graduating and trying to see how we can make it more formal now and it is more explicit. Because it used to be like we would just enter into a MoU [Memorandum of Understanding] and issues to do with technology would not be taken very seriously but now with the coming of the [the project] I think we are graduating to a more formal process’ (UG/NG_1).

Like I said we have plans to put in place a more formal procedure in trying to use some of the tools we have picked from TAF and [the project] generally, to put in place a more formal tool, so that when new technologies are being introduced, or even an improved technology is being introduced, we will be able to really support its introduction’ (G/NG_2).

Regarding the future use of the tools, the most common response in each country was that it was dependent on national government or national offices of utility companies (21 of 41 respondents). It was explained that national government bodies or agencies generally dictated practice at the local level. These stakeholders also play a key role in disseminating such practice, be it through sector implementations manuals, legislative frameworks or through their various sector networks. As explained by one NGO worker, ‘And the starting point is to work with the ministry to accept it as one of their tools and then it's included in the sector guidelines as a tool to use by everybody and then everybody will use it’ (UG/NGO_2).

There was a clear need for further capacity building in the use of project innovations at the local level. Respondents representing national authorities believed there was a clear need for training among stakeholders at the intermediate level. Many respondents (for example, this Local Government employee) explained that the tools were not necessarily intuitive and if they were to be applied effectively there would need to be a process of orientation and capacity building. ‘So definitely we would need training for people to be able to implement this. Why? Because this requires various skills, it encompasses such criteria like environmental issues, social issues, all of which make a kind of axis of six. So having a good command of these six axes would definitely require some training because people are not necessarily keen to be acquainted with this all at a time unless we train them’ (BF/LG_2).

However, in each country respondents suggested that future adoption was dependent on government having ownership of the tools. Key informant interviews revealed how the Learning Alliance in Ghana had progressed the furthest in terms of having the tools adopted by government organisations. This was indicated by the fact that host institutions such as the Community Water and Sanitation Agency and the Environmental Health & Sanitation Directorate had begun to plan further assessments and training activities beyond the project timeline. The facilitating organisations in Ghana had managed to ensure high levels of participation of key stakeholders from national authorities throughout the course of the project. Respondents from these organisations remarked how this had improved the likelihood of future adoption: ‘The government participation has been very high because we have a stake in it, both the ministry and the agencies, and it goes a long way to tell how well it will be patronised when it comes to its final delivery’ (G/NG_3).

Conversely, facilitating organisations in Uganda had been slow to engage key government agencies, namely the Ministry of Water and Environment (MWE). Learning Alliance members in Uganda felt that this had slowed the institutionalisation process down and may even jeopardise the uptake of project innovations in the future.

I think that from the beginning it would have been easier to have the ministry on board. Then it would have been taken up by the sector quite easily but bringing them midway was a challenge. Having them not develop the drafts as well, as in we were giving them half done, so asking them to improve on it, so they don't have ownership of it. So I think that will make them take longer to institutionalise it’ (UG/NGO_1).

The MWE also has a policy committee which would need to approve the tools before they could be disseminated throughout the sector. Nonetheless, the national government stakeholders including the Ministry in charge of water and sanitation were positive that the TAF tool would be taken up. They mentioned that the project had come at an opportune moment with the recent formation of the Appropriate Technology Centre (ATC) in 2011, which operates under the Ministry of Water and Environment. This body was formed with the specific mandate of overseeing WASH technology use within the sector and this was cited as an indication that WASH technology use was on the government agenda. The ATC had been identified as the institutional host of the tool and these stakeholders believed the project innovations would be used by themselves after further adaptation.

Respondents in Burkina Faso suggested the tools needed to be formally validated by the steering committee of the Programme National d'Approvisionnement en Eau Potable et d'Assainissement (the national drinking water and sanitation supply programme) in order for future adoption and use to take place. However, interviews revealed that there was a low level of government awareness of the tools and the progress of the project in general, at the national level. There had been inconsistent involvement of key institutions such as the national water and sanitation utility company. The respondent representing the host institution also commented that there was a low awareness of the project innovations in their organisation and further capacity building was required. ‘There was irregular participation of this institution in the project. He himself took part in the last Palm Beach workshop where the technology, I mean the tool was introduced. Yes and some results were also presented to the participants. Now this was a draft version and work was to be done. Now, how far this has gone later, this is not known’ (BF/NG_5).

Discussion and conclusion

The findings from 40 semi-structured interviews and one focus group discussion have revealed many positive outcomes from the deployment of Learning Alliances, including increased understanding by individuals of technology performance and an opportunity to form consensus around technology issues. The transformation of this individual capacity building into organisational capacity building is reliant on the ability of those taking part in Learning Alliance platforms to disseminate and institutionalise innovation within their work place (Sijbesma et al., 2007). However, it seems in this case attendees did not find the opportunity to share the new knowledge and practice more widely in their organisations. Some Learning Alliance members also changed role during the course of the project. Unfortunately, dissemination activities undertaken by consortium members did not reach a wide audience. This has impacted even this research's methodology as it was only possible to interview those directly involved in the project as others would not have enough familiarity with the project's activities to provide any useful responses and could not validate the findings. In addition, the interviewees may not have been objective as when someone has invested time and effort in a project they are likely to look at the intervention more favourably than a neutral observer. This may explain why, although dissemination was not wide, respondents did feel that Learning Alliances have worked to bring about demand for a more formalised, documented and explicit process of technology introduction and assessment.

Overall this study has demonstrated that developing tools in a multidisciplinary manner and involving stakeholders at multiple institutional levels provides a fertile ground for the generation of capacity among stakeholders taking part in the Learning Alliances. The use of platforms for interaction at different institutional levels has allowed stakeholders to appreciate the various issues with technology and technology assessment in the respective countries. This research has found research activities at the community level to be a particularly important mechanism in the achievement of this understanding. The research has also revealed some changes in attitude towards technology assessment and introduction policy and practice. Key stakeholders in each country have recognised the importance of adopting formalised and documented procedures. If such procedures are adopted in each sector, this has the ability to bring about a clearer process for non-governmental actors and in turn, improve coordination between sector players.

However, at this early stage, it is not clear how far institutional capacity in technology assessment will develop and whether attitudinal changes will be truly institutionalised and converted into expected norms of behaviour and habitualised practices. This is especially the case now that the project has finished; the Learning Alliance platforms are no longer present and there is no external facilitation in place. Thus, the level of capacity brought about by the Learning Alliances will be determined by the ability of influential stakeholders involved in the project to champion the cause of technology assessment and introduction in the respective sectors.

In order for the true potential of WASH interventions to be achieved, in terms of improving service delivery to end beneficiaries, the changes in attitude brought about by such initiatives need to be converted into policy and practice within the sector. This will allow for rigorous assessments of WASH technologies and lead to the use of technology options, which provide improved levels of service. The research also revealed that stakeholders believed that creating ownership of project innovations among government authorities was an important mechanism in securing adoption at this level. Ideally government officials need to be engaged in a truly participatory manner throughout the processes of problem identification, direction setting and generation of service delivery solutions. Involvement of these stakeholders in each of these processes will ensure innovation is developed in line with the institutional context and will take account of the institutional capacity within the sector.

The main failure of the project that Learning Alliance members identified at this stage is that the TAF still has too many questions, despite the process of refinement over three rounds of field testing. Furthermore, stakeholders expressed concerns that the processes involved in the testing phase required unrealistic levels of resources in terms of expertise, time and money. Thus, at the end of this 3-year project-based Learning Alliance, there remains a gap between what Mikhail & Yoder (2008) describe as conceptual advocacy and practical advocacy. Sijbesma et al. (2007) discuss how the time and resources required to set-up platforms at different institutional levels, makes it difficult to harmonise the Learning Alliance approach with current project approaches to implementation. In this case, the main project innovation was initially externally generated by European partners whilst the Learning Alliances were being set up. They had subsequent input, but the innovations could not be said to have been developed in a truly participatory manner. Pahl-Wostl et al. (2007) suggest that content management and social involvement are interdependent and cannot be separated. The points identified here relating to implementation of the Learning Alliances support this notion. Therefore, perhaps the most important institutional contextual element governing the way project-based Learning Alliances are implemented, lies in the way donors fund such projects. Learning Alliances rely on institutionalisation of innovative practices. These processes are likely to take much longer than the conventional 3-year project timeline. This study has demonstrated the need for advocacy towards donors taking a longer term approach to sustainable development and a more flexible attitude towards project outputs.

Constrained project timelines also influenced the reliability and comprehensiveness of project findings. In many cases, the study was only able to report proximate indicators of change rather than more concrete realised outcomes. The timing of the research means that it is hard to determine whether actors reporting intentions to change behaviour, will indeed execute these changes, or sustain them when the project has finished and is no longer fresh in their minds or being pushed by local partners on the ground.

The current literature points to the key role of intermediate institutional levels in delivering innovations intended for the rural WASH sector (Schouten & Moriarty, 2002; Moriarty et al., 2005; Smits et al., 2007). However, this study has found that interaction at the national level is paramount, as ensuring government ownership of project innovations is the most important mechanism for securing future adoption and scale-up within the sector. The main reason for this being that local government and NGOs receive their directives and financial backing from the national level. These findings are consistent with those of Lockwood & Smits (2011) who observed that in many sub-Saharan African countries, processes of decentralisation are largely theoretical, whereby devolution of funding and service authority has not occurred in practice.

The generalisability of qualitative research is often questioned because the context in which a study takes place is regarded to be paramount and it is thought that no intervention will work in the same way in different contexts. However, there are certain factors, which help to strengthen the generalisability of this study. The project took place in the WASH sectors of three different countries in sub-Saharan Africa, all with different contextual factors at play. Although the interventions and activities of the project varied slightly between the different countries, they were very similar. The sampling, data collection and analysis techniques for this study were applied in the same way for all three countries. Therefore, where outcomes and mechanisms of the project have been found to apply across all three country contexts, this certainly adds weight to the prospect that these findings are likely to be found if interventions are applied to similar contexts.

Acknowledgements

We would like to acknowledge the advice provided by Dr Richard Franceys, Jo Smet, Carmen De Silva Wells, Prof Keith Weatherhead and Pam Furniss at various stages of the research and manuscript preparation. We would like to acknowledge the contribution of WaterAid (UK, Ghana, Burkina Faso and Uganda office), IRC and Skat in the development of the TAF. This work was supported by the EC FP7 Programme to the WASHTech consortium, grant number 266200.

References

References
da Silva Wells
C.
Van Lieshout
R.
Uytewaal
E.
(
2013
).
Monitoring for learning and developing capacities in the WASH sector
.
Water Policy
15
,
206
225
.
Davies
R.
Dart
J.
(
2005
).
The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use
.
Care International
,
UK
.
Evans
P.
(
1992
).
Paying the Piper: An Overview of Community Financing of Water and Sanitation
.
IRC International Water and Sanitation Centre
,
Delft
.
Frjins
J.
Buscher
C.
Segrae
A.
van der Zoewen
M.
(
2013
).
Dealing with future challenges: a social learning alliance in the Dutch water sector
.
Water Policy
12
,
212
222
.
Inkpen
A. C.
(
1996
).
Creating knowledge through collaboration
.
California Management Review
39
(
1
),
123
140
.
Khanna
T.
Gulati
R.
Nohria
N.
(
1998
).
The dynamics of learning alliances: competition, cooperation and relative scope
.
Strategic Management Journal
19
(
3
),
193
210
.
Kimera
P.
Smet
J.
Olschewski
A.
Parker
A.
(
2013
).
Context-specific validation and introduction of technologies for sustainable WASH services
. In:
Paper presented at the 36th WEDC International Conference: Delivering Water, Sanitation and Hygiene services in an uncertain environment
,
Nakuru
,
June 2013
.
Lockwood
H.
Smits
S.
(
2011
).
Supporting Rural Water Supply; Moving Towards a Service Delivery Approach
.
Practical Action Publishing
,
Rugby
,
UK
.
Lundy
M.
Gottret
M. V.
(
2007
).
Learning alliances: Building multi-stakeholder innovation systems in agro-enterprise development
. In:
Learning Alliances: Scaling Up Innovations in Water, Sanitation and Hygiene
.
Smits
S.
Moriarty
P.
Cristine
S.
(eds).
IRC International Water and Sanitation Centre
,
The Hague
, pp.
37
54
.
Mikhail
M.
Yoder
R.
(
2008
).
Analysis of the MUS learning alliance process in Nepal
. In:
Paper presented at the International Symposium of the Multiple-Use Water Services (MUS) Group Addis Ababa, Ethiopia
,
4–6 November 2008
.
Moriarty
P.
Fonseca
C.
Smits
S.
Schouten
T.
(
2005
).
Background paper for the symposium on learning alliances for scaling up innovative approaches in the water and sanitation sector
.
IRC International Water and Sanitation Centre
,
Delft
,
The Netherlands
.
NVivo qualitative data analysis software; QSR International Pty Ltd
.
Version 10, 2012
.
Olinto
P.
Uematsu
H.
(
2013
).
The State of the Poor: Where are the Poor and where are they Poorest?
The World Bank
, .
Pahl-Wostl
C.
Craps
M.
Dewulf
A.
Mostert
E.
Tabara
D.
Taillieu
T.
(
2007
).
Social learning and water resources management
.
Ecology and Society
12
(
2
),
5
.
Pawson
R.
(
2006
).
Evidence-Based Policy: A Realist Perspective
.
Sage
,
London
.
Pawson
R.
Tilley
N.
(
1997
).
Realistic Evaluation
.
Sage Publications
,
London
.
Proost
J.
Lewis
C.
(
2007
).
Learning alliances between power and impotence: underpinnings and pitfalls from innovation and social learning theory
. In:
Learning Alliances: Scaling Up Innovations in Water, Sanitation and Hygiene
.
Smits
S.
Moriarty
P.
Cristine
S.
(eds).
IRC International Water and Sanitation Centre
,
The Hague
, pp.
20
33
.
Reed
M. S.
Evely
A. C.
Cundill
G.
Fazey
I.
Glass
J.
Laing
A.
Newig
J.
Parrish
B.
Prell
C.
Raymond
C.
Stringer
L. C.
(
2010
).
What is social learning?
Ecology and Society
15
(
4
),
1
.
Röling
N.
(
2002
).
Beyond the aggregation of individual preferences
. In:
Wheelbarrows Full of Frogs: Social Learning in Rural Resources
.
Leeuwis
C.
Pyburn
R.
(eds).
R. Van Gorcum
,
Assen
, pp.
25
47
.
RWSN
(
2009
).
Myths of the Rural Water Supply Sector, Perspectives No. 4, RWSN Executive Steering Committee
.
Rural Water Supply Network
,
St Gallen
,
Switzerland
.
Schouten
T.
Moriarty
P.
(
2002
).
Community Water, Community Management: From System to Service in Rural Areas
.
ITDG Publishing
,
London
.
Sijbesma
C.
Smits
S.
Moriarty
P.
(
2007
).
Learning about learning alliances – reflections on findings and future
. In:
Learning Alliances: Scaling Up Innovations in Water, Sanitation and Hygiene
.
Smits
S.
Moriarty
P.
Cristine
S.
(eds).
IRC International Water and Sanitation Centre
,
The Hague
, pp.
145
163
.
Smits
S.
Moriarty
P.
Fonseca
C.
Schouten
T.
(
2007
).
Scaling up innovations through learning alliances: an introduction to the approach
. In:
Learning Alliances: Scaling Up Innovations in Water, Sanitation and Hygiene
.
Smits
S.
Moriarty
P.
Cristine
S.
(eds).
IRC International Water and Sanitation Centre
,
The Hague
, pp.
145
163
.
United Nations: Department of Economic and Social Affairs, population division
(
2004
).
World Population to 2300
.
United Nations
,
New York
.
Van der Steen
P.
Howe
C.
(
2009
).
Managing water in the city of the future; strategic planning and science
.
Reviews in Environmental Science and Biotechnology
8
,
15
120
.
Visscher
J. T.
Garcia
M.
Röling
N.
(
2007
).
Multi-stakeholder learning in Colombia; from TRANSCOL to team learning projects
. In:
Learning Alliances: Scaling Up Innovations in Water, Sanitation and Hygiene
.
Smits
S.
Moriarty
P.
Cristine
S.
(eds).
IRC International Water and Sanitation Centre
,
The Hague
, pp.
61
77
.
WASHTech
(
2016
).
Case studies
.
(Accessed April 29, 2016)
.
WHO & UNICEF
(
2015
).
Progress on Sanitation and Drinking-water 2015 Update and MDG Assessment
.
WHO Press
,
Geneva
,
Switzerland
.