Monitoring sanitation performance: unpacking the figures on sanitation coverage

An understanding of the sanitation situation is crucial for planning and evaluating effectiveness of sanitation interventions. Central to gaining such knowledge is the need to monitor sanitation performance based on clear relevant indicators. This is very important considering the role sanitation plays in socio-economic development and disease prevention as consistently shown in numerous studies (Esrey et al. 1991; Norman et al. 2010; Hutton 2012).

Internationally, various efforts have been made to improve the sanitation situation. These include the United Nations Water Conference in Mar del Plata in 1977, which preceded the proclamation of the 1980–90 International Drinking Water Supply and Sanitation Decade (IDWSSD) (Neto & Tropp 2000); the formulation of the International Development Targets (IDTs) by the Development Assistance Committee (DAC) of the Organization for Economic Cooperation and Development (OECD) (OECD-DAC 1996), which preceded and defined the Millennium Development Goals (MDGs) (United Nations 2000; Gabay 2012); the Johannesburg World Summit on Sustainable Development (WSSD) (United Nations 2002), and most recently the adoption of the Sustainable Development Goals Report by the United Nations General Assembly (United Nations 2014). All these efforts, in particular the MDGs, have played a crucial advocacy role in raising awareness on sanitation (Tilmans et al. 2015). This has enabled sanitation coverage to be raised from 49% in 1990 to 64% in 2012 (WHO/UNICEF 2014).

Despite the benefits of sanitation and the various efforts made to improve sanitation, the latest figures show that globally, about 2.5 billion (10⁹) people do not have access to improved sanitation (WHO/UNICEF 2014). With only a few months left before the 2015 deadline for the MDGs, it is now certain that the world will not meet the MDG target on sanitation. The sanitation target as defined at the World Summit on Sustainable Development in Johannesburg in 2002 aims at ‘halving the proportion of people without sustainable access to … basic sanitation by 2015’ (United Nations 2002; Walgate 2002).

Although the sanitation coverage figures give an indication of the sanitation status with regard to progress made towards meeting the sanitation target, there is increasing attention on the approach used by the Joint Monitoring Programme (JMP) in monitoring sanitation and many researchers have questioned the sanitation coverage figures derived thereof (UN-Habitat 2003; Zawahri et al. 2011; Baum et al. 2013; Cumming et al. 2014). This is mainly because of the simplified indicators and targets used by the JMP which have failed to cover the key aspects of the MDG sanitation target (World Bank 2008). These aspects include sustainability issues such as environmental protection, convenience, affordability and maintenance of sanitation facilities. With the sanitation research agenda shifting more towards mainstreaming sanitation into the concept of sustainable development, it is becoming more important that the sustainability of sanitation systems be included in sanitation monitoring.

Presently, it is not clear whether the current sanitation trend would be sustained or if the sanitation situation would further deteriorate (Sparkman 2012) because sustainability is not monitored. Criticism of the JMP estimates also emanates from the likely bias and error introduced through data collection methods, survey instruments, linear regression analysis, and differences in definitions of rural/urban and improved sanitation facilities (Bartram et al. 2014). This paper argues that increasing access to improved sanitation is only a part of the solution towards solving the sanitation problem; the sustained use of the sanitation facilities and the proper management of waste is required in order to maximize the benefits of sanitation. Accordingly, this paper contends that the current sanitation coverage figures are overestimated and do not accurately reflect the real picture regarding sanitation status. It is therefore suggested that reliable information on the patterns, trends and projections of sanitation coverage must be obtained from a sanitation monitoring approach which considers the sustainability and functions of the different sanitation systems.

Against the above background, it becomes crucial to understand the sanitation monitoring approach from which sanitation coverage figures are obtained. To do this, the paper firstly provides a conceptual framework which is used to understand the meaning of sanitation coverage. Secondly, it briefly describes the JMP sanitation monitoring approach. In order to understand the significance of JMP sanitation coverage estimates in explaining the sanitation situation, the strengths and weaknesses of the approach are discussed, supported by evidence from research studies. The paper is therefore based on a literature review of the JMP sanitation monitoring approach and sanitation coverage research studies.

This paper section presents a conceptual framework for understanding the significance of sanitation coverage figures. The analysis is based on concepts of the sanitation system and sustainability.

In this paper, the term ‘sanitation’ is used to refer to the collection, storage, treatment, disposal and re-use of human excreta and associated wastewater. This definition shows sanitation as a system with different subsystems or components as shown in Figure 1.

A sanitation system is defined as a context-specific series of technologies and services involved in the collection, containment, transport, transformation, utilization or disposal of human excreta and wastewater (Tilley et al. 2014b). It can be argued that for sanitation to perform its primary function of protecting health, environment and recycling of nutrients (Ridderstolpe & Rivera 2007) the sanitation system must be sustainable.

The concept of sustainable sanitation has raised much interest among academics and sanitation practitioners following the adoption of the sustainable development approach. The concept of sustainable development, popularized by the 1987 Brundtland Commission Report (Estes 1993; Basiago 1999) is described as, ‘development that meets the needs of the present without compromising the ability of future generations to meet their own needs’ (WCED 1987). From this definition, two major issues can be drawn out which are relevant to sustainable sanitation systems. There is the issue of durability and the maintenance of ecological integrity. For a sanitation system to be sustainable it must meet long-term goals (durable) and waste must be managed in a way that does not upset the resource base.

However, it must be noted that despite the increased focus on sustainable sanitation, the concept has been subject to multiple interpretations and the literature reveals a lack of consensus on the definition of the concept. This is evidenced by different definitions where sustainable sanitation is defined in terms of either the sanitation technology, sanitation service or sanitation system (Balkema et al. 2002; Brikke & Bredero 2003; Bracken et al. 2005). The interests of different stakeholders seem to dictate what needs to be sustained, hence the varied definitions. However, the Sustainable Sanitation Alliance (SuSanA) defines a sustainable sanitation system as one which is economically viable, socially acceptable, technically and institutionally appropriate and one that should also protect the environment and the natural resources (SuSanA 2008). In assessing the sustainability of sanitation systems, therefore, a balance of the different sustainability aspects must be achieved. Different researchers have suggested sustainability criteria which can be used in assessing the sustainability of sanitation systems. The sustainability criteria can be summarized as health, environment, economy, socio-culture and technical function (Balkema et al. 2002; Bracken et al. 2005; Lennartsson et al. 2009).

The definition of sanitation and the brief description of sustainable sanitation provided above suggest the need to consider sanitation beyond the user interface (toilet). There is a need to focus on the whole sanitation system. The unpacking of sanitation figures, which is the theme of this paper, will therefore attempt to analyze the sanitation monitoring approach based on its adherence to the sustainable sanitation concepts.

The global sanitation monitoring approach has gone through a number of significant changes. Initially, in the 1960s, the World Health Organization (WHO) produced sanitation status reports based on information collected from country agencies and Ministries of Health (Cotton & Bartram 2008; Brocklehurst 2012). However, the failure of the IDWSSD, which ran from 1981 to 1990, to achieve its aim of providing safe water and sanitation for all by 1990, led to major changes to the monitoring approach. It was argued that the data used did not give the true picture of the status of sanitation as governments tended to overstate coverage figures (Cotton & Bartram 2008; O'Hara et al. 2008). Furthermore, national authorities provided incomplete information which could not be independently verified and the authorities failed to explain the meaning of the term ‘access’ (Bartram et al. 2014). It was thus recommended that monitoring be based on household surveys where users would provide more accurate information on sanitation facilities in terms of sanitation facility type, functionality and usage (Bartram et al. 2014). While these recommendations were made after the IDWSSD in 1990, reporting continued to be done based on data from government sources. This could probably have been due to lack of data as the first of the two major household surveys was only launched in 1985. This first survey was the Demographic Health Survey (DHS) launched by United States Agency for International Development (USAID). The second survey to be launched was the Multiple Indicator Cluster Survey (MICS) introduced by the United Nations Children's Fund (UNICEF) in 1995.

The JMP for Water Supply and Sanitation of WHO and UNICEF was formed in 1990 to monitor global sanitation using exclusive data from household surveys and national censuses (Cotton & Bartram 2008; Brocklehurst 2012). The JMP derives its mandate from Goal number 7 of the MDGs within which the sanitation target falls. The goal is aimed at ensuring environmental sustainability. The MDG sanitation target, which was revised many times before its adoption in 2006 (Bartram et al. 2014) seeks to ‘halve the proportion of people without sustainable access to basic sanitation by 2015’ (United Nations 2008). However, following recommendations from WHO and UNICEF, the MDG sanitation monitoring is based on the indicator; ‘proportion of population using an improved sanitation facility’ as agreed in 2006 (United Nations 2008) possibly due to data and resource limitations (Bartram et al. 2014). The term ‘improved sanitation’ is based on the type of facilities used for the disposal of human excreta. Improved sanitation technology is one which hygienically separates human excreta from human contact (WHO/UNICEF 2014). It includes a private flush or pour-flush toilet or latrine connected to a piped sewer system or septic system, a simple pit latrine with a slab, a ventilated improved pit (VIP) latrine or a composting toilet. Unimproved sanitation includes any other flush or pour-flush latrine, an open pit latrine, bucket latrines, a hanging latrine, any public or shared facility or open defecation (WHO/UNICEF 2014). The different categories of improved and unimproved sanitation facilities are shown in Table 1.

The JMP sanitation coverage estimates are based on fitting a regression line to a series of data points from household surveys and censuses (WHO/UNICEF 2014). Simple linear regression is used to estimate the proportion of the population using improved types of sanitation facilities (including shared facilities of an improved type) and open defecation, while the remaining population is considered to use unimproved sanitation facilities. The proportion of the population using shared sanitation facilities of an improved type is then subtracted from the estimates of improved sanitation facilities (WHO/UNICEF 2014). However, before the data from surveys can be used to estimate sanitation coverage, each response category of the harmonized core questions used in surveys (Table 2) is scrutinized to ensure that there are no discrepancies (Wolf et al. 2013).

The use of harmonized core questions and response categories is aimed at improving comparability and accuracy of estimates (WHO/UNICEF 2006). However, due to addition of new data, sanitation coverage estimates may differ from and may not be comparable to earlier estimates for the same reference year (including the 1990 baseline year).

The JMP approach provides a simple and robust way of monitoring sanitation performance. The JMP provides crucial information on sanitation coverage through its biennial reports. In reporting sanitation progress, the JMP uses coverage levels shown as percentage of population that uses improved sanitation facilities disaggregated into the rural, urban and national population. Disaggregation is also based on wealth quintiles. The disaggregation is important in that it shows progress in each sector thereby allowing for better targeting of resources and efforts. In addition, in 2012, the JMP introduced a new performance indicator which shows the absolute number of people in a given year who have gained access to improved sanitation facilities since the year 2000. This index is an incentive to countries which are either struggling with high population growth and/or countries that started out with low baseline coverage in 2000 (WHO/UNICEF 2014) since progress has been shown to depend to an extent on the starting point (or legacy) (Anand 2006). This indicator uses 2000 figures as the baseline and not 1990 because there are large data sets available in 2000 compared to 1990. The JMP also reports on whether countries are on-track or off-track to meet the MDG sanitation target by 2015.

The sanitation information reported by the JMP is important for planning purposes. Reporting of information is made possible by use of a clearly defined simple indicator and target. In addition, the JMP has accrued a huge collection of standardized data sets which are obtained from users and can be used to determine national, regional and global sanitation trends (Cotton & Bartram 2008; O'Hara et al. 2008; van der Hoek et al. 2010). In 2014, for example, there was a total of 1797 data sources comprising of national censuses, household surveys and government reports (Bartram et al. 2014). The fact that the JMP continually adds new data whenever it becomes available is envisaged to improve accuracy of data with time (van der Hoek et al. 2010).

While the JMP approach has been useful in reporting progress towards sanitation targets, the appropriateness of the indicators used to monitor sanitation access and type of data used in the assessment has been questioned. The major concern has been on the use of the term ‘improved’ sanitation facilities and the technology-based sanitation ladder and whether such an approach reflects the primary functions of sanitation. To this effect various studies have been conducted focused on understanding the weaknesses of the JMP approach in as far as significance of sanitation coverage is concerned.

An analysis done by UN-Habitat in selected low-income urban areas from Africa, Asia, and Latin America and The Caribbean showed huge discrepancies between access to ‘improved’ sanitation and access to ‘adequate’ sanitation for 2000 (UN-Habitat 2003) (Table 3).

While the improved sanitation coverage estimates are derived from the JMP, indicative estimates for adequate sanitation are based on a review of detailed studies of a few selected areas (McGranahan & Satterthwaite 2006). It is difficult to translate adequate sanitation figures derived from the few studies to global statistics. The figures, however, suggest that there are many more urban dwellers that do not have access to adequate sanitation than those without access to improved sanitation. The huge discrepancies between improved and adequate sanitation figures can be attributed to the different attributes of the two definitions. In improved sanitation, coverage is measured based on access to or use of a technology meaning that only one component (collection) of the whole sanitation system is considered. On the contrary, adequate sanitation considers the sustainability of the whole sanitation service chain as it accounts for issues of affordability, convenience, maintenance, safety and environmental protection (UN-Habitat 2003). This is confirmed by the core questions and response categories used in the DHS and MICS household surveys (Table 2). The majority of the questions focus on type of sanitation facility which is used by the household and there are no questions on issues like maintenance, affordability, treatment and re-use of waste. With a limited focus, sanitation coverage figures are bound to be higher when improved sanitation is used than for adequate sanitation. It is thus argued that improved sanitation based on access to a certain sanitation technology alone is not a good indicator to assess sanitation as it does not reflect the primary functions of sanitation.

In another more recent study by Baum et al. (2013), assessing sanitation coverage based on access to improved sanitation facilities was criticized for its failure to consider quality of services offered by the facilities. In the study, the authors re-assessed sanitation progress so that the new sanitation coverage figures reflected protection of communities and the wider population from exposure to human excreta. This was done by taking into account prevalence of sewage treatment. While the JMP classifies any connections to sewerage systems as improved facilities, in the study by Baum et al. (2013), it was only the connections whose sewage was treated before discharge that were defined as improved. The study found that in 2010 about 1.5 billion (10⁹) people out of the 4.5 billion (10⁹) reported by the JMP to have access to improved sanitation globally used sewage connections without treatment. Discounting this population resulted in a 22% reduction of the global improved sanitation coverage figure for 2010. This result shows that overlooking a key function of sanitation such as wastewater treatment in monitoring can only lead to inaccurate sanitation coverage statistics.

The results from the above study are confirmed by studies done by other researchers (Schertenleib 2005; Zawahri et al. 2011). Research shows that in Asia, out of 85 centralized wastewater treatment systems, less than 40% of these systems are functioning properly (Schertenleib 2005). This means that wastewater from the other 60% of the treatment plants is being discharged partially treated or not treated at all. Globally, Asia accounts for the largest proportion (69%) of the 1.5 billion (10⁹) people whose wastewater is discharged without treatment (Baum et al. 2013). A similar situation prevails in Sub-Saharan Africa (SSA) where most of the wastewater is discharged untreated or partially treated (Table 4).

The wastewater treatment figures are available for different years for the selected countries and no recent figures are available making it difficult to do inter-country comparisons, it can, however, be argued that the figures are indicative of the current situation. It is worth noting that while there seems to be a correlation between percentage of untreated wastewater and sanitation coverage for Burkina Faso and Ghana, the same cannot be said for Senegal and South Africa. The former two countries have very low sanitation coverage which tends to correspond to the high percentages of untreated wastewater while in contrast the latter despite having high sanitation coverage also have high percentages of untreated wastewater. Inadequate sanitation is the major cause of sanitation-related diseases in SSA. For example, despite having a fairly high urban sanitation coverage, Zimbabwe experienced its worst ever cholera outbreak in 2008, mainly attributed to sewer overflows and inadequate water supplies (Government of Zimbabwe 2011; Banana et al. 2015).

Similarly, the same situation prevailing in Asia and Sub-Saharan Africa was observed for the Middle East and North Africa (MENA) Region, which is considered by the JMP to be making adequate progress in sanitation with coverage figures of 93%, 97% and 100% for Iran, Libya and Israel respectively (Zawahri et al. 2011; WHO/UNICEF 2014). Despite the high sanitation coverage, these figures nonetheless do not reflect the sanitation situation on the ground as most MENA states like Palestine discharge approximately 80% of their untreated wastewater despite being connected to a sewerage network (Zawahri et al. 2011).

Further analysis of the JMP method shows that sanitation monitoring is carried out using the same pre-defined technology-based sanitation ladder for different settings including densely populated urban areas and sparsely populated rural areas. However, these areas present different challenges for sanitation provision with urban areas faced with more challenges than rural areas. The challenges in urban areas include the presence of heterogeneous populations (in terms of culture, race and religion), insecurity of land tenure, limited space, low incomes, limited options for treatment and disposal of waste, limited choice of sanitation technology and institutional fragmentation (Lüthi et al. 2010; Satterthwaite et al. 2015). Since the sanitation ladder is predefined and thus hampers innovation and creativity to meet needs within the local context (Kvarnström et al. 2011), it therefore means that communities are forced to adopt the technologies specified in the sanitation ladder, irrespective of the communities’ needs and challenges. For example, dense urban communities using improved VIP latrines are included in sanitation coverage figures to the same extent as sparsely populated rural areas using the same technology. This is despite the fact that use of VIP latrines in urban areas might be inadequate because of space constraints and lack of pit-emptying strategies, unlike in rural areas where space is not a challenge. In this regard, this paper challenges the sanitation coverage figures for rural and urban areas. It is conjectured that a re-calculation of sanitation coverage for urban areas, by discounting the use of technologies such as VIP latrines in densely populated areas, could lead to a significant reduction in the urban sanitation coverage figures, from the current 80% (globally) and 41% (Sub-Saharan Africa) to figures which are similar to those being obtained in rural areas.

The studies presented in this paper have shown that sanitation coverage figures coming out of the current monitoring approach by the JMP are overestimated. While the inclusion of different sanitation systems in the surveys can be taken to mean that the JMP actually assesses the different sanitation systems from the point of waste generation to end of pipe, this is not so, as it has been shown that the JMP bases its coverage statistics on access to technology (user interface) only. This approach has thus been criticized as it is argued that it overestimates sanitation figures.

Although the highlighted studies attempted to give an insight into the actual sanitation status, it is argued that they still failed to consider the key aspects of a sustainable sanitation system. This is based on the following points. Firstly, the study by Baum et al. (2013) focused on untreated wastewater from sewage connections. It is argued that this is just one source of environmental pollution and potential cause of diseases. Burst sewer pipes, and leaching liquids from pit latrines and sewer tanks, all have the potential to pollute the environment. There is substantial evidence to suggest that other sanitation facilities classified as improved have the potential to contaminate the environment at different points of the sanitation system.

A study carried by Dzwairo et al. (2006) in Zimbabwe showed that pit latrines, while classified as improved, had a microbiological impact on groundwater quality. In addition, a recent study by the World Bank focusing on sludge management from on-site sanitation systems in 12 country cities, found that there was prevalence of illegal dumping of sludge, widespread unhygienic manual emptying of sludge and a general lack of appropriate sludge treatment and disposal facilities (Peal et al. 2014).

Secondly, studies by Zawahri et al. (2011) and Baum et al. (2013) disputed sanitation figures based on one pillar of sustainable sanitation, environmental protection, disregarding the economic, socio-cultural and technical aspects of the sanitation system. It is therefore, argued that a further decrease in sanitation coverage could be observed if other improved sanitation facilities that pose health and environmental risks were to be discounted. Further, discounting improved facilities based on their failure to meet other sustainability criteria would have additional impact on sanitation coverage. While the analysis by UN-Habitat (2003) based its analysis on adequate sanitation it is difficult to criticize the approach used because it has not been made clear how the analysis was done (Schertenleib 2005).

All the factors presented above further support the need to take into account the sustainability of the whole sanitation system when assessing sanitation. This is important because the sanitation service chain is only as strong as its weakest link, thus failure of one component will result in failure of the whole system (Galli 2014).

To this end several approaches have been proposed. These include the sanitation service ladder developed by the International Water and Sanitation Centre's (IRC) WASHCost program (Potter et al. 2011) and the function-based sanitation ladder proposed by Kvarnström et al. (2011). The sanitation service ladder specifies sanitation service levels that are provided across the sanitation system, from containment to end-disposal or processing and use. Service levels are assessed based on four key parameters: access, use, reliability, and environmental protection (Table 5).

The implication is that adequacy of sanitation systems will be based on the service parameters and the same sanitation system can offer different service levels depending on its adherence to the four service parameters. This paper contends that this approach is better than the technology-based sanitation ladder by the JMP where all improved sanitation technologies are treated the same irrespective of their performance on the different sustainability aspects. However, the use of only four parameters in the WASHCost sanitation service ladder is not enough to give a complete understanding of the sustainability of sanitation systems (Sparkman 2012) and hence sanitation coverage if SuSanA's definition of sustainable sanitation is considered. There is a need to include other sustainability aspects such as acceptability, affordability and durability of the technology.

In the function-based sanitation ladder, Kvarnström et al. (2011) proposed an evaluation of sanitation services based on their performance on health and environmental functions. It is worth noting that the proposed ladder comprising of seven rungs will have the lower rungs addressing health issues, while the higher rungs include environmental protection and re-use and recovery of resources. Although this approach recognizes the primary functions of the sanitation system, the approach can only be feasible in the presence of an enabling environment which supports the sustainable use of alternative sanitation technologies (Ridderstolpe & Rivera 2007; Kvarnström et al. 2011; Tilley et al. 2014a).

The paper has shown that while the JMP provides a simple and robust way of monitoring sanitation performance, the sanitation coverage figures derived thereof are overestimated. This is based on the fact that they fail to reflect the sanitation situation on the ground. While the statistics are crucial for policy-making, they are problematic as the approach fails to cover key aspects of sanitation. Therefore, there is a risk of overlooking and not giving enough attention to important sanitation-related issues. In summary, it is argued that monitoring sanitation must take cognizance of the need to balance the technological, environmental, social and economic sustainability dimensions. This is in recognition of the inter-linkages between the different sustainability dimensions. It is also argued that this will improve understanding of the sustainability of improved technologies as identified by the JMP. Consequently, this will help to inform sustainable sanitation coverage. An innovative sanitation monitoring approach which considers the primary function of sanitation and sustainability of sanitation systems is therefore recommended.