The unreliability of water supplies in developing countries is a widely recognised concern. However, unreliability means different things in the variety of literature on water supplies, and no unified definition or assessment criteria exists. We review definitions of water supply reliability used in existing literature, as well as the various ways in which it is assessed. Thirty-three papers were selected for review that reported on reliability of domestic water supply and were based on empirical research in developing countries. Explicit definitions of reliability are given in four out of the 33 papers reviewed. These definitions vary, but features common are the functionality of the water supply system itself, and the extent to which it meets the needs of water users. Assessment criteria also vary greatly, with the most common criterion in urban settings being the duration/continuity of supply in hours per day, while in rural settings, the proportion of functional water systems is commonly used. The heterogeneity in the definitions and assessment criteria found in the review is perhaps indicative of a multi-attribute nature of the concept of reliability and any unifying definition and assessment criteria might do well to take this into account.