The alarming state of safe water deprivation among the residents of rural communities in Nigeria is well recognized. Unfortunately, research that shows the policy gaps in the water supply sector of the country and measures to eliminate them in order to improve water supply sustainability in the country is lacking. This paper investigates the landscape of water service provision to rural communities in Nigeria using investigative and qualitative approaches due to the desire to explore the experience and opinions of previous workers/agencies in the region. Primary and secondary data were used in the study. Findings characterized the rural water supply landscape in the region. The community-based service providers are constrained by several policy gaps that negatively impact on the quality and sustainability of rural water supply in the country. Rural water interventions suffer from a high rate of failures due largely to weak institutional framework in the water supply sector. The paper recommends that for rural water supply sustainability to be improved in Nigeria all the stakeholders must work together to prioritize and address the policy gaps constraining service delivery simultaneously in the region. Working on one factor alone may not result in sustainable services.