The design and operational practice of water supply schemes (WSSs) in India is discussed in the context of the prevailing performance of the systems. Issues such as the tremendous gap in design and operation, unskilled manpower, and unmanageably large operation zones are identified as the main causes of the failure of WSSs in India. The failure of large-scale infrastructure creation in meeting service requirements is also discussed. The optimum scale of operation is estimated based on the ideal design and prevailing population densities in India. This work proposes a decentralized infrastructure approach through the use of multi-outlets tanks, shafts and manifolds. Each of these components is described in detail, and the manner in which these will help improve the current operation of WSSs in India is documented. Specifically, multi-outlet tanks will help achieve a design that enables the establishment of district metering areas for water distribution, shafts will serve as hydraulic separators between subzones, and manifolds will achieve a controlled water supply at various stages of the WSSs. These simple low-cost components will help to achieve water supply operations with minimal need for human intervention and automation.