Safe water is essential for life. Consumption of arsenic and manganese contaminated water poses a range of health effects to humans. Physico-chemical and bacteriological characteristics of groundwater at five administrative upazillas in Kushtia District, Bangladesh, have been studied to evaluate the potability of water for drinking purpose from 32 randomly selected tube wells (TWs). APHA (2012) standard analytical methods were applied for analyses of the physico-chemical and bacteriological parameters of the water samples. Arsenic, iron, and manganese content were analyzed by atomic absorption spectroscopy (AAS). The investigated parameters of water samples were found as pH 6.81–8.12, electrical conductivity (EC) 520–1,995 μs/cm, total dissolved solids (TDS) 357.8–1,372.6 mg/L, chloride 10–615 mg/L, total hardness 285–810 mg/L, arsenic (As) 0.001–0.098 mg/L, iron (Fe) 0.04–1.45 mg/L, manganese (Mn) 0.01–6.32 mg/L. About 56.25% of TWs were highly contaminated with fecal coliform (FC) and 68.75% were found to be contaminated with total coliform (TC). Results were compared with World Health Organization (WHO) and Bangladesh Drinking Standards (BDS). The concentrations of water quality parameters are much higher as compared to WHO and BDS standards. This may cause acute public health risks and make water unsuitable for direct human consumption without treatment.