Ammonium pollution of source water has become a challenge in rural water supply. Biological roughing filtration combining with low energy consumption and simple operation would be a potential solution to this issue. This study was conducted to investigate ammonium removal by biological up-flow roughing filter packed with ceramic media. Low flow rate did affect the ammonium removal and higher flow rate was suggested, while intense backwashing only showed a slight impact. In 4 m/h, an average reduction of about 51% was obtained, and NH4+-N effluent concentration could below 0.5 mg/L within a NH4+-N loading rate of 0.1 kg/(m3·d). Biomass and biological activity assessment were performed as well as microbial community analysis. High abundance of nitrifying bacteria contributed to ammonium removal with Nitrospira and Nitrosomonas accounting for 6.59% and 1.12% of bacteria community, respectively. In addition, roughing filter showed high turbidity removal efficiency of about 70%. This study suggested that biological roughing filter could be employed to help rural drinking water plants adapt to the seasonal change and moderate deterioration of source water quality in terms of ammonium pollution with low-cost and simple operation.