Globally, water utilities are grappling with the challenge of predicting the condition of deteriorating pipe network infrastructure amidst financial constraints and data-scarce scenarios. As a result, new innovative approaches such as statistical regression and Markov-based approaches have been introduced to aid water distribution pipe renewal decision making. However, comparison of the performance of these models under limited data has not been undertaken so far. In addition, the models have been applied elsewhere, in different environments and data availability scenarios. This paper addresses therefore the mentioned research gap and compares the performance of statistical regression and Markov models in the prediction of a condition of a pipe in a developing country. In addition, the criticality analysis of a block is studied. The data used for assessment is from Kampala water, the largest area in the National Water and Sewerage Corporation, Uganda. The results show that 78.26% of the prediction of the regression model is accurate in comparison to 88.4% for the Markov model. This means that the Markov-based approach is more superior than a regression model in a data scarce scenario. The approach will go a long way in helping water utilities in development of water decision pipe renewal plan amidst a limited budget and in data scarce scenarios.