As water use is closely associated with economic activities, a growing population, agricultural development, and reallocation of water resources, the Heihe River Basin highlights common issues of water productivity and groundwater overexploitation. We conducted a quasi-dynamic input–output analysis to investigate the changes to water use among different sectors of Zhangye city from 2002 to 2012, and clarified the driving mechanism of these changes. Our results indicated that the direct agricultural water use coefficient (calculated with an input–output table and other water use data) was the largest among the coefficients from all the sectors; notably, the fishery sector still consumed about 1 m3 of water per unit output in 2012. We found that the water-saving technologies clearly contributed to the decrease of agricultural water from 2002 to 2007, but induced a rebound in total water use from 2007 to 2012. This study provides insights into the challenges of water resource management in the Heihe River Basin and sheds light on potential water-saving strategies for the future. This study may also enhance the policy relevance of land use governance and industrial transformation. A comprehensive exploration of the water–ecosystem economy is critical to integrated water resource management.