Dissolved organic carbon (DOC) and chromophoric dissolved organic matter (CDOM) in rivers and reservoirs on the western Loess Plateau, which is an area of severe soil erosion, were investigated in September 2017 to analyze the CDOM characteristics and composition, DOC distribution and influence of environmental factors on these parameters. Great differences of water parameters were exhibited between different groups based on the analysis of variance (p < 0.01). The results indicated that rivers exhibited higher DOC concentrations (mean: 3.70 mg/L) than reservoir waters (mean: 2.04 mg/L). Artificial and agricultural lands exert a large influence on DOC concentrations, which verifies the hypothesis that intense anthropogenic activity results in high DOC concentrations. The CDOM absorption at 350 nm [aCDOM(350)] of tributary water samples was 2.73 m−1, which was higher than that in the Yellow River (1.71 m−1) and reservoir waters (1.33 m−1). The effects of DOC, TC and turbulence (Tur) on CDOM are positive and significant (p < 0.05) according to the multiple linear regressions. An analysis of the optical characteristics of CDOM indicated that waters on the Loess Plateau contained abundant humic acid and higher levels of allochthonous DOM with a higher molecular weight (MW) based on the spectral slopes (S) and specific UV absorbance (SUVA254) values.