Agricultural soil as a potential source of input of heavy metals into the lacustrine sediments of Chaohu Lake via soil erosion and weathering has not been well documented. In the present study, rice soil samples along two main tributaries of Chaohu Lake were collected to determine 30 metal elements which were compared with the data on the elements in surface sediments of Chaohu Lake to explore the impacts of soil erosion on the accumulation in the sediments. The results showed that the concentrations of most elements were less than their soil background values of Anhui Province, but Se, Sb, Sn, As, and Ba were greater than the corresponding background values. Furthermore, these elements except for Sc in the soils appeared a low variability in spatial distribution. More than 90% of agricultural soils were not expected with significant contamination by the residual elements based upon the consideration of pollution level index (PLI) combining probabilistic density function. Compared with the residual elements in the surface sediments of Chaohu Lake, most elements (Ti, Mn, Zr, Sr, Zn, Ga, Hf, U, Se, Sn, Nb, Th, V, Cd) appeared depletion in agricultural soils but enrichment in sediments. Specially for Cd, it was not detectable in all soils samples, but it existed at an extreme average content in the lacustrine sediments. These results possibly implied that elements in agricultural soils in Chaohu Lake were depleted because of soil erosion and weathering during long-term agricultural practice, resulting in enrichment in sediments of the lake. The depletion of elements in soils was predominately due to the fact that these metals existed in the soil with a large mobile fraction. While As, Th and Se moderately accumulated in the soils due to their insusceptibility of transport potentials. Because Sr and U are easier to be leached from soils than Rb and Th, respectively, a greater ratio of Rb/Sr and a lower value of U/Th was found in soil samples than in sediments. Therefore, our study implied agricultural soils could play an important role in the input of heavy metals into sediments via soil erosion and weathering.