Abstract Under the background of high-quality development, it is more practical significance of how to use the agglomeration effect to promote economic growth while reducing the intensity of pollution emissions and improving the efficiency of resource and environmental utilization. This paper divides industrial agglomeration into four modes: specialization, diversification, related diversification and irrelevant diversification, and selects the panel data of 285 cities at prefecture-level and above in China from 2003 to 2017 to investigate the relationship between industrial agglomeration mode and environmental pollution. The results show that both professional agglomeration and diversified agglomeration are helpful to improve environmental pollution. In diversified agglomeration, the inherent correlation of industrial agglomeration is negatively related to environmental pollution. The relevant diversified agglomeration can significantly improve environmental pollution, while irrelevant diversified agglomeration will aggravate environmental pollution. In addition, the above conclusions have a certain degree of regional heterogeneity and are affected by urban characteristics. The improvement of the degree of professional agglomeration has a greater effect on the improvement of environmental pollution in the western region and resource-based cities, while the increase of the degree of diversified agglomeration has a greater effect on the improvement of environmental pollution in the eastern region and non-resource-based cities.