Abstract Based on nationwide panel data of rural micro-households, this study empirically examines the impact of cultivated land fragmentation on grain production and analyzes its underlying mechanisms. The study shows that cultivated land fragmentation significantly reduces grain yield. This conclusion remains robust after addressing potential endogeneity and conducting robustness tests. Heterogeneity analysis further reveals that the inhibitory effect of land fragmentation on grain yield is more pronounced in non-major grain-producing areas, under excessively large or small operational scales, and within households characterized by a high degree of aging. The results of the mechanism analysis indicate that cultivated land fragmentation can affect farm household grain yield by increasing the degree of part-time farming and promoting a “non-grain” shift in the planting structure. Further analysis reveals a single threshold effect based on fertilizer application intensity in the impact of land fragmentation on grain yield. Therefore, to safeguard farmers’ rights and interests, promote agricultural modernization and strengthen the foundation of food security, measures should be implemented through improving the functioning of the farmland transfer market, enhancing the comparative returns of grain production, and advancing the reduction and efficiency improvement of chemical fertilizers and pesticides.