Little is known about the macrohabitat associations of rodents and shrews in prairie landscapes because of the logistic constraints of conventional trapping. We used the remains of 60,972 small mammals in owl pellets to assess factors affecting small mammal composition across 4.3 million hectares of the northern Great Plains of North America. Cropland with clay soils was dominated by deer mice (Peromyscus maniculatus), whereas areas with higher proportions of native grassland and moderately sandy soils supported communities with more sagebrush voles (Lemmiscus curtatus). Areas with clay soils and higher annual precipitation were associated with higher proportions of house mice (Mus musculus), meadow voles (Microtus pennsylvanicus), and shrews (Blarina brevicauda and Sorex species), whereas drier areas with sandier soils and lower annual precipitation were dominated by olive-backed pocket mice (Perognathus fasciatus) and northern grasshopper mice (Onychomys leucogaster). Contrary to extrapolations of previous smaller-scale efforts, soil texture was the primary landscape feature driving small mammal composition in our study, whereas agricultural cropland significantly altered the composition of these assemblages. These associations demonstrate the importance of considering macrohabitats encompassing entire populations.