Soil management practices have the potential to modify the diversity and function of microbes in agricultural fields. The aim of this study was to investigate bacterial and fungal diversity in a 15 yr wheat–pea rotation tillage experiment. The treatments included conventional tillage with stubble removed, no-till with stubble removed, no-till with stubble retained (NTS), and conventional tillage with stubble incorporated. Illumina high-throughput sequencing platform was employed to sequence bacteria 16S rRNA (V3V4) and fungi internal transcribed spacer (ITS2) region genes in 0–10 cm and 10–30 cm of soil sampled. The dominant bacterial and fungal phyla identified at 97% similarity cutoff across both depths of treatments were Proteobacteria (26.3%), Actinobacteria (25.1%), Acidobacteria (15.0%), Gemmatimonadetes (8.8%), Ascomycota (85.8%), and Basidiomycota (8.0%). NTS had significantly (p < 0.05) higher microbial diversity indices, total organic carbon, soil microbial biomass carbon and nitrogen, NO₃-N, and NH₄-N at 0–10 cm depth. Tillage and stubble effects had a significant correlation with some phyla such as Proteobacteria, Actinobacteria, Gemmatimonadetes, JL-ETNP-Z39, Nitrospirae, Chloroflexi, Firmicutes, and other identified and unidentified minor microbial phyla. No-till and residue retention practices influenced fungal and bacterial species diversity through improved soil chemical properties, which have potential to affect the habitat and activity of soil microbes. Therefore, no-till and stubble retention could improve soil quality and promote sustainable agriculture in the rainfed Loess Plateau.