Cytoplasmic male sterile (CMS) lines are important tools for hybrid production but they cannot produce viable pollen. Breeding new CMS lines and studying their sterility mechanism in wheat (Triticum aestivum L.) greatly facilitates the process of hybrid wheat breeding. We conducted transcriptome sequencing for a recently identified Mu-CMS line with Aegilops uniaristata Vis. cytoplasm, named U706A, and its isonuclear maintainer line (706B) at the binucleate stage, which was a critical period when abortion occurred. We found that most of the genes involved in phosphatidylinositol metabolism and pectin degradation were downregulated, as well as genes encoding the MYB21 and MYC2 transcription factors, in U706A compared with 706B. In addition, pectin contents indicated that the production of pectin has been enhanced from the binucleate stage to the trinucleate stage, owing to the downregulation of pectin-degradation-related genes in U706A at the binucleate stage, which confirmed the reliability of the sequencing results. We also discovered that the accumulation period of pectin content in U706A is abnormal compared with 706B, which may be an important reason for abortion. Some differentially expressed genes that might be related to the sterile phenotype were verified by quantitative RT-PCR. Therefore, we suggest that the downregulation of these genes possibly leads to the anther not to crack; the tapetum and microspore membrane system is less metabolised, and the abnormal pectin accumulation results in microspore nutrient deficiencies and abnormal development. These findings provide novel insights into the mechanism responsible for pollen abortion in CMS, which may facilitate hybrid wheat breeding.