Soil microorganisms play an important role in agricultural ecosystem. However, there is little information about the effects of putative allelochemicals on specific soil microorganisms in vivo. Cucumber seedlings were treated with four concentrations of ferulic and p-hydroxybenzoic acids (0–1.0 μmol·g⁻¹ soil) in soil. Effects of ferulic and p-hydroxybenzoic acids on rhizosphere Fusarium community structures and abundance were analyzed by polymerase chain reaction denaturing gradient gel electrophoresis (PCR-DGGE) and real-time PCR techniques, respectively. The results showed that ferulic acid at concentrations of 0.25, 0.5, and 1.0 μmol·g⁻¹ soil significantly reduced the number of bands of Fusarium, and ferulic acid at concentrations of 0.5 and 1.0 μmol·g⁻¹ soil significantly reduced Shannon–Wiener and evenness index of Fusarium community. All concentrations of p-hydroxybenzoic acid changed the community structure of Fusarium, including decreasing the number of bands, Shannon–Wiener, and evenness index. Ferulic and p-hydroxybenzoic acids at all concentrations significantly promoted the abundance of Fusarium. Ferulic and p-hydroxybenzoic acids at 0.5 μmol·g⁻¹ soil had the highest Fusarium abundance among all treatments. These results indicate that four concentrations of ferulic acid had various effects on Fusarium, which was different from p-hydroxybenzoic acid, and this may be related to the cucumber autotoxicity, giving us a further understanding of soil sickness.