The loss of native biodiversity is a major ecological issue in human-dominated landscapes. In particular, the tree regeneration failure of deciduous forests remnants in suburban landscapes is of great concern to land managers and forestry associations. We tested the responses over two growing seasons of herbaceous plants and tree seedling survival and growth to the removal of invasive plant competition and deer herbivory. We first tested the response of understory vegetation to the removal of Microstegium vimineum (Japanese stiltgrass), a non-native invasive grass, and the removal of deer herbivory, via exclosures, in two forest fragments in central New Jersey. We then explored the restoration potential of planted seedlings of canopy tree dominants, Acer rubrum, Fraxinus americana, and Quercus rubra. The herbaceous community responded with an increase in species richness to the removal of M. vimineum. There was no response of the herbaceous community to the removal of deer herbivory, indicating that the herbaceous community will not recover rapidly from removal of these stressors alone. Survival and height growth of naturally regenerating tree seedlings increased with the removal of M. vimineum and the removal of deer herbivory, but these effects were not interactive. The survival and growth of planted seedlings of two tree species, F. americana and Q. rubra, were depressed by the presence of M. vimineum. Our results suggest that intensive management of M. vimineum and deer populations as well as active re-vegetation of herbaceous communities and tree seedlings are necessary to restore plant biodiversity in suburban deciduous forests.