Insects that feed on plants are widely used for studies of host-plant-associated speciation due to their diversity and tractability. The value of these studies necessarily depends upon an accurate understanding of the diet of the focal insects. In North America, the 12 Phytomyza Fallén (Diptera: Agromyzidae) leafmining fly species feeding on hollies (Ilex spp. (Aquifoliaceae: Ilex)) initially appeared to be primarily a host-plant-associated radiation of largely monophagous species. However, our increased collecting efforts found that a majority of the morphospecies are oligophagous, feeding on multiple Ilex species. Patterns of host-associations suggest that colonization of new hosts followed by host-associated speciation may be an important feature of diversification. Analyses of mitochondrial COI data in three oligophagous species found evidence of host-associated genetic structure consistent with processes of host-associated divergence. However, in one of these species, evidence of geographic divergence as well as host-associated divergence was detected, illustrating complexity in factors relating to speciation and host-use evolution in this radiation of leafmining flies.