That chromosomal rearrangements may play an important role in maintaining postzygotic isolation between well-established species is part of the standard theory of speciation. However, little evidence exists on the role of karyotypic change in speciation itself—in the establishment of reproductive barriers between previously interbreeding populations. The large genus Agrodiaetus (Lepidoptera: Lycaenidae) provides a model system to study this question. Agrodiaetus butterflies exhibit unusual interspecific diversity in chromosome number, from n = 10 to n = 134; in contrast, the majority of lycaenid butterflies have n = 23/24. We analyzed the evolution of karyotypic diversity by mapping chromosome numbers on a thoroughly sampled mitochondrial phylogeny of the genus. Karyotypic differences accumulate gradually between allopatric sister taxa, but more rapidly between sympatric sister taxa. Overall, sympatric sister taxa have a higher average karyotypic diversity than allopatric sister taxa. Differential fusion of diverged populations may account for this pattern because the degree of karyotypic difference acquired between allopatric populations may determine whether they will persist as nascent biological species in secondary sympatry. This study therefore finds evidence of a direct role for chromosomal rearrangements in the final stages of animal speciation. Rapid karyotypic diversification is likely to have contributed to the explosive speciation rate observed in Agrodiaetus, 1.6 species per million years.