Small mammals are important to the functioning of ecological communities with changes to their abundances used to track impacts of environmental change. While capture–recapture estimates of absolute abundance are preferred, indices of abundance continue to be used in cases of limited sampling, rare species with little data, or unmarked individuals. Improvement to indices can be achieved by calibrating them to absolute abundance but their reliability across years, sites, or species is unclear. To evaluate this, we used the US National Ecological Observatory Network capture–recapture data for 63 small mammal species over 46 sites from 2013 to 2019. We generated 17,155 absolute abundance estimates using capture–recapture analyses and compared these to two standard abundance indices, and three types of calibrated indices. We found that neither raw abundance indices nor index calibrations were reliable approximations of absolute abundance, with raw indices less correlated with absolute abundance than index calibrations (raw indices overall R² < 0.5, index calibration overall R² > 0.6). Performance of indices and index calibrations varied by species, with those having higher and less variable capture probabilities performing best. We conclude that indices and index calibration methods should be used with caution with a count of individuals being the best index to use, especially if it can be calibrated with capture probability. None of the indices we tested should be used for comparing different species due to high variation in capture probabilities. Hierarchical models that allow for sharing of capture probabilities over species or plots (i.e., joint-likelihood models) may offer a better solution to mitigate the cost and effort of large-scale small mammal sampling while still providing robust estimates of abundance.