We used stereo recordings of point counts from Breeding Bird Survey (BBS) routes to estimate inter-observer variability, to compare recordings with field observers, and to evaluate multiple-observer occupancy-model approaches to adjust for incomplete detection. We asked six birders with point-count experience (listeners) to listen to recordings of 80 stops from four BBS routes and report all the species they detected at each stop. Listeners went through each recording only once, without the use of aids such as spectrograms, to simulate field conditions. Listeners differed significantly in the mean number of species reported, ranging from 7.8 to 10.6 species per stop, compared with 12.6 confirmed on the recordings. On average, 14% of the reported records were identification errors (i.e., species not present on the recording), and 23% of species present on the recordings were overlooked. Field observers reported similar numbers of species as some listeners, but fewer species than were on the recordings. Multiple-observer occupancy models failed to correctly account for identification errors and tended to overestimate species richness by 1.0–8.4 species per stop; species-specific occupancy estimates were positively biased by 69–100%. Although inter-observer variability clearly needs to be considered in analysis of point-count data, multiple-observer occupancy models do not adequately correct for this variation. The number of species detected on recordings suggests that they have considerable potential to supplement current monitoring programs, but further development of analysis methods is required to address observer effects.