Dual-frequency Identification Sonar (DIDSON) is increasingly used as a fish monitoring and enumeration tool, but many studies do not evaluate potential observer biases. In this project, we assessed inter-observer differences in the identification and enumeration of adult Pacific lamprey (Entosphenus tridentatus) imaged with DIDSON passing a large dam fishway. Six trained viewers independently identified lamprey observation ‘events’ in the same ∼ 12 h of DIDSON data collected at several fishway locations using two sonar orientations. Among-viewer variability in lamprey enumeration was high and viewer agreement on individual lamprey was often low. A total of 274 unique potential Pacific lamprey events was identified, but individual viewers scored only 89–173 events each (mean = 131, CV = 24%) or 32–63% of the total. Viewer identification rates (events/h) varied several-fold at some sites which we attributed primarily to event non-detection rather than species misclassification. Identification differences were related to viewer confidence, image duration, total fish density, and sonar orientation. Among-viewer agreement was highest in standard-orientation deployments lateral to the swimming lamprey, where images appeared as though acquired from overhead. Imagery in standard orientation had longer event duration and enhanced detection of anguilliform swimming, the most important characteristic for distinguishing Pacific lamprey from other species. Lamprey events observed in rolled orientation (sonar rotated 90°) tended to be short duration with foreshortened head- or tail-first images that reduced viewer confidence. Our results highlight the importance of quality control assessments in acoustic imaging studies, especially those targeting cryptic species and those conducted in hydraulically challenging, multi-species environments.