Several species of drywood termites, subterranean termites, and fungus-growing termites cause extensive economic losses annually worldwide. Because no universal method is available for controlling all termites, correct species identification is crucial for termite management. Despite deep neural network technologies' promising performance in pest recognition, a method for automatic termite recognition remains lacking. To develop an automated deep learning classifier for termite image recognition suitable for mobile applications, we used smartphones to acquire 18,000 original images each of four termite pest species: Kalotermitidae: Cryptotermes domesticus (Haviland); Rhinotermitidae: Coptotermes formosanus Shiraki and Reticulitermes flaviceps (Oshima); and Termitidae: Odontotermes formosanus (Shiraki). Each original image included multiple individuals, and we applied five image segmentation techniques for capturing individual termites. We used 24,000 individual-termite images (4 species × 2 castes × 3 groups × 1,000 images) for model development and testing. We implemented a termite classification system by using a deep learning–based model, MobileNetV2. Our models achieved high accuracy scores of 0.947, 0.946, and 0.929 for identifying soldiers, workers, and both castes, respectively, which is not significantly different from human expert performance. We further applied image augmentation techniques, including geometrical transformations and intensity transformations, to individual-termite images. The results revealed that the same classification accuracy can be achieved by using 1,000 augmented images derived from only 200 individual-termite images, thus facilitating further model development on the basis of many fewer original images. Our image-based identification system can enable the selection of termite control tools for pest management professionals or homeowners.