Colobine monkeys generally spend less time each day engaged in social interactions than other primates. However, a notable feature of their social interaction involves females exchanging infants (i.e., infant handling). Here, we report on the handling of an infant in relation to pelage color change in a group of black-and-white colobus (Colobus guereza) housed in the Japan Monkey Centre. We found large variation in the speed of infant pelage color change among body parts (range 100–216 days) and documented that infant handling by non-mothers was related to the infant pelage color and/or moving ability (infant independence). Non-mothers, especially females, were clearly attracted to the newborn infant and often handled the infant in a few weeks of its birth, but handling decreased with infant age and corresponded both with the loss of the natal coat color and the infant acquiring effective locomotory independence. We document interesting patterns, but the conclusions we can draw are limited and future research should consider separating effects of infant pelage color and infant locomotory independence and evaluate the effect of infant body mass.