AI opens new era in cognitive studies of wild primates

“This project builds on the legacy of Frans de Waal,” says Federico Sánchez Vargas, first author of the paper and an Emory PhD student in anthropology. De Waal pioneered studies of animal cognition as director of Emory’s Living Links Center for the Advanced Study of Ape and Human Evolution, while also writing best-selling books that helped popularize the field. He passed away in 2024. In addition to lab-based behavioral experiments, de Waal “gave us intimate, beautiful portraits of the lives of primates, treating them as individuals,” Sánchez Vargas says. “Our AI method allows us to more deeply understand individuals that we already have data on through field observation. We can now automate cognitive testing of them and quantify the findings. It’s a way of getting into the minds behind the personalities. Studying individuals in their natural environments, where there are tons of variations in their life experiences, lets us learn how environmental influences shaped them.” Emory primatologist Marcela Benítez, center, in the Taboga Forest Reserve of Costa Rica with two of her graduate students: Nicole Furgala (left) and Federico Sánchez Vargas. Emory primatologist Marcela Benítez, center, in the Taboga Forest Reserve of Costa Rica with two of her graduate students: Nicole Furgala (left) and Federico Sánchez Vargas. Co-authors of the paper include Jacob Abernethy, Georgia Tech associate professor of computer science; and Sai Rakshith Potluri, a former Georgia Tech graduate research assistant who is now a software engineer at ExtraHop in Seattle. The open-source paper includes a guide to the computer coding developed for the system, along with a blueprint to build a low-tech, low-cost field-research platform and to integrate all the components into a closed-loop pipeline. The authors hope other scientists will adapt their AI method to generate cognitive data spanning different species of wild primates, living in a range of environments. Capuchin monkeys are known for their intelligence, complex social structures and curiosity, making them ideal models for investigating the evolution of cognition. Capuchin monkeys are known for their intelligence, complex social structures and curiosity, making them ideal models for investigating the evolution of cognition. Benítez’ work lies at the intersection of anthropology, psychology and evolutionary biology. She studies cooperation and other social behaviors in monkeys, including a captive population of tufted capuchins in a laboratory and wild, white-faced capuchins in the Taboga Forest Reserve of northeastern Costa Rica. She is a co-director of Capuchinos de Taboga, a research project launched in 2017 in collaboration with the Universidad Nacional Técnica of Costa Rica. Experiments with animals in labs can be tightly controlled. The results, however, may be skewed since the animal is not interacting within its natural environment. Animal behavior experiments in the wild provide valid social and ecological contexts but they are challenging to design and to control. “I’m trying to bridge that gap,” Benítez says. She decided to investigate the potential of AI to achieve this aim. To help train a facial-recognition model, undergraduate students did the painstaking work of putting bounding boxes over faces of capuchins in field video and ensuring they were tagged with their proper identities. These individuals are named Pele, Pedro, Pio and Pesto. To help train a facial-recognition model, undergraduate students did the painstaking work of putting bounding boxes over faces of capuchins in field video and ensuring they were tagged with their proper identities. These individuals are named Pele, Pedro, Pio and Pesto. A seed grant from Emory’s AI.Humanity program launched a collaboration between Benítez and Abernethy to design an AI model for facial recognition of wild capuchins. Abernethy and Potluri used an open-source software known as YOLO (You Only Look Once) to develop a model to run on a laptop. The researchers trained the model on high-quality GoPro imagery of six wild capuchins interacting with testing platforms in Taboga. Emory and Georgia Tech undergraduate students performed the labor-intensive task of digitally placing “bounding boxes” to frame the faces of the monkeys in thousands of still images and videos tagged with their identities. The result was a facial-recognition system that could identify these six capuchins with 97% accuracy from static images, video and live footage in the field. Marcela Benítez and Jacob Abernethy in the field in Costa Rica to test the initial version of the facial-recognition software. Marcela Benítez and Jacob Abernethy in the field in Costa Rica to test the initial version of the facial-recognition software. Sánchez Vargas, who joined Emory as a graduate student in 2023, took on the next phase of the AI project: figuring out how to integrate the facial-recognition model into a field-friendly, scalable computer interface that could present tasks to interacting capuchins and dispense food rewards. “The model was great at identifying six monkeys, but there are 100 capuchins at the Costa Rica field site,” he says. “And we didn’t have high quality video of all of these individuals, which is needed to train the model.” Sánchez Vargas’ undergraduate degrees are in evolutionary biology and psychology. He is not an expert computer coder, but he dove into the challenge, using Python programming language to simplify and change the parameters of the original facial-recognition model. “I essentially dumbed it down,” he says, so that instead of classifying individual capuchin faces, the system recognized any capuchin monkey — and only capuchins. The idea, he explains, was to enable the system to trigger a webcam to record video whenever a capuchin approached a computer touchscreen, rapidly generating a larger, more up-to-date dataset of faces from the interacting monkeys. The resulting videos could then be used to keep training the facial-recognition model, expanding its face-recognition repertoire. The project "was a lot of work but also a lot of fun," says Sánchez Vargas, shown enjoying a sunset in the Taboga Forest Reserve. The project "was a lot of work but also a lot of fun," says Sánchez Vargas, shown enjoying a sunset in the Taboga Forest Reserve. A second Python script Sánchez Vargas developed uses a program called “pygame” to facilitate interactive stimuli for use in games or cognitive testing. The researchers created a simple stimulus to habituate the monkeys to the system: a blue square covering the computer touchscreen that records when a capuchin touches it. At the monkey’s touch, the script signals a motor circuit to dispense a food reward. The two Python scripts are integrated to run simultaneously on a Raspberry Pi, a tiny computer about half the size of an iPhone. The entire system can run for eight hours on a lightweight battery pack before it needs recharging. “It was important to us that our technology be both low cost and ecologically friendly,” Sánchez Vargas says. DIY ingenuity: Sánchez Vargas cuts pine boards to fit the design for a box to hold all of the components of CapuchinAI. DIY ingenuity: Sánchez Vargas cuts pine boards to fit the design for a box to hold all of the components of CapuchinAI. The next challenge was to create a wildlife-proof, weather-proof platform to house all the components of the system. “We built it in my garage,” Benítez says. She and Sánchez Vargas bought planks of pine, deck sealant, rubber insulating strips and plastic piping from Home Depot. “A person helping us asked, ‘What is the project you’re working on?’” Benítez says. “We didn’t go into it; it would have been a bit complicated to explain.” Emory TechLab helped Sánchez Vargas create a 3D printed, plastic food dispenser. The two researchers put together a wooden box, only about 20 inches tall, to house the system’s components: a webcam, a computer touchscreen, the Raspberry Pi, the food dispenser and a rotary motor to power the dispenser. “It was a lot of work but also a lot of fun,” Sánchez Vargas says. “One of the best parts of working in animal cognition is getting creative, trying to put yourself into the mind of the animal you’re studying, so you can develop a good way to engage them in experiments.” Finally, it was time to test their creation in the field. Sánchez Vargas and Benítez worked until 11 p.m. the night before flying to Costa Rica for field testing, making last-minute tweaks to the box they designed and built. Sánchez Vargas and Benítez worked until 11 p.m. the night before flying to Costa Rica for field testing, making last-minute tweaks to the box they designed and built. Upon arriving at the Capuchinos de Toboga facility in Costa Rica, Sánchez Vargas remembers feeling a few last-minute jitters. Would the prototype function as envisioned? “We had to get up super early, at four in the morning,” he says, “and drive a long way down this bumpy road to get to the research site.” The team secured the CapuchinAI box to a platform, loaded its food dispenser with dried slices of forest banana, and waited nearby. “For the first couple of days that we did this, no capuchins visited,” Sánchez Vargas says. “I was beginning to worry.” The third day, however, a large male capuchin named Trompudo, which means “big snout,” could not resist the scent of banana. “He climbs on the box and starts slapping the back of it,” Sánchez Vargas recalls. “Finally, he slaps the touchscreen and a banana slice pops out.” Trompudo gobbled up the food then put his hand on the screen again. Another banana slice popped out. “It was almost like you could see him realizing, ‘Ah, that’s what you have to do, touch the screen!’” Sánchez Vargas says. “It was amazing to watch an individual learn something so quickly.”