Sperm whale ‘alphabet’ discovered, thanks to machine learning

Researchers at MIT CSAIL and Project CETI believe that they have unlocked a kind of sperm whale “alphabet” with the aid of machine learning technologies. Results from the study, which were published under the title, “Contextual and Combinatorial Structure in Sperm Whale Vocalizations,” point to key breakthroughs in our understanding of cetacean communication.

The study deals with codas — a series of clicks that serve different linguistic functions. “What we have discovered is that there is previously undescribed variation in the coda structure,” CSAIL director Daniela Rus told TechCrunch. “We have discovered that coda types are not arbitrary, but rather they form a newly discovered combinatory coding system.”

While whale vocalization has been a key subject of research for decades, the teams behind this new research suggest that they’ve uncovered a level of previously unknown nuance among the chatty sea mammals. The paper notes that previous research has noted 150 different sperm whale codas.

“A subset of these have been shown to encode information about caller and clan identity,” it explains. “However, almost everything else about the sperm whale communication system, including basic questions about its structure and information-carrying capacity, remains unknown.”

The teams drew on work from Roger Payne, the pioneering marine biologist who passed away last June. Payne’s most influential work involved the songs of humpback whales. “He has really inspired us to want to use our most advanced technologies to want to have a deeper understanding of the whales,” says Rus.

The teams deployed machine learning solutions to analyze a dataset of 8,719 sperm whale codas collected by researcher Shane Gero off the coast of the small eastern Caribbean island, Dominica.

“We would get the inputs, and then we adjust our machine learning, to visualize better and to understand more,” explains Rus. “And then we would analyze the output with a biologist.”

The team’s method marked a change from older analysis, which studied individual coda. A richer picture forms when the sounds are studied in context, as exchanges between whales. Contextual details are classified using music terminology. That includes tempo, rhythm, ornamentation and rubato. From there, the team isolated what it refers to as a sperm whale phonetic alphabet.

“This phonetic alphabet makes it possible to systematically explain the observed variability in the coda structure,” says Rus. “We believe that it’s possible that this is the first instance outside of human language where a communication provides an example of the linguistic concept of duality of patterning. That refers to a set of individually meaningless elements that can be combined to form larger meaningful units, sort of like combining syllables into words.”

The meaning of those “words” take on different meanings based on various context. The paper adds:

Our results demonstrate that sperm whale vocalizations form a complex combinatorial communication system: the seemingly arbitrary inventory of coda types can be explained by combinations of rhythm, tempo, rubato, and ornamentation features. Sizable combinatorial vocalization systems are exceedingly rare in nature; however, their use by sperm whales shows that they are not uniquely human, and can arise from dramatically different physiological, ecological, and social pressures.

While the breakthrough is exciting for all involved, there’s still a lot of work to be done, first with sperm whales and then potentially broadening out to other species like humpbacks.

“We decided to go to sperm whales because we had an extensive dataset, and we have the possibility of collecting many more datasets,” says Rus. “Also, because the clicks form a kind of discrete communication system, it is much easier to analyze than a continuous communication system. But even Roger Payne’s work showed that the songs of humpback whales are not random. There are segments that get repeated and there is interesting structure there. We just haven’t gotten to do an in-depth study.”