WhAM: Towards A Translative Model of Sperm Whale Vocalization
Not on the map. Why?
Abstract (arXiv)
Sperm whales communicate in short sequences of clicks known as codas. We present WhAM (Whale Acoustics Model), the first transformer-based model capable of generating synthetic sperm whale codas from any audio prompt. WhAM is built by finetuning VampNet, a masked acoustic token model pretrained on musical audio, using 10k coda recordings collected over the past two decades. Through iterative masked token prediction, WhAM generates high-fidelity synthetic codas that preserve key acoustic features of the source recordings. We evaluate WhAM's synthetic codas using Fr'echet Audio Distance and through perceptual studies with expert marine biologists. On downstream classification tasks including rhythm, social unit, and vowel classification, WhAM's learned representations achieve strong performance, despite being trained for generation rather than classification. Our code is available at https://github.com/Project-CETI/wham
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 92 references that are in the map. First posted on arXiv 1 Dec 2025.
Builds on
26 of 92 references.
References · 26
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
- 2019Cross-lingual Language Model Pretrainingcited by 1,088
- 2022MaskGIT: Masked Generative Image Transformercited by 1,082
- 2021Transformers in Vision: A Surveycited by 1,009
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
- 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
- 2022High Fidelity Neural Audio Compressioncited by 673
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
14 more
- 2020Jukebox: A Generative Model for Musiccited by 535
- 2023MusicLM: Generating Music From Textcited by 442
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
- 2021Generative Spoken Language Modeling from Raw Audiocited by 336
- 2022HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detectioncited by 174
- 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
- 2023Adapting Frechet Audio Distance for Generative Music Evaluationcited by 92
- 2023VampNet: Music Generation via Masked Acoustic Token Modelingcited by 57
- 2022AVES: Animal Vocalization Encoder based on Self-Supervisioncited by 53
- 2024BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacousticscited by 36
- 2022A Theory of Unsupervised Translation Motivated by Understanding Animal Communicationcited by 6
- 2023Approaching an unknown communication system by latent space exploration and causal inferencecited by 4
- 2022An Initial study on Birdsong Re-synthesis Using Neural Vocoderscited by 1
- 2023Evidence of social learning across symbolic cultural barriers in sperm whalescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.