# Transferable Models for Bioacoustics with Human Language Supervision

arXiv: https://arxiv.org/abs/2308.04978 · HTML: https://mlsanity.com/paper/2308.04978/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 53 references that are in the map.
- First posted on arXiv 9 Aug 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research](https://mlsanity.com/paper/2303.17395/) (2023)
- [HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection](https://mlsanity.com/paper/2202.00874/) (2022)
- [Computational bioacoustics with deep learning: a review and roadmap](https://mlsanity.com/paper/2112.06725/) (2021)
- [AVES: Animal Vocalization Encoder based on Self-Supervision](https://mlsanity.com/paper/2210.14493/) (2022)
- [BEANS: The Benchmark of Animal Sounds](https://mlsanity.com/paper/2210.12300/) (2022)
- [HumBugDB: A Large-scale Acoustic Mosquito Dataset](https://mlsanity.com/paper/2110.07607/) (2021)

## Built on by (most cited first, 21 of 21)

- [animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics](https://mlsanity.com/paper/2406.01253/) (2024)
- [Foundation Models for Bioacoustics -- a Comparative Review](https://mlsanity.com/paper/2508.01277/) (2025)
- [Clustering and novel class recognition: evaluating bioacoustic deep learning feature extractors](https://mlsanity.com/paper/2504.06710/) (2025)
- [AVEX: What Matters for Animal Vocalization Encoding](https://mlsanity.com/paper/2508.11845/) (2025)
- [Crossing the Species Divide: Transfer Learning from Speech to Animal Sounds](https://mlsanity.com/paper/2509.04166/) (2025)
- [Synthetic data enables context-aware bioacoustic sound event detection](https://mlsanity.com/paper/2503.00296/) (2025)
- [Model Merging Improves Zero-Shot Generalization in Bioacoustic Foundation Models](https://mlsanity.com/paper/2511.05171/) (2025)
- [bacpipe: a Python package to make bioacoustic deep learning models accessible](https://mlsanity.com/paper/2604.11560/) (2026)
- [Dolph2Vec: Self-Supervised Representations of Dolphin Vocalizations](https://mlsanity.com/paper/2606.12503/) (2026)
- [MetaPerch: Learning from metadata for bioacoustics foundation models](https://mlsanity.com/paper/2607.14072/) (2026)
- [AnimalCLAP: Taxonomy-Aware Language-Audio Pretraining for Species Recognition and Trait Inference](https://mlsanity.com/paper/2603.22053/) (2026)
- [Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study](https://mlsanity.com/paper/2605.14031/) (2026)
- [Entity Embeddings : Perspectives Towards an Omni-Modality Era for Large Language Models](https://mlsanity.com/paper/2310.18390/) (2023)
- [An Entropy-Guided Curriculum Learning Strategy for Data-Efficient Acoustic Scene Classification under Domain Shift](https://mlsanity.com/paper/2509.11168/) (2025)
- [The Double Contingency Problem: AI Recursion and the Limits of Interspecies Understanding](https://mlsanity.com/paper/2511.08927/) (2025)
- [Compact Hypercube Embeddings for Fast Text-based Wildlife Observation Retrieval](https://mlsanity.com/paper/2601.22783/) (2026)
- [Audio-to-Image Bird Species Retrieval without Audio-Image Pairs via Text Distillation](https://mlsanity.com/paper/2602.00681/) (2026)
- [BioVITA: Biological Dataset, Model, and Benchmark for Visual-Textual-Acoustic Alignment](https://mlsanity.com/paper/2603.23883/) (2026)
- [OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations](https://mlsanity.com/paper/2609.34839/) (2026)
- [A Strength-Monotonic Law for Domain Alignment in Frozen-Embedding Bioacoustic Classification](https://mlsanity.com/paper/2610.09737/) (2026)
- [UA-Bench: Benchmarking Large Audio-Language Models on Underwater Acoustic Recognition](https://mlsanity.com/paper/2610.16942/) (2026)

## Abstract (arXiv)

Passive acoustic monitoring offers a scalable, non-invasive method for tracking global biodiversity and anthropogenic impacts on species. Although deep learning has become a vital tool for processing this data, current models are inflexible, typically cover only a handful of species, and are limited by data scarcity. In this work, we propose BioLingual, a new model for bioacoustics based on contrastive language-audio pretraining. We first aggregate bioacoustic archives into a language-audio dataset, called AnimalSpeak, with over a million audio-caption pairs holding information on species, vocalization context, and animal behavior. After training on this dataset to connect language and audio representations, our model can identify over a thousand species' calls across taxa, complete bioacoustic tasks zero-shot, and retrieve animal vocalization recordings from natural text queries. When fine-tuned, BioLingual sets a new state-of-the-art on nine tasks in the Benchmark of Animal Sounds. Given its broad taxa coverage and ability to be flexibly queried in human language, we believe this model opens new paradigms in ecological monitoring and research, including free-text search on the world's acoustic monitoring archives. We open-source our models, dataset, and code.