# NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics

arXiv: https://arxiv.org/abs/2411.07186 · HTML: https://mlsanity.com/paper/2411.07186/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2024.

## Built on by (most cited first, 30 of 33)

- [Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2505.15957/) (2025)
- [Can Masked Autoencoders Also Listen to Birds?](https://mlsanity.com/paper/2504.12880/) (2025)
- [Foundation Models for Bioacoustics -- a Comparative Review](https://mlsanity.com/paper/2508.01277/) (2025)
- [AVEX: What Matters for Animal Vocalization Encoding](https://mlsanity.com/paper/2508.11845/) (2025)
- [Robust detection of overlapping bioacoustic sound events](https://mlsanity.com/paper/2503.02389/) (2025)
- [Crossing the Species Divide: Transfer Learning from Speech to Animal Sounds](https://mlsanity.com/paper/2509.04166/) (2025)
- [FinchGPT: a Transformer based language model for birdsong analysis](https://mlsanity.com/paper/2502.00344/) (2025)
- [Synthetic data enables context-aware bioacoustic sound event detection](https://mlsanity.com/paper/2503.00296/) (2025)
- [WoW-Bench: Evaluating Fine-Grained Acoustic Perception in Audio-Language Models via Marine Mammal Vocalizations](https://mlsanity.com/paper/2508.20976/) (2025)
- [Model Merging Improves Zero-Shot Generalization in Bioacoustic Foundation Models](https://mlsanity.com/paper/2511.05171/) (2025)
- [bacpipe: a Python package to make bioacoustic deep learning models accessible](https://mlsanity.com/paper/2604.11560/) (2026)
- [Dolph2Vec: Self-Supervised Representations of Dolphin Vocalizations](https://mlsanity.com/paper/2606.12503/) (2026)
- [MetaPerch: Learning from metadata for bioacoustics foundation models](https://mlsanity.com/paper/2607.14072/) (2026)
- [AnimalCLAP: Taxonomy-Aware Language-Audio Pretraining for Species Recognition and Trait Inference](https://mlsanity.com/paper/2603.22053/) (2026)
- [Multi-layer attentive probing improves transfer of audio representations for bioacoustics](https://mlsanity.com/paper/2605.10494/) (2026)
- [Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study](https://mlsanity.com/paper/2605.14031/) (2026)
- [OmniScientist: An Omni-Modal Omni-Discipline AI Scientist](https://mlsanity.com/paper/2608.13558/) (2026)
- [BioAnalyst: A Foundation Model for Biodiversity](https://mlsanity.com/paper/2507.09080/) (2025)
- [Knowledge-Augmented Vision Language Models for Underwater Bioacoustic Spectrogram Analysis](https://mlsanity.com/paper/2509.05703/) (2025)
- [An Entropy-Guided Curriculum Learning Strategy for Data-Efficient Acoustic Scene Classification under Domain Shift](https://mlsanity.com/paper/2509.11168/) (2025)
- [The Double Contingency Problem: AI Recursion and the Limits of Interspecies Understanding](https://mlsanity.com/paper/2511.08927/) (2025)
- [BioVITA: Biological Dataset, Model, and Benchmark for Visual-Textual-Acoustic Alignment](https://mlsanity.com/paper/2603.23883/) (2026)
- [BAGEL: Benchmarking Animal Knowledge Expertise in Language Models](https://mlsanity.com/paper/2604.16241/) (2026)
- [Beyond the Baseband: Adaptive Multi-Band Encoding for Full-Spectrum Bioacoustics Classification](https://mlsanity.com/paper/2604.27936/) (2026)
- [BeeVe: Unsupervised Acoustic State Discovery in Honey Bee Buzzing](https://mlsanity.com/paper/2605.07903/) (2026)
- [Language Game: Talking to Non-Human Systems](https://mlsanity.com/paper/2605.16321/) (2026)
- [CoarseSoundNet: Building a reliable model for ecological soundscape analysis](https://mlsanity.com/paper/2605.21143/) (2026)
- [Beyond task performance: Decoding bioacoustic embeddings with speech features](https://mlsanity.com/paper/2606.14662/) (2026)
- [MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training](https://mlsanity.com/paper/2609.15221/) (2026)
- [When Big Data Becomes a Curse: Spatial Heterogeneity and the Limits of Learning from Passive Acoustic Monitoring Data](https://mlsanity.com/paper/2609.26125/) (2026)

## Abstract (arXiv)

Large language models (LLMs) prompted with text and audio have achieved state-of-the-art performance across various auditory tasks, including speech, music, and general audio, showing emergent abilities on unseen tasks. However, their potential has yet to be fully demonstrated in bioacoustics tasks, such as detecting animal vocalizations in large recordings, classifying rare and endangered species, and labeling context and behavior -- tasks that are crucial for conservation, biodiversity monitoring, and animal behavior studies. In this work, we present NatureLM-audio, the first audio-language foundation model specifically designed for bioacoustics. Our training dataset consists of carefully curated text-audio pairs spanning bioacoustics, speech, and music, designed to address the field's limited availability of annotated data. We demonstrate successful transfer of learned representations from music and speech to bioacoustics, and our model shows promising generalization to unseen taxa and tasks. We evaluate NatureLM-audio on a novel benchmark (BEANS-Zero) and it sets a new state of the art on several bioacoustics tasks, including zero-shot classification of unseen species. To advance bioacoustics research, we release our model weights, benchmark data, and open-source the code for training and benchmark data generation and model training.