# Can Masked Autoencoders Also Listen to Birds?

arXiv: https://arxiv.org/abs/2504.12880 · HTML: https://mlsanity.com/paper/2504.12880/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 74 references that are in the map.
- First posted on arXiv 17 Apr 2025.

## Builds on (most cited first, 30 of 55)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey](https://mlsanity.com/paper/2403.14608/) (2024)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Context Autoencoder for Self-Supervised Representation Learning](https://mlsanity.com/paper/2202.03026/) (2022)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [Contrastive Learning of General-Purpose Audio Representations](https://mlsanity.com/paper/2010.10915/) (2020)
- [CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders](https://mlsanity.com/paper/2311.00566/) (2023)
- [Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes](https://mlsanity.com/paper/2111.15000/) (2021)
- [What to Hide from Your Students: Attention-Guided Masked Image Modeling](https://mlsanity.com/paper/2203.12719/) (2022)

## Built on by (most cited first, 26 of 26)

- [Perch 2.0: The Bittern Lesson for Bioacoustics](https://mlsanity.com/paper/2508.04665/) (2025)
- [Foundation Models for Bioacoustics -- a Comparative Review](https://mlsanity.com/paper/2508.01277/) (2025)
- [Unmute the Patch Tokens: Rethinking Probing in Multi-Label Audio Classification](https://mlsanity.com/paper/2509.24901/) (2025)
- [AVEX: What Matters for Animal Vocalization Encoding](https://mlsanity.com/paper/2508.11845/) (2025)
- [Model Merging Improves Zero-Shot Generalization in Bioacoustic Foundation Models](https://mlsanity.com/paper/2511.05171/) (2025)
- [Linear probing enables Ship-Radiated Noise recognition with pretrained audio embeddings](https://mlsanity.com/paper/2601.08358/) (2026)
- [bacpipe: a Python package to make bioacoustic deep learning models accessible](https://mlsanity.com/paper/2604.11560/) (2026)
- [MetaPerch: Learning from metadata for bioacoustics foundation models](https://mlsanity.com/paper/2607.14072/) (2026)
- [Uncertainty Calibration of Multi-Label Bird Sound Classifiers](https://mlsanity.com/paper/2511.08261/) (2025)
- [Data-Efficient Self-Supervised Algorithms for Fine-Grained Birdsong Analysis](https://mlsanity.com/paper/2511.12158/) (2025)
- [Multi-layer attentive probing improves transfer of audio representations for bioacoustics](https://mlsanity.com/paper/2605.10494/) (2026)
- [Masked Autoencoders with Limited Data: Does It Work? A Fine-Grained Bioacoustics Case Study](https://mlsanity.com/paper/2605.14031/) (2026)
- [BioDCASE: Active Learning for Bioacoustics](https://mlsanity.com/paper/2609.15255/) (2026)
- [Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classification](https://mlsanity.com/paper/2507.13727/) (2025)
- [Better audio representations are more brain-like: linking model-brain alignment with performance in downstream auditory tasks](https://mlsanity.com/paper/2511.16849/) (2025)
- [Channel-Aware Probing for Multi-Channel Imaging](https://mlsanity.com/paper/2602.12696/) (2026)
- [BAT: Better Audio Transformer Guided by Convex Gated Probing](https://mlsanity.com/paper/2602.16305/) (2026)
- [Meow-Omni 1: A Multimodal Large Language Model for Feline Ethology](https://mlsanity.com/paper/2605.09152/) (2026)
- [Beyond task performance: Decoding bioacoustic embeddings with speech features](https://mlsanity.com/paper/2606.14662/) (2026)
- [Adaptive Loss Balancing for Multi-Task Bioacoustic Classification of Bird Species and Call Types](https://mlsanity.com/paper/2607.03304/) (2026)
- [A Self-Supervised Approach for Minimal-Annotation Hydroacoustic Data Exploration](https://mlsanity.com/paper/2607.07733/) (2026)
- [Can Tokens Compete? Token Representations against Supervised CNN Backbones for BirdCLEF+ 2026](https://mlsanity.com/paper/2607.14474/) (2026)
- [Efficient Passive Acoustic Monitoring of Killer Whales Using a Two-Stage Detection and Ecotype Classification Cascade](https://mlsanity.com/paper/2609.01792/) (2026)
- [MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training](https://mlsanity.com/paper/2609.15221/) (2026)
- [Active Learning for Biodiversity Monitoring: From Label Efficiency to Reliable Ecological Inference](https://mlsanity.com/paper/2609.27409/) (2026)
- [A Strength-Monotonic Law for Domain Alignment in Frozen-Embedding Bioacoustic Classification](https://mlsanity.com/paper/2610.09737/) (2026)

## Abstract (arXiv)

Masked Autoencoders (MAEs) learn rich semantic representations in audio classification through an efficient self-supervised reconstruction task. However, general-purpose models fail to generalize well when applied directly to fine-grained audio domains. Specifically, bird-sound classification requires distinguishing subtle inter-species differences and managing high intra-species acoustic variability, revealing the performance limitations of general-domain Audio-MAEs. This work demonstrates that bridging this domain gap domain gap requires full-pipeline adaptation, not just domain-specific pretraining data. We systematically revisit and adapt the pretraining recipe, fine-tuning methods, and frozen feature utilization to bird sounds using BirdSet, a large-scale bioacoustic dataset comparable to AudioSet. Our resulting Bird-MAE achieves new state-of-the-art results in BirdSet's multi-label classification benchmark. Additionally, we introduce the parameter-efficient prototypical probing, enhancing the utility of frozen MAE representations and closely approaching fine-tuning performance in low-resource settings. Bird-MAE's prototypical probes outperform linear probing by up to 37 percentage points in mean average precision and narrow the gap to fine-tuning across BirdSet downstream tasks. Bird-MAE also demonstrates robust few-shot capabilities with prototypical probing in our newly established few-shot benchmark on BirdSet, highlighting the potential of tailored self-supervised learning pipelines for fine-grained audio domains.