# OpenBEATs: A Fully Open-Source General-Purpose Audio Encoder

arXiv: https://arxiv.org/abs/2507.14129 · HTML: https://mlsanity.com/paper/2507.14129/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2025.

## Built on by (most cited first, 4 of 4)

- [Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation](https://mlsanity.com/paper/2511.16757/) (2025)
- [FISHER: A Foundation Model for Multi-Modal Industrial Signal Comprehensive Representation](https://mlsanity.com/paper/2507.16696/) (2025)
- [From Physics to Representation: Audio Learning with Synthetic Pre-training via Procedural Generation](https://mlsanity.com/paper/2606.14791/) (2026)
- [Structure Across Voices: Comparing acoustic-event type accumulation and sequence dependence across four vocal repertoires using frozen audio encoders](https://mlsanity.com/paper/2609.16612/) (2026)

## Abstract (arXiv)

Masked token prediction has emerged as a powerful pre-training objective across language, vision, and speech, offering the potential to unify these diverse modalities through a single pre-training task. However, its application for general audio understanding remains underexplored, with BEATs being the only notable example. BEATs has seen limited modifications due to the absence of open-source pre-training code. Furthermore, BEATs was trained only on AudioSet, restricting its broader downstream applicability. To address these gaps, we present OpenBEATs, an open-source framework that extends BEATs via multi-domain audio pre-training. We conduct comprehensive evaluations across six types of tasks, twenty five datasets, and three audio domains, including audio reasoning tasks such as audio question answering, entailment, and captioning. OpenBEATs achieves state-of-the-art performance on six bioacoustics datasets, two environmental sound datasets and five reasoning datasets, performing better than models exceeding a billion parameters at one-fourth their parameter size. These results demonstrate the effectiveness of multi-domain datasets and masked token prediction task to learn general-purpose audio representations. To promote further research and reproducibility, we release all pre-training and evaluation code, pretrained and fine-tuned checkpoints, and training logs at https://github.com/Shikhar-S/OpenBEATs