# FineLAP: Taming Heterogeneous Supervision for Fine-grained Language-Audio Pretraining

arXiv: https://arxiv.org/abs/2604.01155 · HTML: https://mlsanity.com/paper/2604.01155/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2026.

## Built on by (most cited first, 3 of 3)

- [Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding](https://mlsanity.com/paper/2607.04383/) (2026)
- [Local Multimodal Music Alignment from Global Supervision](https://mlsanity.com/paper/2607.10023/) (2026)
- [Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception](https://mlsanity.com/paper/2609.15215/) (2026)

## Abstract (arXiv)

Contrastively pretrained audio-language models (e.g., CLAP) excel at clip-level understanding but struggle with frame-level tasks. Existing extensions fail to exploit the varying granularity of real-world audio-text data, where massive clip-level textual descriptions coexist with limited frame-level annotations. This paper proposes Fine-grained Language-Audio Pretraining (FineLAP), a novel training paradigm that advances both clip- and frame-level alignment in CLAP with heterogeneous data. FineLAP introduces a dual-stream sigmoid loss with a cluster-based sampling strategy to jointly learn from clip- and frame-level supervision. To capture both global semantics and local details, FineLAP uses a decoupled audio projector on top of a self-supervised encoder. To alleviate the scarcity of temporally annotated data, we present FineLAP-100k, a large-scale synthetic SED dataset constructed through a scalable curation pipeline. Extensive experiments demonstrate that FineLAP achieves SOTA performance across multiple audio understanding tasks, including retrieval, classification, sound event detection, and text-to-audio grounding. Ablation studies further show that coarse- and fine-grained alignment are mutually beneficial, providing insights for building better audio-language models (ALMs).