ACAVCaps: Enabling large-scale training for fine-grained and diverse audio understanding
Not on the map. Why?
Abstract (arXiv)
General audio understanding is a fundamental goal for large audio-language models, with audio captioning serving as a cornerstone task for their development. However, progress in this domain is hindered by existing datasets, which lack the scale and descriptive granularity required to train truly versatile models. To address this gap, we introduce ACAVCaps, a new large-scale, fine-grained, and multi-faceted audio captioning dataset. Derived from the ACAV100M collection, ACAVCaps is constructed using a multi-expert pipeline that analyzes audio from diverse perspectives-including speech, music, and acoustic properties-which are then synthesized into rich, detailed descriptions by a large language model. Experimental results demonstrate that models pre-trained on ACAVCaps exhibit substantially stronger generalization capabilities on various downstream tasks compared to those trained on other leading captioning datasets. The dataset is available at https://github.com/xiaomi-research/acavcaps.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Mar 2026.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 2.
2 papers.