# Text-to-Audio Grounding: Building Correspondence Between Captions and Sound Events

arXiv: https://arxiv.org/abs/2102.11474 · HTML: https://mlsanity.com/paper/2102.11474/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2021.

## Built on by (most cited first, 16 of 16)

- [AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension](https://mlsanity.com/paper/2402.07729/) (2024)
- [Large Language Models are Few-Shot Health Learners](https://mlsanity.com/paper/2305.15525/) (2023)
- [DiffSED: Sound Event Detection with Denoising Diffusion](https://mlsanity.com/paper/2308.07293/) (2023)
- [Dissecting Temporal Understanding in Text-to-Audio Retrieval](https://mlsanity.com/paper/2409.00851/) (2024)
- [Smooth-Foley: Creating Continuous Sound for Video-to-Audio Generation Under Semantic Guidance](https://mlsanity.com/paper/2412.18157/) (2024)
- [Automated Audio Captioning and Language-Based Audio Retrieval](https://mlsanity.com/paper/2207.04156/) (2022)
- [MusTBench: Benchmarking and Advancing Temporal Grounding in Music LLMs](https://mlsanity.com/paper/2605.29300/) (2026)
- [Text-to-Audio Grounding Based Novel Metric for Evaluating Audio Caption Similarity](https://mlsanity.com/paper/2210.06354/) (2022)
- [Think Smart, Not Hard: Difficulty Adaptive Reasoning for Large Audio Language Models](https://mlsanity.com/paper/2509.21960/) (2025)
- [Towards Fine-grained Temporal Perception: Post-Training Large Audio-Language Models with Audio-Side Time Prompt](https://mlsanity.com/paper/2604.13715/) (2026)
- [Language-Based Audio Retrieval with Converging Tied Layers and Contrastive Loss](https://mlsanity.com/paper/2206.14659/) (2022)
- [MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation](https://mlsanity.com/paper/2406.10591/) (2024)
- [Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding](https://mlsanity.com/paper/2607.04383/) (2026)
- [GigaChat Audio: Time-aware Large Audio Language Model](https://mlsanity.com/paper/2607.10387/) (2026)
- [What Did the MLLM Hear? Token-Level Spectro-Temporal Grounding for Audio MLLM Explainability](https://mlsanity.com/paper/2609.12663/) (2026)
- [Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception](https://mlsanity.com/paper/2609.15215/) (2026)

## Abstract (arXiv)

Automated Audio Captioning is a cross-modal task, generating natural language descriptions to summarize the audio clips' sound events. However, grounding the actual sound events in the given audio based on its corresponding caption has not been investigated. This paper contributes an AudioGrounding dataset, which provides the correspondence between sound events and the captions provided in Audiocaps, along with the location (timestamps) of each present sound event. Based on such, we propose the text-to-audio grounding (TAG) task, which interactively considers the relationship between audio processing and language understanding. A baseline approach is provided, resulting in an event-F1 score of 28.3% and a Polyphonic Sound Detection Score (PSDS) score of 14.7%.