# Audio Caption: Listen and Tell

arXiv: https://arxiv.org/abs/1902.09254 · HTML: https://mlsanity.com/paper/1902.09254/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2019.

## Built on by (most cited first, 20 of 20)

- [WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research](https://mlsanity.com/paper/2303.17395/) (2023)
- [Diffsound: Discrete Diffusion Model for Text-to-sound Generation](https://mlsanity.com/paper/2207.09983/) (2022)
- [VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset](https://mlsanity.com/paper/2305.18500/) (2023)
- [Audio Captioning Transformer](https://mlsanity.com/paper/2107.09817/) (2021)
- [Can Audio Captions Be Evaluated with Image Caption Metrics?](https://mlsanity.com/paper/2110.04684/) (2021)
- [Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation](https://mlsanity.com/paper/2410.10676/) (2024)
- [MusCaps: Generating Captions for Music Audio](https://mlsanity.com/paper/2104.11984/) (2021)
- [Automated Audio Captioning: An Overview of Recent Progress and New Challenges](https://mlsanity.com/paper/2205.05949/) (2022)
- [Zero-shot audio captioning with audio-language model guidance and audio context keywords](https://mlsanity.com/paper/2311.08396/) (2023)
- [Typing to Listen at the Cocktail Party: Text-Guided Target Speaker Extraction](https://mlsanity.com/paper/2310.07284/) (2023)
- [Bridging Language Gaps in Audio-Text Retrieval](https://mlsanity.com/paper/2406.07012/) (2024)
- [AVCap: Leveraging Audio-Visual Features as Text Tokens for Captioning](https://mlsanity.com/paper/2407.07801/) (2024)
- [MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks](https://mlsanity.com/paper/2507.23511/) (2025)
- [Evaluating Off-the-Shelf Machine Listening and Natural Language Models for Automated Audio Captioning](https://mlsanity.com/paper/2110.07410/) (2021)
- [Caption Feature Space Regularization for Audio Captioning](https://mlsanity.com/paper/2204.08409/) (2022)
- [Audio Captioning with Composition of Acoustic and Semantic Information](https://mlsanity.com/paper/2105.06355/) (2021)
- [Text-to-Audio Grounding Based Novel Metric for Evaluating Audio Caption Similarity](https://mlsanity.com/paper/2210.06354/) (2022)
- [Improving Audio Caption Fluency with Automatic Error Correction](https://mlsanity.com/paper/2306.10090/) (2023)
- [Audio Difference Learning for Audio Captioning](https://mlsanity.com/paper/2309.08141/) (2023)
- [Classifier-Guided Captioning Across Modalities](https://mlsanity.com/paper/2501.03183/) (2025)

## Abstract (arXiv)

Increasing amount of research has shed light on machine perception of audio events, most of which concerns detection and classification tasks. However, human-like perception of audio scenes involves not only detecting and classifying audio sounds, but also summarizing the relationship between different audio events. Comparable research such as image caption has been conducted, yet the audio field is still quite barren. This paper introduces a manually-annotated dataset for audio caption. The purpose is to automatically generate natural sentences for audio scene description and to bridge the gap between machine perception of audio and image. The whole dataset is labelled in Mandarin and we also include translated English annotations. A baseline encoder-decoder model is provided for both English and Mandarin. Similar BLEU scores are derived for both languages: our model can generate understandable and data-related captions based on the dataset.