# AC/DC: LLM-based Audio Comprehension via Dialogue Continuation

arXiv: https://arxiv.org/abs/2506.10312 · HTML: https://mlsanity.com/paper/2506.10312/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 18 of 18)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [SALMONN: Towards Generic Hearing Abilities for Large Language Models](https://mlsanity.com/paper/2310.13289/) (2023)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research](https://mlsanity.com/paper/2303.17395/) (2023)
- [Pengi: An Audio Language Model for Audio Tasks](https://mlsanity.com/paper/2305.11834/) (2023)
- [GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities](https://mlsanity.com/paper/2406.11768/) (2024)
- [AudioBench: A Universal Benchmark for Audio Large Language Models](https://mlsanity.com/paper/2406.16020/) (2024)
- [Clotho-AQA: A Crowdsourced Dataset for Audio Question Answering](https://mlsanity.com/paper/2204.09634/) (2022)
- [AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs](https://mlsanity.com/paper/2311.06753/) (2023)
- [EAT: Self-Supervised Pre-Training with Efficient Audio Transformer](https://mlsanity.com/paper/2401.03497/) (2024)
- [CED: Consistent ensemble distillation for audio tagging](https://mlsanity.com/paper/2308.11957/) (2023)
- [An Encoder-Decoder Based Audio Captioning System With Transfer and Reinforcement Learning](https://mlsanity.com/paper/2108.02752/) (2021)
- [Automated Audio Captioning: An Overview of Recent Progress and New Challenges](https://mlsanity.com/paper/2205.05949/) (2022)
- [SLAM-AAC: Enhancing Audio Captioning with Paraphrasing Augmentation and CLAP-Refine through LLMs](https://mlsanity.com/paper/2410.09503/) (2024)
- [Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding](https://mlsanity.com/paper/2406.13275/) (2024)
- [The NTT DCASE2020 Challenge Task 6 system: Automated Audio Captioning with Keywords and Sentence Length Estimation](https://mlsanity.com/paper/2007.00225/) (2020)

## Built on by (most cited first, 1 of 1)

- [How Auditory Knowledge in LLM Backbones Shapes Audio Language Models: A Holistic Evaluation](https://mlsanity.com/paper/2603.19195/) (2026)

## Abstract (arXiv)

We propose an instruction-following audio comprehension model that leverages the dialogue continuation ability of large language models (LLMs). Instead of directly generating target captions in training data, the proposed method trains a model to produce responses as if the input caption triggered a dialogue. This dialogue continuation training mitigates the caption variation problem. Learning to continue a dialogue effectively captures the caption's meaning beyond its surface-level words. As a result, our model enables zero-shot instruction-following capability without multitask instruction tuning, even trained solely on audio captioning datasets. Experiments on AudioCaps, WavCaps, and Clotho datasets with AudioBench audio-scene question-answering tests demonstrate our model's ability to follow various unseen instructions.