# MiDashengLM: Efficient Audio Understanding with General Audio Captions

arXiv: https://arxiv.org/abs/2508.03983 · HTML: https://mlsanity.com/paper/2508.03983/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2025.

## Built on by (most cited first, 30 of 33)

- [InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue](https://mlsanity.com/paper/2510.13747/) (2025)
- [SpeechJudge: Towards Human-Level Judgment for Speech Naturalness](https://mlsanity.com/paper/2511.07931/) (2025)
- [STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence](https://mlsanity.com/paper/2510.24693/) (2025)
- [SARSteer: Safeguarding Large Audio-Language Models via Safe-Ablated Refusal Steering](https://mlsanity.com/paper/2510.17633/) (2025)
- [Cross-Modal Knowledge Distillation for Speech Large Language Models](https://mlsanity.com/paper/2509.14930/) (2025)
- [AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking](https://mlsanity.com/paper/2601.17645/) (2026)
- [Logics-Parsing-Omni Technical Report](https://mlsanity.com/paper/2603.09677/) (2026)
- [Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation](https://mlsanity.com/paper/2511.16757/) (2025)
- [Echo: Towards Advanced Audio Comprehension via Audio-Interleaved Reasoning](https://mlsanity.com/paper/2602.11909/) (2026)
- [AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following](https://mlsanity.com/paper/2606.03116/) (2026)
- [CMDAR: A Chinese Multi-scene Dynamic Audio Reasoning Benchmark with Diverse Challenges](https://mlsanity.com/paper/2509.22461/) (2025)
- [SonicBench: Dissecting the Physical Perception Bottleneck in Large Audio Language Models](https://mlsanity.com/paper/2601.11039/) (2026)
- [Beyond Transcription: Unified Audio Schema for Perception-Aware AudioLLMs](https://mlsanity.com/paper/2604.12506/) (2026)
- [Audio Jailbreaks in Large Audio-Language Models: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation](https://mlsanity.com/paper/2605.30031/) (2026)
- [UAT: Unified Audio-Text Diffusion for Audio Generation, Editing, and Captioning](https://mlsanity.com/paper/2606.04939/) (2026)
- [USAD 2.0: Scaling Representation Distillation for Universal Audio Understanding](https://mlsanity.com/paper/2606.06444/) (2026)
- [ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models](https://mlsanity.com/paper/2606.30682/) (2026)
- [PALM-Bench: A Comprehensive Benchmark for Personalized Audio-Language Models](https://mlsanity.com/paper/2601.03531/) (2026)
- [MCGA: A Multi-task Classical Chinese Literary Genre Audio Corpus](https://mlsanity.com/paper/2601.09270/) (2026)
- [Hello-Chat: Towards Realistic Social Audio Interactions](https://mlsanity.com/paper/2602.23387/) (2026)
- [CAF-Score: Calibrating CLAP with LALMs for Reference-free Audio Captioning Evaluation](https://mlsanity.com/paper/2603.19615/) (2026)
- [LoSATok: Low-Dimensional Semantic-Acoustic Tokenizer for Cross-Domain Audio Understanding and Generation](https://mlsanity.com/paper/2605.27840/) (2026)
- [LambdaMark: Semantic Audio Watermarking for Robustness and Radioactivity](https://mlsanity.com/paper/2606.21365/) (2026)
- [MMAC: A Massive Multi-dimensional Benchmark for Audio Captioning](https://mlsanity.com/paper/2607.27109/) (2026)
- [Hear, Invoke, and Understand: A Skill-Calling Multimodal Agent for Large Audio Language Models](https://mlsanity.com/paper/2608.01881/) (2026)
- [MiDashengLM-Gen: Unified Audio Scene Generation via LLM-Driven Autoregressive Flow Matching](https://mlsanity.com/paper/2608.11804/) (2026)
- [Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models](https://mlsanity.com/paper/2608.18132/) (2026)
- [HEAR Who Said What: Unlocking Speaker-Attributed Reasoning via Counterfactual Voice Grounding](https://mlsanity.com/paper/2608.29120/) (2026)
- [Knowing When Not to Answer: Pseudo-Ensembles for Abstention in Music Audio-Language Models](https://mlsanity.com/paper/2609.04362/) (2026)
- [VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrieval](https://mlsanity.com/paper/2609.18521/) (2026)

## Abstract (arXiv)

Current approaches for large audio language models (LALMs) often rely on closed data sources or proprietary models, limiting their generalization and accessibility. This paper introduces MiDashengLM, a novel open audio-language model designed for efficient and comprehensive audio understanding through the use of general audio captions using our novel ACAVCaps training dataset. MiDashengLM exclusively relies on publicly available pretraining and supervised fine-tuning (SFT) datasets, ensuring full transparency and reproducibility. At its core, MiDashengLM integrates Dasheng, an open-source audio encoder, specifically engineered to process diverse auditory information effectively. Unlike previous works primarily focused on Automatic Speech Recognition (ASR) based audio-text alignment, our strategy centers on general audio captions, fusing speech, sound and music information into one textual representation, enabling a holistic textual representation of complex audio scenes. Lastly, MiDashengLM provides an up to 4x speedup in terms of time-to-first-token (TTFT) and up to 20x higher throughput than comparable models. Checkpoints are available online at https://huggingface.co/mispeech/midashenglm-7b and https://github.com/xiaomi-research/dasheng-lm.