mlsanityData to 14 Oct 2026

Beyond Transcription: Unified Audio Schema for Perception-Aware AudioLLMs

arXiv 2604.12506 ↗First posted 14 Apr 2026cs.CL cs.SDcited by 1

Not on the map. Why?

Abstract (arXiv)

Recent Audio Large Language Models (AudioLLMs) exhibit a striking performance inversion: while excelling at complex reasoning tasks, they consistently underperform on fine-grained acoustic perception. We attribute this gap to a fundamental limitation of ASR-centric training, which provides precise linguistic targets but implicitly teaches models to suppress paralinguistic cues and acoustic events as noise. To address this, we propose Unified Audio Schema (UAS), a holistic and structured supervision framework that organizes audio information into three explicit components -- Transcription, Paralinguistics, and Non-linguistic Events -- within a unified JSON format. This design achieves comprehensive acoustic coverage without sacrificing the tight audio-text alignment that enables reasoning. We validate the effectiveness of this supervision strategy by applying it to both discrete and continuous AudioLLM architectures. Extensive experiments on MMSU, MMAR, and MMAU demonstrate that UAS-Audio yields consistent improvements, boosting fine-grained perception by 10.9% on MMSU over the same-size state-of-the-art models while preserving robust reasoning capabilities. Our code and model are publicly available at https://github.com/Tencent/Unified_Audio_Schema.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 25 of its 42 references that are in the map. First posted on arXiv 14 Apr 2026.

Builds on

25 of 42 references.

References · 25

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  5. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  6. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  7. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  8. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  9. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  10. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  11. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  12. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
13 more
  1. 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
  2. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 220
  3. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 219
  4. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  5. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  6. 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 103
  7. 2025Step-Audio: Unified Understanding and Generation in Intelligent Speech Interactioncited by 83
  8. 2025Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answeringcited by 65
  9. 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
  10. 2024YODAS: Youtube-Oriented Dataset for Audio and Speechcited by 60
  11. 2025MiDashengLM: Efficient Audio Understanding with General Audio Captionscited by 33
  12. 2025WildSpeech-Bench: Benchmarking End-to-End SpeechLLMs in the Wildcited by 13
  13. 2025AudSemThinker: Enhancing Audio-Language Models through Reasoning over Semantics of Soundcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Eureka-Audio: Triggering Audio Intelligence in Compact Language Models
  2. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities
  3. 2026ALARM: Audio-Language Alignment for Reasoning Models
  4. 2026X3-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment
  5. 2026CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillation
  6. 2026Do Audio-Visual Large Language Models Really See and Hear?
  7. 2026Listen Then Reason: Perception-Grounded Test-Time Reinforcement Learning for Large Audio-Language Models
  8. 2026Echo: Towards Advanced Audio Comprehension via Audio-Interleaved Reasoning