TORUS: A Test of Rendering-Understanding Self-Coherence for Unified Audio Models
Not on the map. Why?
Abstract (arXiv)
Unified audio models capable of audio understanding, audio generation and, increasingly, audio editing are proliferating rapidly. Yet a basic question about them remains unanswered: do the two heads of a unified model agree about the same audio? Current practice evaluates each capability in isolation on specialized benchmarks, and never asks whether a model can make sense of its own generations. We present TORUS, the first self-coherence test for audio-native unified models. TORUS comprises 48 three-stage self-coherence tests carrying 432 six-option questions spanning speech, sound and music across five task families. We holistically evaluate five open unified models alongside a Cascaded Baseline that combines state-of-the-art specialized generation, editing and understanding models. The best unified model answers 50.5% of questions against the Cascaded Baseline's 63.2% and a 16.7% chance floor. Models struggle on audio editing. Among the evaluated audio models (specialized and unified), we observe limited self-coherence, and thus position self-coherence as an essential test for future audio systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 42 references that are in the map. First posted on arXiv 30 Jul 2026.
Builds on
25 of 42 references.
References · 25
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
- 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
- 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
- 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
- 2021Efficient Training of Audio Transformers with Patchoutcited by 222
- 2025MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmarkcited by 101
- 2024Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Modelcited by 70
- 2024TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimizationcited by 67
- 2024Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversioncited by 58
- 2025MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Modelscited by 24
13 more
- 2026ACE-Step 1.5: Pushing the Boundaries of Open-Source Music Generationcited by 22
- 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Modelscited by 11
- 2025UALM: Unified Audio Language Model for Understanding, Generation and Reasoningcited by 10
- 2026Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editingcited by 10
- 2025MMEDIT: A Unified Framework for Multi-Type Audio Editing via Audio Language Modelcited by 8
- 2026AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answeringcited by 8
- 2026UniAudio 2.0: A Unified Audio Language Model with Text-Aligned Factorized Audio Tokenizationcited by 5
- 2026All That Glitters Is Not Audio: Rethinking Text Priors and Audio Reliance in Audio-Language Evaluationcited by 5
- 2026MMAE: A Massive Multitask Audio Editing Benchmarkcited by 5
- 2026Beyond Accuracy: Benchmarking Cross-Task Consistency in Unified Multimodal Modelscited by 4
- 2026SpeechEditBench: A Bilingual Multi-Attribute Benchmark for Instruction-Guided Speech Editingcited by 4
- 2024RiTTA: Modeling Event Relations in Text-to-Audio Generationcited by 3
- 2025UmniBench: Unified Understand and Generation Model Oriented Omni-dimensional Benchmarkcited by 2
Built on by
Not cited yet.