SAIL: Spatial Audio Intelligence with Large Language Models via Disentangled Acoustic-Spatial Encoding and Dual-Stream Q-Former
Not on the map. Why?
Abstract (arXiv)
Spatial audio large language models (LLMs) enable embodied agents, wearable assistants, and immersive systems to recognize sound events, localize sources, and reason about their spatial relationships. However, existing spatial audio LLMs often rely on early fusion of acoustic and spatial features and source-agnostic token representations. These designs make it difficult to preserve the correspondence between individual sound events and their spatial attributes, particularly in multi-source scenes. To address this limitation, we propose SAIL, a Spatial Audio Intelligence framework with LLMs that preserves acoustic-spatial structure and source-level correspondence from audio encoding to LLM alignment. SAIL introduces a Disentangled Spatial Audio Transformer that represents Mel-spectrogram and interaural phase difference features as separate acoustic and spatial streams. Source-discriminative task queries further learn event, direction, and distance information for each source. A Dual-Stream Q-Former then aligns the two streams with the LLM using acoustic and spatial queries organized by source slots. Compared with the early-fusion baseline, SAIL achieves consistent improvements in dual-source sound event detection, direction and distance estimation, and spatial reasoning. These results demonstrate the importance of structured, source-discriminative audio representations for multi-source spatial understanding and reasoning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 43 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
38 of 43 references.
References · 38
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
- 2024Qwen2-Audio Technical Reportcited by 560
- 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
- 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
26 more
- 2022Masked Autoencoders that Listencited by 273
- 2025Kimi-Audio Technical Reportcited by 222
- 2023Listen, Think, and Understandcited by 209
- 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 191
- 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 186
- 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 156
- 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 133
- 2025Step-Audio 2 Technical Reportcited by 105
- 2018Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networkscited by 85
- 2024BAT: Learning to Reason about Spatial Sounds with Large Language Modelscited by 36
- 2024Both Ears Wide Open: Towards Language-Driven Spatial Audio Generationcited by 25
- 2021SALSA: Spatial Cue-Augmented Log-Spectrogram Features for Polyphonic Sound Event Localization and Detectioncited by 15
- 2025ALMTokenizer: A Low-bitrate and Semantic-rich Audio Codec Tokenizer for Audio Language Modelingcited by 12
- 2025STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligencecited by 11
- 2025OWL: Geometry-Aware Spatial Reasoning for Audio Large Language Modelscited by 10
- 2025SPUR: A Plug-and-Play Framework for Integrating Spatial Audio Understanding and Reasoning into Large Audio-Language Modelscited by 8
- 2026SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processingcited by 8
- 2025WearVox: An Egocentric Multichannel Voice Assistant Benchmark for Wearablescited by 7
- 2024PSELDNets: Pre-trained Neural Networks on a Large-scale Synthetic Dataset for Sound Event Localization and Detectioncited by 6
- 2026The World is Not Mono: Enabling Spatial Understanding in Large Audio-Language Modelscited by 6
- 2026PhaseCoder: Microphone Geometry-Agnostic Spatial Audio Understanding for Multimodal LLMscited by 6
- 2025DSpAST: Disentangled Representations for Spatial Audio Reasoning with Large Language Modelscited by 5
- 2025Sci-Phi: A Large Language Model Spatial Audio Descriptorcited by 5
- 2026Spatio-Temporal Audio Language Modeling for Dynamic Sound Sourcescited by 2
- 2026AudioWorldSim: Realistic Binaural Audio Datasets For World Modelscited by 2
- 2026SelectTSL: Prompt-Guided Selective Target Sound Localization in Complex Scenarioscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SEA-LM: Egocentric Spatial Audio Understanding for Wearable Microphone Arrays
- 2026Spatial-Omni: Spatial Audio Understanding Integration in Multimodal LLMs via FOA Encoding
- 2025Spatial Audio Motion Understanding and Reasoning
- 2025In-the-wild Audio Spatialization with Flexible Text-guided Localization
- 2026SonicWorld: Source-Centric Latent Dynamics for Spatial Audio Reasoning
- 2026BinauralVAE: Spatial Audio Reconstruction For World Models
- 2025AudioScene: Integrating Object-Event Audio into 3D Scenes
- 2026Spatial Audio Question Answering and Reasoning on Dynamic Source Movements