# PhaseCoder: Microphone Geometry-Agnostic Spatial Audio Understanding for Multimodal LLMs

arXiv: https://arxiv.org/abs/2601.21124 · HTML: https://mlsanity.com/paper/2601.21124/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 28 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/)

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages](https://mlsanity.com/paper/2303.01037/) (2023)
- [Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models](https://mlsanity.com/paper/2507.08128/) (2025)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Models](https://mlsanity.com/paper/2310.02943/) (2023)
- [BAT: Learning to Reason about Spatial Sounds with Large Language Models](https://mlsanity.com/paper/2402.01591/) (2024)
- [Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation](https://mlsanity.com/paper/2410.10676/) (2024)
- [Learning Spatially-Aware Language and Audio Embeddings](https://mlsanity.com/paper/2409.11369/) (2024)
- [Broadband DOA estimation using Convolutional neural networks trained with noise signals](https://mlsanity.com/paper/1705.00919/) (2017)
- [Can Large Language Models Understand Spatial Audio?](https://mlsanity.com/paper/2406.07914/) (2024)
- [SALM: Spatial Audio Language Model with Structured Embeddings for Understanding and Editing](https://mlsanity.com/paper/2507.16724/) (2025)
- [Localization of Simultaneous Moving Sound Sources for Mobile Robot Using a Frequency-Domain Steered Beamformer Approach](https://mlsanity.com/paper/1602.08629/) (2016)
- [Ambisonics Encoding For Arbitrary Microphone Arrays Incorporating Residual Channels For Binaural Reproduction](https://mlsanity.com/paper/2402.17362/) (2024)
- [M-BEST-RQ: A Multi-Channel Speech Foundation Model for Smart Glasses](https://mlsanity.com/paper/2409.11494/) (2024)
- [Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition](https://mlsanity.com/paper/2506.14973/) (2025)
- [SpeechCompass: Enhancing Mobile Captioning with Diarization and Directional Guidance via Multi-Microphone Localization](https://mlsanity.com/paper/2502.08848/) (2025)

## Built on by (most cited first, 6 of 6)

- [Spatial-Omni: Spatial Audio Understanding Integration in Multimodal LLMs via FOA Encoding](https://mlsanity.com/paper/2606.10738/) (2026)
- [OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agents](https://mlsanity.com/paper/2609.23407/) (2026)
- [SceneBind: Binding What and Where Across Vision, Audio and Language](https://mlsanity.com/paper/2607.15265/) (2026)
- [SAIL: Spatial Audio Intelligence with Large Language Models via Disentangled Acoustic-Spatial Encoding and Dual-Stream Q-Former](https://mlsanity.com/paper/2609.34347/) (2026)
- [SEA-LM: Egocentric Spatial Audio Understanding for Wearable Microphone Arrays](https://mlsanity.com/paper/2610.05610/) (2026)
- [FloorSAV: Elucidating Spatial Audio-Visual Context with 2D Floormap for AV-LLMs](https://mlsanity.com/paper/2610.11310/) (2026)

## Abstract (arXiv)

Current multimodal LLMs process audio as a mono stream, ignoring the rich spatial information essential for embodied AI. Existing spatial audio models, conversely, are constrained to fixed microphone geometries, preventing deployment across diverse devices. We present PhaseCoder, a transformer-only spatial audio encoder that is agnostic to microphone geometry. PhaseCoder takes raw multichannel audio and microphone coordinates as inputs to perform localization and produces robust spatial embeddings. We demonstrate that Gemma 3n LLM can be fine-tuned to reason over "Spatial Audio Tokens" produced by PhaseCoder. We show our encoder achieves state-of-the-art results on microphone-invariant localization benchmarks and, for the first time, enables an LLM to perform complex spatial reasoning and targeted transcription tasks from an arbitrary microphone array.