# SIREN: Spatially-Informed Reconstruction of Binaural Audio with Vision

arXiv: https://arxiv.org/abs/2603.29820 · HTML: https://mlsanity.com/paper/2603.29820/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2026.

## Built on by (most cited first, 1 of 1)

- [Binaural Audio-Visual Instance Segmentation](https://mlsanity.com/paper/2609.32180/) (2026)

## Abstract (arXiv)

Binaural audio delivers spatial cues essential for immersion, yet most consumer videos are monaural due to capture constraints. We introduce SIREN, a visually guided mono to binaural framework that explicitly predicts left and right channels. A ViT-based encoder learns dual-head self-attention to produce a shared scene map and end-to-end L/R attention, replacing hand-crafted masks. A soft, annealed spatial prior gently biases early L/R grounding, and a two-stage, confidence-weighted waveform-domain fusion (guided by mono reconstruction and interaural phase consistency) suppresses crosstalk when aggregating multi-crop and overlapping windows. Evaluated on FAIR-Play and MUSIC-Stereo, SIREN yields consistent gains on time-frequency and phase-sensitive metrics with competitive SNR. The design is modular and generic, requires no task-specific annotations, and integrates with standard audio-visual pipelines.