mlsanityData to 14 Oct 2026

Hear the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation

arXiv 2609.38748 ↗First posted 30 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent joint video-audio generation models have achieved strong semantic correspondence and temporal synchronization. However, applications such as AR/VR and interactive gaming further require stereo audio to provide an immersive sense, which remains largely overlooked. Effective stereo audio requires the perceived sound location to evolve consistently with the motion of its corresponding visual source. We refer to this property as Dynamic Spatial Correspondence and propose StereoBind, a framework that binds visual source motion to stereo sound generation. StereoBind uses motion tracks to coordinate visual motion and stereo audio through three complementary mechanisms. Visual Motion Binding establishes source-aware audiovisual correspondence, the Spatial Track Encoder captures absolute source positions, and Residual Track RoPE models relative motion. For supervision and evaluation, we construct StereoWorld-29K, a large-scale stereo audio-video dataset with paired motion tracks, and StereoWorldBench for measuring audiovisual spatial consistency. Experiments show that StereoBind substantially improves spatial alignment in stereo audio generation over existing models while preserving overall audiovisual quality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 43 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

41 of 43 references.

References · 41

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  5. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  6. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  7. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 648
  8. 2025Qwen3-Omni Technical Reportcited by 497
  9. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
  10. 2022MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generationcited by 227
  11. 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 160
  12. 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 137
29 more
  1. 2024Synchformer: Efficient Synchronization from Sparse Cuescited by 97
  2. 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 88
  3. 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 83
  4. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  5. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 58
  6. 2018gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Accelerationcited by 46
  7. 2026MOVA: Towards Scalable and Synchronized Video-Audio Generationcited by 43
  8. 2024AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video Generationcited by 41
  9. 2025UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactionscited by 38
  10. 2024BAT: Learning to Reason about Spatial Sounds with Large Language Modelscited by 36
  11. 2024Both Ears Wide Open: Towards Language-Driven Spatial Audio Generationcited by 25
  12. 2025Harmony: Harmonizing Audio and Video Generation through Cross-Task Synergycited by 21
  13. 2026AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generationcited by 21
  14. 2026DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generationcited by 20
  15. 2026Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillationcited by 13
  16. 2024Hearing Anything Anywherecited by 12
  17. 2025ViSAGe: Video-to-Spatial Audio Generationcited by 11
  18. 2024SEE-2-SOUND: Zero-Shot Spatial Environment-to-Spatial Soundcited by 10
  19. 2024SAVGBench: Benchmarking Spatially Aligned Audio-Video Generationcited by 10
  20. 2025OWL: Geometry-Aware Spatial Reasoning for Audio Large Language Modelscited by 10
  21. 2025PrismAudio: Decomposed Chain-of-Thoughts and Multi-dimensional Rewards for Video-to-Audio Generationcited by 9
  22. 2025Syncphony: Synchronized Audio-to-Video Generation with Diffusion Transformerscited by 7
  23. 2025FoleyBench: A Benchmark For Video-to-Audio Modelscited by 7
  24. 2025ISDrama: Immersive Spatial Drama Generation through Multimodal Promptingcited by 6
  25. 2025Sonic4D: Spatial Audio Generation for Immersive 4D Scene Explorationcited by 6
  26. 2025ASAudio: A Survey of Advanced Spatial Audio Researchcited by 5
  27. 2026AVControl: Efficient Framework for Training Audio-Visual Controlscited by 5
  28. 2026SonoWorld: From One Image to a 3D Audio-Visual Scenecited by 3
  29. 2026FoleyDesigner: Immersive Stereo Foley Generation with Precise Spatio-Temporal Alignment for Film Clipscited by 1

Built on by

Not cited yet.