mlsanityData to 7 Oct 2026

SonoWorld: From One Image to a 3D Audio-Visual Scene

arXiv 2603.28757 ↗First posted 30 Mar 2026cs.CV cs.MM cs.SDcited by 3

Not on the map. Why?

Abstract (arXiv)

Tremendous progress in visual scene generation now turns a single image into an explorable 3D world, yet immersion remains incomplete without sound. We introduce Image2AVScene, the task of generating a 3D audio-visual scene from a single image, and present SonoWorld, the first framework to tackle this challenge. From one image, our pipeline outpaints a 360deg panorama, lifts it into a navigable 3D scene, places language-guided sound anchors, and renders ambisonics for point, areal, and ambient sources, yielding spatial audio aligned with scene geometry and semantics. Quantitative evaluations on a newly curated real-world dataset and a controlled user study confirm the effectiveness of our approach. Beyond free-viewpoint audio-visual rendering, we also demonstrate applications to one-shot acoustic learning and audio-visual spatial source separation. Project website: https://humathe.github.io/sonoworld/

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 72 of its 101 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

72 of 101 references.

References · 72

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  3. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  4. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  5. 2025SAM 3: Segment Anything with Conceptscited by 835
  6. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  7. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  8. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  9. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  10. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  11. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
  12. 2022AudioGen: Textually Guided Audio Generationcited by 292
38 more
  1. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  2. 2018The Sound of Pixelscited by 258
  3. 2017Objects that Soundcited by 254
  4. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  5. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  6. 2018Learning to Localize Sound Source in Visual Scenescited by 188
  7. 2020Self-Supervised Learning of Audio-Visual Objects from Videocited by 186
  8. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  9. 2021Localizing Visual Sounds the Hard Waycited by 169
  10. 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 160
  11. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  12. 2023WonderJourney: Going from Anywhere to Everywherecited by 157
  13. 2022Audio-Visual Segmentationcited by 148
  14. 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
  15. 2015Visually Indicated Soundscited by 128
  16. 2018Learning to Separate Object Sounds by Watching Unlabeled Videocited by 114
  17. 2021VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistencycited by 114
  18. 2023BuboGPT: Enabling Visual Grounding in Multi-Modal LLMscited by 114
  19. 2020Music Gesture for Visual Sound Separationcited by 112
  20. 2019Co-Separating Sounds of Visual Objectscited by 110
  21. 2020Discriminative Sounding Objects Localization via Self-supervised Audiovisual Matchingcited by 104
  22. 2024GroundingGPT:Language Enhanced Multi-modal Grounding Modelcited by 101
  23. 2017Visual to Sound: Generating Natural Sound for Videos in the Wildcited by 98
  24. 2018Self-Supervised Generation of Spatial Audio for 360 Videocited by 93
  25. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  26. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
  27. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
  28. 2022A Closer Look at Weakly-Supervised Audio-Visual Source Localizationcited by 72
  29. 2022Learning Neural Acoustic Fieldscited by 65
  30. 2017Deep Cross-Modal Audio-Visual Generationcited by 63
  31. 20182.5D Visual Soundcited by 63
  32. 2020Into the Wild with AudioScope: Unsupervised Audio-Visual Separation of On-Screen Soundscited by 55
  33. 2019Recursive Visual Sound Separation Using Minus-Plus Netcited by 53
  34. 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
  35. 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Modelscited by 51
  36. 2023AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesiscited by 49
  37. 2024Video-Guided Foley Sound Generation with Multimodal Controlscited by 49
  38. 2023Sound Source Localization is All about Cross-Modal Alignmentcited by 30

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
  2. 2026Scene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worldscited by 0
  3. 2026Hear the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generationcited by 0