mlsanityData to 7 Oct 2026

LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Control

arXiv 2406.16038 ↗First posted 23 Jun 2024cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper scales object-level reconstruction to complex scenes, advancing interactive scene reconstruction. We introduce two datasets, OmniSim and InterReal, featuring 28 scenes with multiple interactive objects. To tackle the challenge of inaccurate interactive motion recovery in complex scenes, we propose LiveScene, a scene-level language-embedded interactive radiance field that efficiently reconstructs and controls multiple objects. By decomposing the interactive scene into local deformable fields, LiveScene enables separate reconstruction of individual object motions, reducing memory consumption. Additionally, our interaction-aware language embedding localizes individual interactive objects, allowing for arbitrary control using natural language. Our approach demonstrates significant superiority in novel view synthesis, interactive scene control, and language grounding performance through extensive experiments. Project page: https://livescenes.github.io.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 76 of its 91 references that are in the map. First posted on arXiv 23 Jun 2024.

Builds on

76 of 91 references.

References · 76

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Segment Anythingcited by 8,158
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  10. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  11. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  12. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
38 more
  1. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  3. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  4. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 753
  5. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  6. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  7. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  8. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  9. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  10. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  11. 2023LERF: Language Embedded Radiance Fieldscited by 538
  12. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  13. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  14. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  15. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  16. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  17. 2023LangSplat: 3D Language Gaussian Splattingcited by 410
  18. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  19. 2022Visual Language Maps for Robot Navigationcited by 362
  20. 2022Fast Dynamic Radiance Fields with Time-Aware Neural Voxelscited by 344
  21. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
  22. 2023Text-to-3D using Gaussian Splattingcited by 299
  23. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  24. 2023GS-SLAM: Dense Visual SLAM with 3D Gaussian Splattingcited by 278
  25. 2020Neural Radiance Flow for 4D View Synthesis and Video Processingcited by 270
  26. 2022Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Renderingcited by 217
  27. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
  28. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 206
  29. 2023Language Embedded 3D Gaussians for Open-Vocabulary Scene Understandingcited by 205
  30. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 195
  31. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  32. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 164
  33. 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 159
  34. 2022NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fieldscited by 151
  35. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 140
  36. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  37. 2022RigNeRF: Fully Controllable Neural 3D Portraitscited by 126
  38. 2022Mixed Neural Voxels for Fast Multi-view Video Synthesiscited by 126

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 4.

6 papers.

  1. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  2. 2024Learning 2D Invariant Affordance Knowledge for 3D Affordance Groundingcited by 21
  3. 2025GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulationcited by 16
  4. 2025AerialVG: A Challenging Benchmark for Aerial Visual Grounding by Exploring Positional Relationscited by 10
  5. 2025M3: 3D-Spatial MultiModal Memorycited by 3
  6. 2024FreeGaussian: Annotation-free Control of Articulated Objects via 3D Gaussian Splats with Flow Derivativescited by 1