mlsanityData to 14 Oct 2026

Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding

arXiv 2608.29177 ↗First posted 29 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The integration of novel view synthesis (NVS) and open-vocabulary segmentation (OVS) has recently yielded powerful feed-forward 3D foundation models. However, their inherent reliance on static-scene assumptions leads to severe misalignment of spatial features in unconstrained dynamic environments. To bridge this critical gap, we propose SPAR, a novel joint semantic-geometric encoding architecture that explicitly isolates transient dynamic noise prior to latent space aggregation. Furthermore, we introduce a dynamic-region-aware end-to-end training paradigm that structurally couples motion estimation with multi-view visual and semantic learning. This unified approach enables the network to inherently resolve motion conflicts and distill multi-view consistent, temporally stable scene representations from dynamic inputs. Extensive experiments on the challenging D-RE10K benchmark demonstrate that SPAR achieves state-of-the-art performance. Our end-to-end approach achieves exceptional novel view synthesis quality, yielding a PSNR of 22.15 dB and 23.33 dB given only 3 and 4 input views respectively. Despite being trained in a self-supervised manner, our model achieves an mIoU of 88.5% for motion mask prediction. Furthermore, our analysis reveals a strong inter-task synergy between photometric scene reconstruction and semantic understanding, where semantic synthesis learning consistently enhances photometric fidelity in novel view rendering. Code will be available at https://github.com/dmucby/SPAR.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 48 of its 55 references that are in the map. First posted on arXiv 29 Aug 2026.

Builds on

48 of 55 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  5. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  8. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  10. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  11. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  12. 2022Language-driven Semantic Segmentationcited by 717
36 more
  1. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  2. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
  3. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  4. 2023LERF: Language Embedded Radiance Fieldscited by 543
  5. 2023LangSplat: 3D Language Gaussian Splattingcited by 415
  6. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  7. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 364
  8. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  9. 2021Deep ViT Features as Dense Visual Descriptorscited by 330
  10. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 303
  11. 2024No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Imagescited by 248
  12. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  13. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 227
  14. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 189
  15. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 160
  16. 2024WildGaussians: 3D Gaussian Splatting in the Wildcited by 152
  17. 2024Long-LRM: Long-sequence Large Reconstruction Model for Wide-coverage Gaussian Splatscited by 114
  18. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  19. 2024FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understandingcited by 86
  20. 2024NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wildcited by 85
  21. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
  22. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 77
  23. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 77
  24. 2024Large Spatial Model: End-to-end Unposed Images to Semantic 3Dcited by 73
  25. 2022Improving Visual Grounding with Visual-Linguistic Verification and Iterative Reasoningcited by 69
  26. 2020Crowdsampling the Plenoptic Functioncited by 62
  27. 2024SpotlessSplats: Ignoring Distractors in 3D Gaussian Splattingcited by 61
  28. 2023Urban Radiance Field Representation with Deformable Neural Mesh Primitivescited by 55
  29. 2023DVIS++: Improved Decoupled Framework for Universal Video Segmentationcited by 46
  30. 2025Segment Any Motion in Videoscited by 35
  31. 2025SIU3R: Simultaneous Scene Understanding and 3D Reconstruction Beyond Feature Alignmentcited by 16
  32. 2024A Unified Framework for 3D Scene Understandingcited by 15
  33. 2024T-3DGS: Removing Transient Objects for 3D Scene Reconstructioncited by 13
  34. 2021PDNet: Toward Better One-Stage Object Detection With Prediction Decouplingcited by 5
  35. 2025Novel View Synthesis from A Few Glimpses via Test-Time Natural Video Completioncited by 4
  36. 2026WildRayZer: Self-supervised Large View Synthesis in Dynamic Environmentscited by 4

Built on by

Not cited yet.