mlsanityData to 14 Oct 2026

MoonSeg3R: Monocular Online Zero-Shot Segment Anything in 3D with Reconstructive Foundation Priors

arXiv 2512.15577 ↗First posted 17 Dec 2025cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we focus on online zero-shot monocular 3D instance segmentation, a novel practical setting where existing approaches fail to perform because they rely on posed RGB-D sequences. To overcome this limitation, we leverage CUT3R, a recent Reconstructive Foundation Model (RFM), to provide reliable geometric priors from a single RGB stream. We propose MoonSeg3R, which introduces three key components: (1) a self-supervised query refinement module with spatial-semantic distillation that transforms segmentation masks from 2D visual foundation models (VFMs) into discriminative 3D queries; (2) a 3D query index memory that provides temporal consistency by retrieving contextual queries; and (3) a state-distribution token from CUT3R that acts as a mask identity descriptor to strengthen cross-frame fusion. Experiments on ScanNet200 and SceneNN show that MoonSeg3R is the first method to enable online monocular 3D segmentation and achieves performance competitive with state-of-the-art RGB-D-based systems. Our code is available at https://github.com/VICO-UoE/MoonSeg3R.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 63 of its 74 references that are in the map. First posted on arXiv 17 Dec 2025.

Builds on

63 of 74 references.

References · 63

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Segment Anythingcited by 8,234
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  8. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  10. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  11. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  12. 2025DINOv3cited by 1,414
38 more
  1. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  2. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  3. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  4. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  5. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  6. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  7. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  8. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  9. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
  10. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
  11. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
  12. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
  13. 20243D Reconstruction with Spatial Memorycited by 277
  14. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 246
  15. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  16. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 242
  17. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  18. 2017Learning a Multi-View Stereo Machinecited by 194
  19. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 174
  20. 2023SAM3D: Segment Anything in 3D Scenescited by 157
  21. 2022Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentationcited by 152
  22. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 141
  23. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 141
  24. 2016SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networkscited by 135
  25. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
  26. 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 124
  27. 2024SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videoscited by 102
  28. 2019PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Thingscited by 98
  29. 2022High-Quality Entity Segmentationcited by 93
  30. 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
  31. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 87
  32. 2022SimpleRecon: 3D Reconstruction Without 3D Convolutionscited by 83
  33. 2023OVIR-3D: Open-Vocabulary 3D Instance Retrieval Without Training on 3D Datacited by 81
  34. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
  35. 2023Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusioncited by 64
  36. 2023Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without Manual Labelscited by 64
  37. 2024MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentationcited by 56
  38. 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Feed-Forward 3D Scene Modeling: A Problem-Driven Perspectivecited by 6
  2. 2026EmbodiedSplat: Online Feed-Forward Semantic 3DGS for Open-Vocabulary 3D Scene Understandingcited by 1
  3. 2026Stream3Dv2: Geometric-Semantic Fusion Enhanced Streaming Zero-Shot 3D Scene Understandingcited by 0
  4. 2026Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstructioncited by 0