RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction
Not on the map. Why?
Abstract (arXiv)
Streaming feed-forward 3D reconstruction enables real-time joint estimation of scene geometry and camera poses from RGB images. However, without explicit dynamic reasoning, streaming models can be affected by moving objects, causing artifacts and drift. In this work, we propose RayMap3R, a training-free streaming framework for dynamic scene reconstruction. We observe that RayMap-based predictions exhibit a static-scene bias, providing an internal cue for dynamic identification. Based on this observation, we construct a dual-branch inference scheme that identifies dynamic regions by contrasting RayMap and image predictions, suppressing their interference during memory updates. We further introduce reset metric alignment and state-aware smoothing to preserve metric consistency and stabilize predicted trajectories. Our method achieves state-of-the-art performance among streaming approaches on dynamic scene reconstruction across multiple benchmarks.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 21 Mar 2026.
Builds on
37 of 47 references.
References · 37
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
- 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
- 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
25 more
- 2025Continuous 3D Perception Model with Persistent Statecited by 548
- 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
- 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
- 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
- 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
- 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
- 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
- 20243D Reconstruction with Spatial Memorycited by 277
- 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
- 2022Deep Patch Visual Odometrycited by 190
- 2019ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residualscited by 181
- 2024MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motioncited by 162
- 2025Streaming 4D Visual Geometry Transformercited by 161
- 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifoldcited by 136
- 2024Cameras as Rays: Pose Estimation via Ray Diffusioncited by 131
- 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
- 2025VGGT-Long: Chunk it, Loop it, Align it -- Pushing VGGT's Limits on Kilometer-scale Long RGB Sequencescited by 115
- 2025Aether: Geometric-Aware Unified World Modelingcited by 97
- 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 87
- 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
- 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 56
- 2025Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstructioncited by 44
- 2025Back on Track: Bundle Adjustment for Dynamic Scene Reconstructioncited by 24
- 2025AnyCam: Learning to Recover Camera Poses and Intrinsics from Casual Videoscited by 18
Built on by
Citing papers per year: 2026: 3.
3 papers.