mlsanityData to 14 Oct 2026

MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion

arXiv 2410.03825 ↗First posted 4 Oct 2024cs.CVcited by 442

Not on the map. Why?

Cited by 32 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Estimating geometry from dynamic scenes, where objects move and deform over time, remains a core challenge in computer vision. Current approaches often rely on multi-stage pipelines or global optimizations that decompose the problem into subtasks, like depth and flow, leading to complex systems prone to errors. In this paper, we present Motion DUSt3R (MonST3R), a novel geometry-first approach that directly estimates per-timestep geometry from dynamic scenes. Our key insight is that by simply estimating a pointmap for each timestep, we can effectively adapt DUST3R's representation, previously only used for static scenes, to dynamic scenes. However, this approach presents a significant challenge: the scarcity of suitable training data, namely dynamic, posed videos with depth labels. Despite this, we show that by posing the problem as a fine-tuning task, identifying several suitable datasets, and strategically training the model on this limited data, we can surprisingly enable the model to handle dynamics, even without an explicit motion representation. Based on this, we introduce new optimizations for several downstream video-specific tasks and demonstrate strong performance on video depth and camera pose estimation, outperforming prior work in terms of robustness and efficiency. Moreover, MonST3R shows promising results for primarily feed-forward 4D reconstruction.

Key facts. Cited by 442 papers in the map (data through 14 Oct 2026). Builds on 58 of its 77 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

58 of 77 references.

References · 58

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  4. 2015Fast R-CNNcited by 2,933
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  6. 2021Vision Transformers for Dense Predictioncited by 1,803
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  8. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  9. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  10. 2024Depth Anything V2cited by 1,422
  11. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  12. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
38 more
  1. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  2. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  3. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  4. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  5. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  6. 2019Deep Learning for 3D Point Clouds: A Surveycited by 558
  7. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
  8. 2016Direct Sparse Odometrycited by 485
  9. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 405
  10. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 327
  11. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 276
  12. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 250
  13. 2020Consistent Video Depth Estimationcited by 220
  14. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 216
  15. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 204
  16. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  17. 2022Deep Patch Visual Odometrycited by 190
  18. 2020Robust Consistent Video Depth Estimationcited by 188
  19. 2018DeepV2D: Video to Depth with Differentiable Structure from Motioncited by 185
  20. 2019ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residualscited by 181
  21. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  22. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 179
  23. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  24. 2018BA-Net: Dense Bundle Adjustment Networkcited by 175
  25. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 174
  26. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 172
  27. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  28. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 166
  29. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 160
  30. 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
  31. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  32. 2020TartanVO: A Generalizable Learning-based VOcited by 108
  33. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  34. 2022ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wildcited by 92
  35. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  36. 2021Consistent Depth of Moving Objects in Videocited by 56
  37. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 56
  38. 2022SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenescited by 52

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 25; 2025: 218; 2026: 199.

442 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  2. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  3. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  4. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
  5. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 319
  6. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  7. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  8. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
  9. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 166
  10. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  11. 2025Streaming 4D Visual Geometry Transformercited by 161
  12. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 141
38 more
  1. 2025Video World Models with Long-term Spatial Memorycited by 136
  2. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
  3. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 125
  4. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 117
  5. 2024SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videoscited by 102
  6. 2025Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditioncited by 98
  7. 2025Aether: Geometric-Aware Unified World Modelingcited by 97
  8. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 91
  9. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  10. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 89
  11. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 87
  12. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 87
  13. 2024Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localizationcited by 82
  14. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 82
  15. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  16. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 77
  17. 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priorscited by 72
  18. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 70
  19. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 69
  20. 2025Visual Imitation Enables Contextual Humanoid Controlcited by 68
  21. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 67
  22. 2025WorldMirror: Universal 3D World Reconstruction with Any-Prior Promptingcited by 67
  23. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 65
  24. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 62
  25. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 57
  26. 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 54
  27. 2025Efficiently Reconstructing Dynamic Scenes One D4RT at a Timecited by 54
  28. 2025WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environmentscited by 50
  29. 2024Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videoscited by 49
  30. 20254DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videoscited by 49
  31. 2025Light3R-SfM: Towards Feed-forward Structure-from-Motioncited by 47
  32. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  33. 2025CamCloneMaster: Enabling Reference-based Camera Control for Video Generationcited by 46
  34. 2025DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoningcited by 46
  35. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videoscited by 46
  36. 2025LONG3R: Long Sequence Streaming 3D Reconstructioncited by 45
  37. 2025Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstructioncited by 44
  38. 2025GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Trackingcited by 39

50 most cited of 442 All 442 (JSON)