# Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstruction

arXiv: https://arxiv.org/abs/2503.16318 · HTML: https://mlsanity.com/paper/2503.16318/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 81 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Builds on (most cited first, 30 of 56)

- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting](https://mlsanity.com/paper/2310.10642/) (2023)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)

## Built on by (most cited first, 30 of 44)

- [ViPE: Video Pose Engine for 3D Geometric Perception](https://mlsanity.com/paper/2508.10934/) (2025)
- [Efficiently Reconstructing Dynamic Scenes One D4RT at a Time](https://mlsanity.com/paper/2512.08924/) (2025)
- [LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction](https://mlsanity.com/paper/2604.14141/) (2026)
- [MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Second](https://mlsanity.com/paper/2507.10065/) (2025)
- [Any4D: Unified Feed-Forward Metric 4D Reconstruction](https://mlsanity.com/paper/2512.10935/) (2025)
- [V-DPM: 4D Video Reconstruction with Dynamic Point Maps](https://mlsanity.com/paper/2601.09499/) (2026)
- [Trace Anything: Representing Any Video in 4D via Trajectory Fields](https://mlsanity.com/paper/2510.13802/) (2025)
- [VGGT-Omega](https://mlsanity.com/paper/2605.15195/) (2026)
- [Back on Track: Bundle Adjustment for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2504.14516/) (2025)
- [4RC: 4D Reconstruction via Conditional Querying Anytime and Anywhere](https://mlsanity.com/paper/2602.10094/) (2026)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Video](https://mlsanity.com/paper/2601.05251/) (2026)
- [DELTAv2: Accelerating Dense 3D Tracking](https://mlsanity.com/paper/2508.01170/) (2025)
- [4D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimation](https://mlsanity.com/paper/2511.18416/) (2025)
- [MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer](https://mlsanity.com/paper/2603.05078/) (2026)
- [C4D: 4D Made from 3D through Dual Correspondences](https://mlsanity.com/paper/2510.14960/) (2025)
- [Flow4R: Unifying 4D Reconstruction and Tracking with Scene Flow](https://mlsanity.com/paper/2602.14021/) (2026)
- [E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models](https://mlsanity.com/paper/2506.01933/) (2025)
- [NOVA3R: Non-pixel-aligned Visual Transformer for Amodal 3D Reconstruction](https://mlsanity.com/paper/2603.04179/) (2026)
- [OccAny: Generalized Unconstrained Urban 3D Occupancy](https://mlsanity.com/paper/2603.23502/) (2026)
- [UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images](https://mlsanity.com/paper/2602.24290/) (2026)
- [Syn4D: A Multiview Synthetic 4D Dataset](https://mlsanity.com/paper/2605.05207/) (2026)
- [OmniX: Any-view and Any-time 4D Reconstruction via Feed-forward Trajectory Fields](https://mlsanity.com/paper/2607.10840/) (2026)
- [Mem4D: Decoupling Static and Dynamic Memory for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2508.07908/) (2025)
- [EAG3R: Event-Augmented 3D Geometry Estimation for Dynamic and Extreme-Lighting Scenes](https://mlsanity.com/paper/2512.00771/) (2025)
- [CogniMap3D: Cognitive 3D Mapping and Rapid Retrieval](https://mlsanity.com/paper/2601.08175/) (2026)
- [EVolSplat4D: Efficient Volume-based Gaussian Splatting for 4D Urban Scene Synthesis](https://mlsanity.com/paper/2601.15951/) (2026)
- [MotionCrafter: Dense Geometry and Motion Reconstruction with a 4D VAE](https://mlsanity.com/paper/2602.08961/) (2026)
- [RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction](https://mlsanity.com/paper/2603.20588/) (2026)
- [PointSt3R: Point Tracking through 3D Grounded Correspondence](https://mlsanity.com/paper/2510.26443/) (2025)

## Abstract (arXiv)

DUSt3R has recently shown that one can reduce many tasks in multi-view geometry, including estimating camera intrinsics and extrinsics, reconstructing the scene in 3D, and establishing image correspondences, to the prediction of a pair of viewpoint-invariant point maps, i.e., pixel-aligned point clouds defined in a common reference frame. This formulation is elegant and powerful, but unable to tackle dynamic scenes. To address this challenge, we introduce the concept of Dynamic Point Maps (DPM), extending standard point maps to support 4D tasks such as motion segmentation, scene flow estimation, 3D object tracking, and 2D correspondence. Our key intuition is that, when time is introduced, there are several possible spatial and time references that can be used to define the point maps. We identify a minimal subset of such combinations that can be regressed by a network to solve the sub tasks mentioned above. We train a DPM predictor on a mixture of synthetic and real data and evaluate it across diverse benchmarks for video depth prediction, dynamic point cloud reconstruction, 3D scene flow and object pose tracking, achieving state-of-the-art performance. Code, models and additional results are available at https://www.robots.ox.ac.uk/~vgg/research/dynamic-point-maps/.