mlsanityData to 14 Oct 2026

Shape of Motion: 4D Reconstruction from a Single Video

arXiv 2407.13764 ↗First posted 18 Jul 2024cs.CVcited by 250

Not on the map. Why?

Cited by 22 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Monocular dynamic reconstruction is a challenging and long-standing vision problem due to the highly ill-posed nature of the task. Existing approaches depend on templates, are effective only in quasi-static scenes, or fail to model 3D motion explicitly. We introduce a method for reconstructing generic dynamic scenes, featuring explicit, persistent 3D motion trajectories in the world coordinate frame, from casually captured monocular videos. We tackle the problem with two key insights: First, we exploit the low-dimensional structure of 3D motion by representing scene motion with a compact set of SE(3) motion bases. Each point's motion is expressed as a linear combination of these bases, facilitating soft decomposition of the scene into multiple rigidly-moving groups. Second, we take advantage of off-the-shelf data-driven priors such as monocular depth maps and long-range 2D tracks, and devise a method to effectively consolidate these noisy supervisory signals, resulting in a globally consistent representation of the dynamic scene. Experiments show that our method achieves state-of-the-art performance for both long-range 3D/2D motion estimation and novel view synthesis on dynamic scenes. Project Page: https://shape-of-motion.github.io/

Key facts. Cited by 250 papers in the map (data through 14 Oct 2026). Builds on 80 of its 138 references that are in the map. First posted on arXiv 18 Jul 2024.

Builds on

80 of 138 references.

References · 80

  1. 2023Segment Anythingcited by 8,234
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  4. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  5. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  6. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  7. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
  8. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  9. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  10. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,076
  11. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  12. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
38 more
  1. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  2. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 759
  3. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  4. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  5. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 737
  6. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  7. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  8. 2023CoTracker: It is Better to Track Togethercited by 522
  9. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  10. 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 476
  11. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  12. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  13. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
  14. 2022Kubric: A scalable dataset generatorcited by 381
  15. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 344
  16. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 327
  17. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 287
  18. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 276
  19. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 275
  20. 2020Neural Radiance Flow for 4D View Synthesis and Video Processingcited by 270
  21. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 263
  22. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 242
  23. 2023Track Anything: Segment Anything Meets Videoscited by 234
  24. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 224
  25. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  26. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  27. 2020Consistent Video Depth Estimationcited by 220
  28. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 217
  29. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 197
  30. 2023Tracking Everything Everywhere All at Oncecited by 193
  31. 20244D-Rotor Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenescited by 189
  32. 2020Robust Consistent Video Depth Estimationcited by 188
  33. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 181
  34. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 179
  35. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  36. 2020Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Cameracited by 166
  37. 2022Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-timecited by 165
  38. 2023HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motioncited by 161

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 23; 2025: 130; 2026: 97.

250 papers.

Sort:

By area:

  1. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  2. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  3. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 166
  4. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  5. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 140
  6. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 125
  7. 2025TesserAct: Learning 4D Embodied World Modelscited by 94
  8. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  9. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 89
  10. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 87
  11. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 82
  12. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
38 more
  1. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 70
  2. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 69
  3. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 67
  4. 2024STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenescited by 55
  5. 2024Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videoscited by 49
  6. 20254DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videoscited by 49
  7. 2024SplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Videocited by 44
  8. 2025Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstructioncited by 44
  9. 2025GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Trackingcited by 39
  10. 2025GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priorscited by 37
  11. 2025Segment Any Motion in Videoscited by 35
  12. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 35
  13. 2025Any4D: Unified Feed-Forward Metric 4D Reconstructioncited by 35
  14. 2025EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Meshcited by 32
  15. 2025Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Videocited by 29
  16. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 27
  17. 2025CityGS-X: A Scalable Architecture for Efficient and Geometrically Accurate Large-Scale Scene Reconstructioncited by 26
  18. 2025POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstructioncited by 25
  19. 2024SpectroMotion: Dynamic 3D Reconstruction of Specular Scenescited by 24
  20. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 24
  21. 2024Make-It-Animatable: An Efficient Framework for Authoring Animation-Ready 3D Characterscited by 23
  22. 2025Viser: Imperative, Web-based 3D Visualization in Pythoncited by 23
  23. 2025Generative Video Motion Editing with 3D Point Trackscited by 23
  24. 2025HiMoR: Monocular Deformable Gaussian Reconstruction with Hierarchical Motion Representationcited by 22
  25. 20264RC: 4D Reconstruction via Conditional Querying Anytime and Anywherecited by 22
  26. 2025FluidNexus: 3D Fluid Reconstruction and Prediction from a Single Videocited by 21
  27. 2024DynOMo: Online Point Tracking by Dynamic Online Monocular Gaussian Reconstructioncited by 20
  28. 2024Deblur4DGS: 4D Gaussian Splatting from Blurry Monocular Videocited by 20
  29. 2025BARD-GS: Blur-Aware Reconstruction of Dynamic Scenes via Gaussian Splattingcited by 19
  30. 2025Optimized Minimal 3D Gaussian Splattingcited by 19
  31. 2025iTACO: Interactable Digital Twins of Articulated Objects from Casually Captured RGBD Videoscited by 19
  32. 2025MonoFusion: Sparse-View 4D Reconstruction via Monocular Fusioncited by 19
  33. 2026Vista4D: Video Reshooting with 4D Point Cloudscited by 18
  34. 2025GaussianArt: Unified Modeling of Geometry and Motion for Articulated Objectscited by 17
  35. 2025Multi-View 3D Point Trackingcited by 17
  36. 2025PanoGS: Gaussian-based Panoptic Segmentation for 3D Open Vocabulary Scene Understandingcited by 16
  37. 2025Zero4D: Training-Free 4D Video Generation From Single Video Using Off-the-Shelf Video Diffusioncited by 16
  38. 2025Exploring Temporally-Aware Features for Point Trackingcited by 15

50 most cited of 250 All 250 (JSON)