mlsanityData to 7 Oct 2026

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking

arXiv 2307.15055 ↗First posted 27 Jul 2023cs.CVcited by 274

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks. Our data and code are publicly available at: https://pointodyssey.com

Key facts. Cited by 274 papers in the map (data through 7 Oct 2026). Builds on 46 of its 63 references that are in the map. First posted on arXiv 27 Jul 2023.

Builds on

46 of 63 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2017The Kinetics Human Action Video Datasetcited by 1,894
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  7. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  9. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  10. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  11. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  12. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
34 more
  1. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  2. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  3. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  4. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  5. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  6. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  7. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 424
  8. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  9. 2022Kubric: A scalable dataset generatorcited by 373
  10. 2021Learning Spatio-Temporal Transformer for Visual Trackingcited by 348
  11. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  12. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
  13. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  14. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  15. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  16. 2019MOTS: Multi-Object Tracking and Segmentationcited by 222
  17. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 221
  18. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
  19. 2018Tracking Emerges by Colorizing Videoscited by 166
  20. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151
  21. 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 136
  22. 2015MOTChallenge 2015: Towards a Benchmark for Multi-Target Trackingcited by 133
  23. 20214DComplete: Non-Rigid Motion Estimation Beyond the Observable Surfacecited by 123
  24. 2019Learning the Depths of Moving People by Watching Frozen Peoplecited by 120
  25. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  26. 2016Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothnesscited by 101
  27. 2018Creatures great and SMAL: Recovering the shape and motion of animals from videocited by 84
  28. 2022GIMO: Gaze-Informed Human Motion Prediction in Contextcited by 81
  29. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  30. 2019C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motioncited by 60
  31. 2022MIME: Human-Aware 3D Scene Generationcited by 57
  32. 2021Tracking Pedestrian Heads in Dense Crowdcited by 42
  33. 2021FEAR: Fast, Efficient, Accurate and Robust Visual Trackercited by 38
  34. 2022Learning Pixel Trajectories with Multiscale Contrastive Random Walkscited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 68; 2025: 101; 2026: 95.

274 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  3. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  4. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  5. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  6. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  7. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  8. 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
  9. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  10. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  11. 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 202
  12. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
38 more
  1. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  2. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  3. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  4. 2025Streaming 4D Visual Geometry Transformercited by 147
  5. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 138
  6. 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 122
  7. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  8. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  9. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 115
  10. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  11. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  12. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
  13. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  14. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 82
  15. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  16. 2024Local All-Pair Correspondence for Point Trackingcited by 77
  17. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
  18. 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3Dcited by 69
  19. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 67
  20. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  21. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  22. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  23. 2024HumanVid: Demystifying Training Data for Camera-controllable Human Image Animationcited by 63
  24. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
  25. 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 60
  26. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 59
  27. 2023Dense Optical Tracking: Connecting the Dotscited by 58
  28. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 54
  29. 2024Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videoscited by 54
  30. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 54
  31. 2024Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videoscited by 48
  32. 2025Efficiently Reconstructing Dynamic Scenes One D4RT at a Timecited by 46
  33. 2024TAPTR: Tracking Any Point with Transformers as Detectioncited by 43
  34. 2023DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulationcited by 42
  35. 2024Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modelingcited by 42
  36. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  37. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videoscited by 39
  38. 2023Tracking and Mapping in Medical Computer Vision: A Reviewcited by 36

50 most cited of 274 All 274 (JSON)