mlsanityData to 7 Oct 2026

Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories

arXiv 2204.04153 ↗First posted 8 Apr 2022cs.CVcited by 215

Not on the map. Why?

Abstract (arXiv)

Tracking pixels in videos is typically studied as an optical flow estimation problem, where every pixel is described with a displacement vector that locates it in the next frame. Even though wider temporal context is freely available, prior efforts to take this into account have yielded only small gains over 2-frame methods. In this paper, we revisit Sand and Teller's "particle video" approach, and study pixel tracking as a long-range motion estimation problem, where every pixel is described with a trajectory that locates it in multiple future frames. We re-build this classic approach using components that drive the current state-of-the-art in flow and object tracking, such as dense cost maps, iterative optimization, and learned appearance updates. We train our models using long-range amodal point trajectories mined from existing optical flow data that we synthetically augment with multi-frame occlusions. We test our approach in trajectory estimation benchmarks and in keypoint label propagation tasks, and compare favorably against state-of-the-art optical flow and feature tracking methods.

Key facts. Cited by 215 papers in the map (data through 7 Oct 2026). Builds on 27 of its 45 references that are in the map. First posted on arXiv 8 Apr 2022.

Builds on

27 of 45 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  6. 2017The Kinetics Human Action Video Datasetcited by 1,894
  7. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  8. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  9. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  10. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  11. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  12. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
15 more
  1. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  2. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  3. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 228
  4. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 182
  5. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  6. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
  7. 2020MAST: A Memory-Augmented Self-supervised Trackercited by 109
  8. 2020Learning Feature Descriptors using Camera Pose Supervisioncited by 104
  9. 2018Creatures great and SMAL: Recovering the shape and motion of animals from videocited by 84
  10. 2021Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspectivecited by 81
  11. 2019C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motioncited by 60
  12. 2018Long-term Tracking in the Wild: A Benchmarkcited by 55
  13. 2021Tracking Pedestrian Heads in Dense Crowdcited by 42
  14. 2018A Fusion Approach for Multi-Frame Optical Flow Estimationcited by 37
  15. 2021Visual Correspondence Hallucinationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 32; 2024: 64; 2025: 85; 2026: 32.

215 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  3. 2023CoTracker: It is Better to Track Togethercited by 516
  4. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  5. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  6. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  7. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  8. 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
  9. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  10. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  11. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  12. 2023Tracking Everything Everywhere All at Oncecited by 190
38 more
  1. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  2. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  3. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  4. 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 122
  5. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 115
  6. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  7. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesiscited by 99
  8. 2024Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videoscited by 96
  9. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  10. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  11. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 88
  12. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
  13. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  14. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  15. 2023Segment Anything Meets Point Trackingcited by 81
  16. 2024Local All-Pair Correspondence for Point Trackingcited by 77
  17. 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3Dcited by 69
  18. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 67
  19. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  20. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
  21. 2023MFT: Long-Term Tracking of Every Pixelcited by 61
  22. 2024FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descentcited by 61
  23. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 59
  24. 2023Dense Optical Tracking: Connecting the Dotscited by 58
  25. 2023A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyondcited by 55
  26. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 54
  27. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 54
  28. 2024Matching Anything by Segmenting Anythingcited by 49
  29. 2025Efficiently Reconstructing Dynamic Scenes One D4RT at a Timecited by 46
  30. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  31. 2024TAPTR: Tracking Any Point with Transformers as Detectioncited by 43
  32. 2023DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulationcited by 42
  33. 2024Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modelingcited by 42
  34. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  35. 2023DragVideo: Interactive Drag-style Video Editingcited by 40
  36. 2024Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Reviewcited by 40
  37. 2025GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Trackingcited by 37
  38. 2023Tracking and Mapping in Medical Computer Vision: A Reviewcited by 36

50 most cited of 215 All 215 (JSON)