mlsanityData to 7 Oct 2026

CoTracker: It is Better to Track Together

arXiv 2307.07635 ↗First posted 14 Jul 2023cs.CVcited by 516

Not on the map. Why?

Cited by 63 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce CoTracker, a transformer-based model that tracks a large number of 2D points in long video sequences. Differently from most existing approaches that track points independently, CoTracker tracks them jointly, accounting for their dependencies. We show that joint tracking significantly improves tracking accuracy and robustness, and allows CoTracker to track occluded points and points outside of the camera view. We also introduce several innovations for this class of trackers, including using token proxies that significantly improve memory efficiency and allow CoTracker to track 70k points jointly and simultaneously at inference on a single GPU. CoTracker is an online algorithm that operates causally on short windows. However, it is trained utilizing unrolled windows as a recurrent network, maintaining tracks for long periods of time even when points are occluded or leave the field of view. Quantitatively, CoTracker substantially outperforms prior trackers on standard point-tracking benchmarks.

Key facts. Cited by 516 papers in the map (data through 7 Oct 2026). Builds on 47 of its 72 references that are in the map. First posted on arXiv 14 Jul 2023.

Builds on

47 of 72 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  6. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  7. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  8. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  9. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  10. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  11. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  12. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
35 more
  1. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  2. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 566
  3. 2021Transformer Trackingcited by 412
  4. 2022Kubric: A scalable dataset generatorcited by 373
  5. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  6. 2019Learning Discriminative Model Prediction for Trackingcited by 344
  7. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
  8. 2018ATOM: Accurate Tracking by Overlap Maximizationcited by 285
  9. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  10. 2022MixFormer: End-to-End Tracking with Iterative Mixed Attentioncited by 278
  11. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  12. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  13. 2016ECO: Efficient Convolution Operators for Trackingcited by 239
  14. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
  15. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 195
  16. 2023Tracking Everything Everywhere All at Oncecited by 190
  17. 2023DynamicStereo: Consistent Dynamic Depth from Stereo Videoscited by 147
  18. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  19. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  20. 2016Learning to Track at 100 FPS with Deep Regression Networkscited by 89
  21. 2018Creatures great and SMAL: Recovering the shape and motion of animals from videocited by 84
  22. 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
  23. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  24. 2017Accurate Optical Flow via Direct Cost Volume Processingcited by 66
  25. 2022Global Matching with Overlapping Attention for Optical Flow Estimationcited by 64
  26. 2018Learning Spatial-Temporal Regularized Correlation Filters for Visual Trackingcited by 63
  27. 2023MFT: Long-Term Tracking of Every Pixelcited by 61
  28. 2022SKFlow: Learning Optical Flow with Super Kernelscited by 56
  29. 2021High-Resolution Optical Flow from 1D Attention and Correlationcited by 54
  30. 2021Learning Optical Flow from a Few Matchescited by 53
  31. 2020Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimationcited by 49
  32. 2018A Fusion Approach for Multi-Frame Optical Flow Estimationcited by 37
  33. 2014Deformable Part Models are Convolutional Neural Networkscited by 35
  34. 2017CREST: Convolutional Residual Learning for Visual Trackingcited by 29
  35. 2023Real-time volumetric rendering of dynamic humanscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 136; 2025: 208; 2026: 157.

516 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  3. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  4. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  5. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 289
  6. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  7. 20243D Reconstruction with Spatial Memorycited by 269
  8. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  9. 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
  10. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  11. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  12. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
38 more
  1. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  2. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  3. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  4. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  5. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  6. 2025Streaming 4D Visual Geometry Transformercited by 147
  7. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  8. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
  9. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 138
  10. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
  11. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
  12. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  13. 2024GameGen-X: Interactive Open-world Game Video Generationcited by 124
  14. 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 122
  15. 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
  16. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 116
  17. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 115
  18. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 114
  19. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 92
  20. 2024Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Modelcited by 90
  21. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
  22. 2023SyncTalk: The Devil is in the Synchronization for Talking Head Synthesiscited by 86
  23. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  24. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  25. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 80
  26. 2024Local All-Pair Correspondence for Point Trackingcited by 77
  27. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  28. 2024ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generationcited by 74
  29. 2023VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondencecited by 69
  30. 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3Dcited by 69
  31. 2024SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generationcited by 68
  32. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 67
  33. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  34. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  35. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  36. 2024Image Conductor: Precision Control for Interactive Video Synthesiscited by 62
  37. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 62
  38. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62

50 most cited of 516 All 516 (JSON)