mlsanityData to 7 Oct 2026

BootsTAP: Bootstrapped Training for Tracking-Any-Point

arXiv 2402.00847 ↗First posted 1 Feb 2024cs.CV stat.MLcited by 115

Not on the map. Why?

Abstract (arXiv)

To endow models with greater understanding of physics and motion, it is useful to enable them to perceive how solid surfaces move and deform in real scenes. This can be formalized as Tracking-Any-Point (TAP), which requires the algorithm to track any point on solid surfaces in a video, potentially densely in space and time. Large-scale groundtruth training data for TAP is only available in simulation, which currently has a limited variety of objects and motion. In this work, we demonstrate how large-scale, unlabeled, uncurated real-world data can improve a TAP model with minimal architectural changes, using a selfsupervised student-teacher setup. We demonstrate state-of-the-art performance on the TAP-Vid benchmark surpassing previous results by a wide margin: for example, TAP-Vid-DAVIS performance improves from 61.3% to 67.4%, and TAP-Vid-Kinetics from 57.2% to 62.5%. For visualizations, see our project webpage at https://bootstap.github.io/

Key facts. Cited by 115 papers in the map (data through 7 Oct 2026). Builds on 47 of its 77 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

47 of 77 references.

References · 47

  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  5. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  6. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  7. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  8. 2023CoTracker: It is Better to Track Togethercited by 516
  9. 2022Kubric: A scalable dataset generatorcited by 373
  10. 2023Perception Test: A Diagnostic Benchmark for Multimodal Video Modelscited by 310
  11. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  12. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
35 more
  1. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  2. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  3. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  4. 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
  5. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
  6. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  7. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  8. 2017Multi-task Self-Supervised Visual Learningcited by 200
  9. 2023Tracking Everything Everywhere All at Oncecited by 190
  10. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 182
  11. 2018Tracking Emerges by Colorizing Videoscited by 166
  12. 2019SelFlow: Self-Supervised Learning of Optical Flowcited by 128
  13. 2020Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimationcited by 120
  14. 2020MAST: A Memory-Augmented Self-supervised Trackercited by 109
  15. 2016Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothnesscited by 101
  16. 2017Occlusion Aware Unsupervised Learning of Optical Flowcited by 101
  17. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 88
  18. 2023Segment Anything Meets Point Trackingcited by 81
  19. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 80
  20. 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
  21. 2019DDFlow: Learning Optical Flow with Unlabeled Data Distillationcited by 63
  22. 2023MFT: Long-Term Tracking of Every Pixelcited by 61
  23. 2023Dense Optical Tracking: Connecting the Dotscited by 58
  24. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 54
  25. 2014Unsupervised Learning of Spatiotemporally Coherent Metricscited by 45
  26. 2022Learning Pixel Trajectories with Multiscale Contrastive Random Walkscited by 36
  27. 2015Learning to Linearize Under Uncertaintycited by 25
  28. 2023Self-Supervised 3D Scene Flow Estimation Guided by Superpointscited by 24
  29. 2023DriveTrack: A Benchmark for Long-Range Point Tracking in Real-World Videoscited by 24
  30. 2023Surgical Tattoos in Infrared: A Dataset for Quantifying Tissue Tracking and Mappingcited by 15
  31. 2024Refining Pre-Trained Motion Modelscited by 12
  32. 2022Self-supervised AutoFlowcited by 10
  33. 2022Semi-Supervised Learning of Optical Flow by Flow Supervisorcited by 9
  34. 2021Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimationcited by 6
  35. 2023Learning Fine-Grained Features for Pixel-wise Video Correspondencescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 18; 2025: 58; 2026: 39.

115 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  3. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  4. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  5. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  6. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
  7. 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 122
  8. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  9. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  10. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 74
  11. 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3Dcited by 69
  12. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
38 more
  1. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 54
  2. 2024TAPTR: Tracking Any Point with Transformers as Detectioncited by 43
  3. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  4. 2025GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Trackingcited by 37
  5. 2025TAPNext: Tracking Any Point (TAP) as Next Token Predictioncited by 35
  6. 2025Segment Any Motion in Videoscited by 34
  7. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 33
  8. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 31
  9. 2024Scaling 4D Representationscited by 30
  10. 2025Dynamic Camera Poses and Where to Find Themcited by 26
  11. 2024TAPTRv2: Attention-based Position Update Improves Tracking Any Pointcited by 18
  12. 2025BARD-GS: Blur-Aware Reconstruction of Dynamic Scenes via Gaussian Splattingcited by 18
  13. 2025Articulated Object Estimation in the Wildcited by 18
  14. 2025Multi-View 3D Point Trackingcited by 17
  15. 2025Exploring Temporally-Aware Features for Point Trackingcited by 15
  16. 2024Can Visual Foundation Models Achieve Long-term Point Tracking?cited by 14
  17. 2025Learning segmentation from point trajectoriescited by 14
  18. 2025L4P: Towards Unified Low-Level 4D Vision Perceptioncited by 12
  19. 2026CoWTracker: Tracking by Warping instead of Correlationcited by 12
  20. 2025MoBGS: Motion Deblurring Dynamic 3D Gaussian Splatting for Blurry Monocular Videocited by 11
  21. 2025TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videoscited by 11
  22. 2024EgoPoints: Advancing Point Tracking for Egocentric Videoscited by 10
  23. 2025Direct Motion Models for Assessing Generated Videoscited by 10
  24. 2026Track4World: Feedforward World-centric Dense 3D Tracking of All Pixelscited by 10
  25. 2024VeGaS: Video Gaussian Splattingcited by 9
  26. 2025DELTAv2: Accelerating Dense 3D Trackingcited by 9
  27. 2024TAPTRv3: Spatial and Temporal Context Foster Robust Tracking of Any Point in Long Videocited by 8
  28. 2025Tracktention: Leveraging Point Tracking to Attend Videos Faster and Bettercited by 8
  29. 2025Orientation-anchored Hyper-Gaussian for 4D Reconstruction from Casual Videoscited by 8
  30. 2026AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstructioncited by 8
  31. 2026MLLM-4D: Towards Visual-based Spatial-Temporal Intelligencecited by 8
  32. 2025Point Tracking in Surgery--The 2024 Surgical Tattoos in Infrared (STIR) Challengecited by 6
  33. 20254D-Animal: Freely Reconstructing Animatable 3D Animals from Videoscited by 6
  34. 2026LabelAny3D: Label Any Object 3D in the Wildcited by 6
  35. 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 6
  36. 2025LiteTracker: Leveraging Temporal Causality for Accurate Low-latency Tissue Trackingcited by 5
  37. 2025Point Prompting: Counterfactual Tracking with Video Diffusion Modelscited by 5
  38. 2025Prior-Enhanced Gaussian Splatting for Dynamic Scene Reconstruction from Casual Videocited by 5

50 most cited of 115 All 115 (JSON)