mlsanityData to 7 Oct 2026

Local All-Pair Correspondence for Point Tracking

arXiv 2407.15420 ↗First posted 22 Jul 2024cs.CVcited by 77

Not on the map. Why?

Abstract (arXiv)

We introduce LocoTrack, a highly accurate and efficient model designed for the task of tracking any point (TAP) across video sequences. Previous approaches in this task often rely on local 2D correlation maps to establish correspondences from a point in the query image to a local region in the target image, which often struggle with homogeneous regions or repetitive features, leading to matching ambiguities. LocoTrack overcomes this challenge with a novel approach that utilizes all-pair correspondences across regions, i.e., local 4D correlation, to establish precise correspondences, with bidirectional correspondence and matching smoothness significantly enhancing robustness against ambiguities. We also incorporate a lightweight correlation encoder to enhance computational efficiency, and a compact Transformer architecture to integrate long-term temporal information. LocoTrack achieves unmatched accuracy on all TAP-Vid benchmarks and operates at a speed almost 6 times faster than the current state-of-the-art.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Builds on 53 of its 70 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

53 of 70 references.

References · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  9. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  10. 2017The Kinetics Human Action Video Datasetcited by 1,894
  11. 2018Group Normalizationcited by 1,514
  12. 2017Deformable Convolutional Networkscited by 1,481
38 more
  1. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  2. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  3. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  4. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  5. 2018Self-Attention with Relative Position Representationscited by 1,112
  6. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  7. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  8. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  9. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 556
  10. 2023CoTracker: It is Better to Track Togethercited by 516
  11. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  12. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
  13. 2022Kubric: A scalable dataset generatorcited by 373
  14. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  15. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  16. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  17. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
  18. 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
  19. 2023Tracking Everything Everywhere All at Oncecited by 190
  20. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  21. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 182
  22. 2017Learning to Find Good Correspondencescited by 176
  23. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  24. 2017Convolutional neural network architecture for geometric matchingcited by 138
  25. 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
  26. 2021Learning Accurate Dense Correspondences and When to Trust Themcited by 113
  27. 2020Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutionscited by 92
  28. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 88
  29. 2021PDC-Net+: Enhanced Probabilistic Dense Correspondence Networkcited by 80
  30. 2021CATs: Cost Aggregation Transformers for Visual Correspondencecited by 77
  31. 2021Relational Embedding for Few-Shot Classificationcited by 74
  32. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  33. 2020Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learningcited by 69
  34. 2018DGC-Net: Dense Geometric Correspondence Networkcited by 67
  35. 2023MFT: Long-Term Tracking of Every Pixelcited by 61
  36. 2019SFNet: Learning Object-aware Semantic Correspondencecited by 60
  37. 2023Dense Optical Tracking: Connecting the Dotscited by 58
  38. 2020GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Networkcited by 53

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 42; 2026: 25.

77 papers.

Sort:

By area:

  1. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  2. 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 122
  3. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  4. 2025TAPIP3D: Tracking Any Point in Persistent 3D Geometrycited by 67
  5. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
  6. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 54
  7. 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generationcited by 50
  8. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  9. 2025TAPNext: Tracking Any Point (TAP) as Next Token Predictioncited by 35
  10. 2025Trace Anything: Representing Any Video in 4D via Trajectory Fieldscited by 31
  11. 2025Emergent Temporal Correspondences from Video Diffusion Transformerscited by 30
  12. 2025Back on Track: Bundle Adjustment for Dynamic Scene Reconstructioncited by 23
38 more
  1. 2025Generative Video Motion Editing with 3D Point Trackscited by 21
  2. 2024TAPTRv2: Attention-based Position Update Improves Tracking Any Pointcited by 18
  3. 2025Articulated Object Estimation in the Wildcited by 18
  4. 2025Multi-View 3D Point Trackingcited by 17
  5. 2025Exploring Temporally-Aware Features for Point Trackingcited by 15
  6. 2025Seurat: From Moving Points to Depthcited by 13
  7. 2025MATCHA:Towards Matching Anythingcited by 12
  8. 2026CoWTracker: Tracking by Warping instead of Correlationcited by 12
  9. 2024EgoPoints: Advancing Point Tracking for Egocentric Videoscited by 10
  10. 2026Track4World: Feedforward World-centric Dense 3D Tracking of All Pixelscited by 10
  11. 2024ETAP: Event-based Tracking of Any Pointcited by 9
  12. 2025TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motioncited by 9
  13. 2025DELTAv2: Accelerating Dense 3D Trackingcited by 9
  14. 2024TAPTRv3: Spatial and Temporal Context Foster Robust Tracking of Any Point in Long Videocited by 8
  15. 2025Tracktention: Leveraging Point Tracking to Attend Videos Faster and Bettercited by 8
  16. 2026AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstructioncited by 8
  17. 2025Stereo Any Video: Temporally Consistent Stereo Matchingcited by 7
  18. 2025Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognitioncited by 7
  19. 2025C4D: 4D Made from 3D through Dual Correspondencescited by 7
  20. 2025TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixelscited by 6
  21. 2024Perception Test 2024: Challenge Summary and a Novel Hour-Long VideoQA Benchmarkcited by 5
  22. 2025LiteTracker: Leveraging Temporal Causality for Accurate Low-latency Tissue Trackingcited by 5
  23. 2026Video Understanding: From Geometry and Semantics to Unified Modelscited by 5
  24. 2026MegaFlow: Zero-Shot Large Displacement Optical Flowcited by 5
  25. 2026TAPNext++: What's Next for Tracking Any Point (TAP)?cited by 5
  26. 2024MATE: Motion-Augmented Temporal Consistency for Event-based Point Trackingcited by 4
  27. 2025MV-TAP: Tracking Any Point in Multi-View Videoscited by 4
  28. 2025ProTracker: Probabilistic Integration for Robust and Accurate Point Trackingcited by 3
  29. 2025AnthroTAP: Learning Point Tracking with Real-World Motioncited by 3
  30. 2025Track-On2: Enhancing Online Point Tracking with Memorycited by 3
  31. 2025Probing and Leveraging Video Diffusion Transformer Features for Robust Point Trackingcited by 3
  32. 2026TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Trackingcited by 3
  33. 2025Taming Modern Point Tracking for Speckle Tracking Echocardiography via Impartial Motioncited by 2
  34. 2025Is This Tracker On? A Benchmark Protocol for Dynamic Trackingcited by 2
  35. 2025PointSt3R: Point Tracking through 3D Grounded Correspondencecited by 2
  36. 2025Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understandingcited by 2
  37. 2026MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generationcited by 2
  38. 2025SAM 2++: Tracking Anything at Any Granularitycited by 1

50 most cited of 77 All 77 (JSON)