mlsanityData to 7 Oct 2026

Can Visual Foundation Models Achieve Long-term Point Tracking?

arXiv 2408.13575 ↗First posted 24 Aug 2024cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Large-scale vision foundation models have demonstrated remarkable success across various tasks, underscoring their robust generalization capabilities. While their proficiency in two-view correspondence has been explored, their effectiveness in long-term correspondence within complex environments remains unexplored. To address this, we evaluate the geometric awareness of visual foundation models in the context of point tracking: (i) in zero-shot settings, without any training; (ii) by probing with low-capacity layers; (iii) by fine-tuning with Low Rank Adaptation (LoRA). Our findings indicate that features from Stable Diffusion and DINOv2 exhibit superior geometric correspondence abilities in zero-shot settings. Furthermore, DINOv2 achieves performance comparable to supervised models in adaptation settings, demonstrating its potential as a strong initialization for correspondence learning.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 31 of its 35 references that are in the map. First posted on arXiv 24 Aug 2024.

Builds on

31 of 35 references.

References · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2023Vision Transformers Need Registerscited by 844
  9. 2023CoTracker: It is Better to Track Togethercited by 516
  10. 2023Emergent Correspondence from Image Diffusioncited by 437
  11. 2022DeiT III: Revenge of the ViTcited by 417
  12. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
19 more
  1. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  2. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  3. 2023Visual Geometry Grounded Deep Structure From Motioncited by 225
  4. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
  5. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
  6. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 186
  7. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  8. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  9. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  10. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 115
  11. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 88
  12. 2023Telling Left from Right: Identifying Geometry-Aware Semantic Correspondencecited by 85
  13. 2016Universal Correspondence Networkcited by 82
  14. 2023Segment Anything Meets Point Trackingcited by 81
  15. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 54
  16. 2020Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondencecited by 53
  17. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  18. 2023VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentationcited by 36
  19. 2023MeLo: Low-rank Adaptation is Better than Fine-tuning for Medical Image Diagnosiscited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 6.

14 papers.

Sort:

By area:

  1. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  2. 2025Emergent Temporal Correspondences from Video Diffusion Transformerscited by 30
  3. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 26
  4. 2025Exploring Temporally-Aware Features for Point Trackingcited by 15
  5. 2024Robust Bird's Eye View Segmentation by Adapting DINOv2cited by 6
  6. 2026Video Understanding: From Geometry and Semantics to Unified Modelscited by 5
  7. 2026MegaFlow: Zero-Shot Large Displacement Optical Flowcited by 5
  8. 2025ProTracker: Probabilistic Integration for Robust and Accurate Point Trackingcited by 3
  9. 2025Probing and Leveraging Video Diffusion Transformer Features for Robust Point Trackingcited by 3
  10. 2025Benchmarking Feature Upsampling Methods for Vision Foundation Models using Interactive Segmentationcited by 1
  11. 2026Spectral Probing of Feature Upsamplers in 2D-to-3D Scene Reconstructioncited by 0
  12. 2026M2P: Improving Visual Foundation Models with Mask-to-Point Weakly-Supervised Learning for Dense Point Trackingcited by 0
2 more
  1. 2026SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Modelscited by 0
  2. 2026Two Global Crops Suffice: Locating Semantic Emergence in DINO-Style Self-Supervised Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Online Long-term Point Tracking in the Foundation Model Era
  2. 2026Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence
  3. 2025CAMEO: Correspondence-Attention Alignment for Multi-View Diffusion Models
  4. 2026Repurposing Geometric Foundation Models for Multi-view Diffusion
  5. 2026Landscape-Awareness for Geometric View Diffusion Model
  6. 2026GOT-Edit: Geometry-Aware Generic Object Tracking via Online Model Editing
  7. 2025Evaluating Foundation Models' 3D Understanding Through Multi-View Correspondence Analysis
  8. 2026MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation