# Autogenic Language Embedding for Coherent Point Tracking

arXiv: https://arxiv.org/abs/2407.20730 · HTML: https://mlsanity.com/paper/2407.20730/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 50 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement](https://mlsanity.com/paper/2306.08637/) (2023)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories](https://mlsanity.com/paper/2204.04153/) (2022)
- [Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence](https://mlsanity.com/paper/2305.14334/) (2023)
- [Tracking Everything Everywhere All at Once](https://mlsanity.com/paper/2306.05422/) (2023)
- [Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark](https://mlsanity.com/paper/2103.16746/) (2021)
- [Transformer Tracking with Cyclic Shifting Window Attention](https://mlsanity.com/paper/2205.03806/) (2022)
- [Accurate Optical Flow via Direct Cost Volume Processing](https://mlsanity.com/paper/1704.07325/) (2017)
- [MFT: Long-Term Tracking of Every Pixel](https://mlsanity.com/paper/2305.12998/) (2023)
- [Prompting for Multi-Modal Tracking](https://mlsanity.com/paper/2207.14571/) (2022)
- [Learning to Compose Hypercolumns for Visual Correspondence](https://mlsanity.com/paper/2007.10587/) (2020)
- [Compact Transformer Tracker with Correlative Masked Modeling](https://mlsanity.com/paper/2301.10938/) (2023)
- [Divert More Attention to Vision-Language Tracking](https://mlsanity.com/paper/2207.01076/) (2022)
- [DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception](https://mlsanity.com/paper/2405.15232/) (2024)
- [All in One: Exploring Unified Vision-Language Tracking with Multi-Modal Alignment](https://mlsanity.com/paper/2307.03373/) (2023)

## Built on by (most cited first, 17 of 17)

- [ReTrack: Evidence-Driven Dual-Stream Directional Anchor Calibration Network for Composed Video Retrieval](https://mlsanity.com/paper/2604.17898/) (2026)
- [INTENT: Invariance and Discrimination-aware Noise Mitigation for Robust Composed Image Retrieval](https://mlsanity.com/paper/2604.18051/) (2026)
- [HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval](https://mlsanity.com/paper/2604.18037/) (2026)
- [Air-Know: Arbiter-Calibrated Knowledge-Internalizing Robust Network for Composed Image Retrieval](https://mlsanity.com/paper/2604.19386/) (2026)
- [ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrieval](https://mlsanity.com/paper/2604.20358/) (2026)
- [Video Anomaly Detection with Motion and Appearance Guided Patch Diffusion Model](https://mlsanity.com/paper/2412.09026/) (2024)
- [MVP: Winning Solution to SMP Challenge 2025 Video Track](https://mlsanity.com/paper/2507.00950/) (2025)
- [Large Language Model as Token Compressor and Decompressor](https://mlsanity.com/paper/2603.25340/) (2026)
- [HotComment: A Benchmark for Evaluating Popularity of Online Comments](https://mlsanity.com/paper/2604.25614/) (2026)
- [RAGTrack: Language-aware RGBT Tracking with Retrieval-Augmented Generation](https://mlsanity.com/paper/2603.03617/) (2026)
- [Hypergraph-State Collaborative Reasoning for Multi-Object Tracking](https://mlsanity.com/paper/2604.12665/) (2026)
- [IP-MOT: Instance Prompt Learning for Cross-Domain Multi-Object Tracking](https://mlsanity.com/paper/2410.23907/) (2024)
- [IMAGINE: Adaptive Schema-Imagery Enhanced Composition for Composed Video Retrieval](https://mlsanity.com/paper/2606.08144/) (2026)
- [From Topology to Trajectory: LLM-Driven World Models For Supply Chain Resilience](https://mlsanity.com/paper/2604.11041/) (2026)
- [GateMOT: Q-Gated Attention for Dense Object Tracking](https://mlsanity.com/paper/2604.26353/) (2026)
- [PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization](https://mlsanity.com/paper/2606.03444/) (2026)
- [RankVR: Low-Rank Structure Perception and Value Recalibration for Robust Composed Image Retrieval](https://mlsanity.com/paper/2606.11689/) (2026)

## Abstract (arXiv)

Point tracking is a challenging task in computer vision, aiming to establish point-wise correspondence across long video sequences. Recent advancements have primarily focused on temporal modeling techniques to improve local feature similarity, often overlooking the valuable semantic consistency inherent in tracked points. In this paper, we introduce a novel approach leveraging language embeddings to enhance the coherence of frame-wise visual features related to the same object. Our proposed method, termed autogenic language embedding for visual feature enhancement, strengthens point correspondence in long-term sequences. Unlike existing visual-language schemes, our approach learns text embeddings from visual features through a dedicated mapping network, enabling seamless adaptation to various tracking tasks without explicit text annotations. Additionally, we introduce a consistency decoder that efficiently integrates text tokens into visual features with minimal computational overhead. Through enhanced visual consistency, our approach significantly improves tracking trajectories in lengthy videos with substantial appearance variations. Extensive experiments on widely-used tracking benchmarks demonstrate the superior performance of our method, showcasing notable enhancements compared to trackers relying solely on visual cues.