ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits
Not on the map. Why?
Abstract (arXiv)
Video-text models adapted from image-text architectures (e.g., CLIP) frequently exhibit temporal blindness, the inability to perceive fundamental cues like order, direction, and motion dynamics. Standard datasets mask this limitation by enabling models to exploit static spatial shortcuts. To systematically evaluate this, we introduce XTE-Bench, a diagnostic probe revealing that even large-scale video-language models struggle with basic temporal reasoning, indicating that parameter scaling alone is insufficient to resolve this flaw. To address this, we propose Cross-Modal Temporal Edits (XTE), a self-supervised framework that injects precise temporal supervision. By performing synchronized video-text transformations, XTE generates hard temporal negatives without manual annotation. We instantiate this with ViTAL-X, a lightweight model that equips frozen image-text backbones with temporal awareness while preserving their foundational spatial knowledge. Across six temporal benchmarks, ViTAL-X achieves state-of-the-art performance. Utilizing only 0.4B parameters and 1M training clips, ViTAL-X outperforms 7B-parameter models and surpasses baselines trained on 600x more data. These results demonstrate that targeted, high-quality temporal alignment provides a highly efficient alternative to pure scaling.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 54 of its 68 references that are in the map. First posted on arXiv 1 Sep 2026.
Builds on
54 of 68 references.
References · 54
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
- 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
- 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
- 2017Dense-Captioning Events in Videoscited by 793
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
- 2017Localizing Moments in Video with Natural Languagecited by 663
38 more
- 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
- 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
- 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
- 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
- 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
- 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
- 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
- 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 320
- 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 320
- 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
- 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
- 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 280
- 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 225
- 2019Cross-task weakly supervised learning from instructional videoscited by 199
- 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 179
- 2020SpeedNet: Learning the Speediness in Videoscited by 171
- 2021VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examplescited by 154
- 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
- 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 105
- 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 97
- 2022CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignmentcited by 74
- 2023Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferringcited by 58
- 2023Open-VCLIP: Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimizationcited by 55
- 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
- 2021Time-Equivariant Contrastive Video Representation Learningcited by 44
- 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videoscited by 44
- 2023Paxion: Patching Action Knowledge in Video-Language Foundation Modelscited by 42
- 2024Lost in Time: A New Temporal Benchmark for VideoLLMscited by 42
- 2024Text Is MASS: Modeling as Stochastic Embedding for Text-Video Retrievalcited by 33
- 2023ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Modelscited by 26
- 2024On the Consistency of Video Large Language Models in Temporal Comprehensioncited by 18
- 2023EZ-CLIP: Efficient Zeroshot Video Action Recognitioncited by 14
- 2025When the Future Becomes the Past: Taming Temporal Correspondence for Self-supervised Video Representation Learningcited by 14
- 2024Reversed in Time: A Novel Temporal-Emphasized Benchmark for Cross-Modal Video-Text Retrievalcited by 13
- 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMscited by 12
- 2025VideoComp: Advancing Fine-Grained Compositional and Temporal Alignment in Video-Text Modelscited by 11
- 2025Narrating the Video: Boosting Text-Video Retrieval via Comprehensive Utilization of Frame-Level Captionscited by 10
50 most cited of 54 All 54 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Dynamic Reflections: Probing Video Representations with Text Alignment
- 2024TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment
- 2025TempoControl: Temporal Attention Guidance for Text-to-Video Models
- 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding
- 2026ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering
- 2026Temporal and Cross-Modal Alignment for Enhanced Audiovisual Video Captioning
- 2025MANTA: Cross-Modal Semantic Alignment and Information-Theoretic Optimization for Long-form Multimodal Understanding
- 2024PiTe: Pixel-Temporal Alignment for Large Video-Language Model