mlsanityData to 14 Oct 2026

ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits

arXiv 2609.00505 ↗First posted 1 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Video-text models adapted from image-text architectures (e.g., CLIP) frequently exhibit temporal blindness, the inability to perceive fundamental cues like order, direction, and motion dynamics. Standard datasets mask this limitation by enabling models to exploit static spatial shortcuts. To systematically evaluate this, we introduce XTE-Bench, a diagnostic probe revealing that even large-scale video-language models struggle with basic temporal reasoning, indicating that parameter scaling alone is insufficient to resolve this flaw. To address this, we propose Cross-Modal Temporal Edits (XTE), a self-supervised framework that injects precise temporal supervision. By performing synchronized video-text transformations, XTE generates hard temporal negatives without manual annotation. We instantiate this with ViTAL-X, a lightweight model that equips frozen image-text backbones with temporal awareness while preserving their foundational spatial knowledge. Across six temporal benchmarks, ViTAL-X achieves state-of-the-art performance. Utilizing only 0.4B parameters and 1M training clips, ViTAL-X outperforms 7B-parameter models and surpasses baselines trained on 600x more data. These results demonstrate that targeted, high-quality temporal alignment provides a highly efficient alternative to pure scaling.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 54 of its 68 references that are in the map. First posted on arXiv 1 Sep 2026.

Builds on

54 of 68 references.

References · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  5. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  7. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  8. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  9. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  10. 2017Dense-Captioning Events in Videoscited by 793
  11. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  12. 2017Localizing Moments in Video with Natural Languagecited by 663
38 more
  1. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
  2. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  3. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  4. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  5. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  6. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  7. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  8. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  9. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 320
  10. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 320
  11. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
  12. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  13. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 280
  14. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 225
  15. 2019Cross-task weakly supervised learning from instructional videoscited by 199
  16. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 179
  17. 2020SpeedNet: Learning the Speediness in Videoscited by 171
  18. 2021VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examplescited by 154
  19. 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
  20. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 105
  21. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 97
  22. 2022CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignmentcited by 74
  23. 2023Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferringcited by 58
  24. 2023Open-VCLIP: Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimizationcited by 55
  25. 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
  26. 2021Time-Equivariant Contrastive Video Representation Learningcited by 44
  27. 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videoscited by 44
  28. 2023Paxion: Patching Action Knowledge in Video-Language Foundation Modelscited by 42
  29. 2024Lost in Time: A New Temporal Benchmark for VideoLLMscited by 42
  30. 2024Text Is MASS: Modeling as Stochastic Embedding for Text-Video Retrievalcited by 33
  31. 2023ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Modelscited by 26
  32. 2024On the Consistency of Video Large Language Models in Temporal Comprehensioncited by 18
  33. 2023EZ-CLIP: Efficient Zeroshot Video Action Recognitioncited by 14
  34. 2025When the Future Becomes the Past: Taming Temporal Correspondence for Self-supervised Video Representation Learningcited by 14
  35. 2024Reversed in Time: A Novel Temporal-Emphasized Benchmark for Cross-Modal Video-Text Retrievalcited by 13
  36. 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMscited by 12
  37. 2025VideoComp: Advancing Fine-Grained Compositional and Temporal Alignment in Video-Text Modelscited by 11
  38. 2025Narrating the Video: Boosting Text-Video Retrieval via Comprehensive Utilization of Frame-Level Captionscited by 10

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Dynamic Reflections: Probing Video Representations with Text Alignment
  2. 2024TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment
  3. 2025TempoControl: Temporal Attention Guidance for Text-to-Video Models
  4. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding
  5. 2026ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering
  6. 2026Temporal and Cross-Modal Alignment for Enhanced Audiovisual Video Captioning
  7. 2025MANTA: Cross-Modal Semantic Alignment and Information-Theoretic Optimization for Long-form Multimodal Understanding
  8. 2024PiTe: Pixel-Temporal Alignment for Large Video-Language Model