# Frame-Difference Guided Dynamic Region Perception for CLIP Adaptation in Text-Video Retrieval

arXiv: https://arxiv.org/abs/2510.21806 · HTML: https://mlsanity.com/paper/2510.21806/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 22 references that are in the map.
- First posted on arXiv 21 Oct 2025.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval](https://mlsanity.com/paper/2207.07285/) (2022)
- [CLIP2Video: Mastering Video-Text Retrieval via Image CLIP](https://mlsanity.com/paper/2106.11097/) (2021)
- [Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoning](https://mlsanity.com/paper/2003.00392/) (2020)
- [Alpha-CLIP: A CLIP Model Focusing on Wherever You Want](https://mlsanity.com/paper/2312.03818/) (2023)
- [X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval](https://mlsanity.com/paper/2203.15086/) (2022)
- [TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval](https://mlsanity.com/paper/2207.07852/) (2022)
- [T2VLAD: Global-Local Sequence Alignment for Text-Video Retrieval](https://mlsanity.com/paper/2104.10054/) (2021)
- [HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrieval](https://mlsanity.com/paper/2103.15049/) (2021)
- [CenterCLIP: Token Clustering for Efficient Text-Video Retrieval](https://mlsanity.com/paper/2205.00823/) (2022)
- [Disentangled Representation Learning for Text-Video Retrieval](https://mlsanity.com/paper/2203.07111/) (2022)
- [Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations](https://mlsanity.com/paper/2211.11427/) (2022)
- [Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval](https://mlsanity.com/paper/2308.07648/) (2023)
- [Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning](https://mlsanity.com/paper/2309.01083/) (2023)
- [Unified Pre-training with Pseudo Texts for Text-To-Image Person Re-identification](https://mlsanity.com/paper/2309.01420/) (2023)
- [CLIP2TV: Align, Match and Distill for Video-Text Retrieval](https://mlsanity.com/paper/2111.05610/) (2021)

## Abstract (arXiv)

With the rapid growth of video data, text-video retrieval technology has become increasingly important in numerous application scenarios such as recommendation and search. Early text-video retrieval methods suffer from two critical drawbacks: first, they heavily rely on large-scale annotated video-text pairs, leading to high data acquisition costs; second, there is a significant modal gap between video and text features, which limits cross-modal alignment accuracy. With the development of vision-language model, adapting CLIP to video tasks has attracted great attention. However, existing adaptation methods generally lack enhancement for dynamic video features and fail to effectively suppress static redundant features. To address this issue, this paper proposes FDA-CLIP (Frame Difference Alpha-CLIP), which is a concise CLIP-based training framework for text-video alignment. Specifically, the method uses frame differences to generate dynamic region masks, which are input into Alpha-CLIP as an additional Alpha channel. This proactively guides the model to focus on semantically critical dynamic regions while suppressing static background redundancy. Experiments demonstrate that frame difference-guided video semantic encoding can effectively balance retrieval efficiency and accuracy.