# HVD: Human Vision-Driven Video Representation Learning for Text-Video Retrieval

arXiv: https://arxiv.org/abs/2601.16155 · HTML: https://mlsanity.com/paper/2601.16155/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 22 Jan 2026.

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [MACE: Mass Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2403.06135/) (2024)
- [TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition](https://mlsanity.com/paper/2307.12493/) (2023)
- [FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving](https://mlsanity.com/paper/2505.17685/) (2025)
- [X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval](https://mlsanity.com/paper/2203.15086/) (2022)
- [Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances](https://mlsanity.com/paper/2410.18775/) (2024)
- [CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment](https://mlsanity.com/paper/2209.06430/) (2022)
- [Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations](https://mlsanity.com/paper/2211.11427/) (2022)
- [UATVR: Uncertainty-Adaptive Text-Video Retrieval](https://mlsanity.com/paper/2301.06309/) (2023)
- [Unified Coarse-to-Fine Alignment for Video-Text Retrieval](https://mlsanity.com/paper/2309.10091/) (2023)
- [Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning](https://mlsanity.com/paper/2303.14369/) (2023)
- [Chat-Driven Text Generation and Interaction for Person Retrieval](https://mlsanity.com/paper/2509.12662/) (2025)
- [Text Is MASS: Modeling as Stochastic Embedding for Text-Video Retrieval](https://mlsanity.com/paper/2403.17998/) (2024)
- [Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment](https://mlsanity.com/paper/2305.12218/) (2023)
- [Delving Deeper: Hierarchical Visual Perception for Robust Video-Text Retrieval](https://mlsanity.com/paper/2601.12768/) (2026)
- [Multimodal Multi-Agent Empowered Legal Judgment Prediction](https://mlsanity.com/paper/2601.12815/) (2026)
- [Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learning](https://mlsanity.com/paper/2401.00701/) (2024)
- [Dynamic Uncertainty Learning with Noisy Correspondence for Text-Based Person Search](https://mlsanity.com/paper/2505.06566/) (2025)
- [From Production Envelopes to Executable Schedules: Sound Constructive Refinement for High-Mix Manufacturing](https://mlsanity.com/paper/2512.10358/) (2025)

## Built on by (most cited first, 14 of 14)

- [Air-Know: Arbiter-Calibrated Knowledge-Internalizing Robust Network for Composed Image Retrieval](https://mlsanity.com/paper/2604.19386/) (2026)
- [ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrieval](https://mlsanity.com/paper/2604.20358/) (2026)
- [CONQUER: Context-Aware Representation with Query Enhancement for Text-Based Person Search](https://mlsanity.com/paper/2601.18625/) (2026)
- [Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance](https://mlsanity.com/paper/2606.00305/) (2026)
- [Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation](https://mlsanity.com/paper/2605.09253/) (2026)
- [Recursive Self-Evolving Agents via Held-Out Selection](https://mlsanity.com/paper/2606.28374/) (2026)
- [Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models](https://mlsanity.com/paper/2602.19926/) (2026)
- [BlackMirror: Black-Box Backdoor Detection for Text-to-Image Models via Instruction-Response Deviation](https://mlsanity.com/paper/2603.05921/) (2026)
- [Mitigating Hallucination on Hallucination in RAG via Ensemble Voting](https://mlsanity.com/paper/2603.27253/) (2026)
- [Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs](https://mlsanity.com/paper/2603.29736/) (2026)
- [When Denoising Becomes Unsigning: Theoretical and Empirical Analysis of Watermark Fragility Under Diffusion-Based Image Editing](https://mlsanity.com/paper/2603.04696/) (2026)
- [Editing Away the Evidence: Diffusion-Based Image Manipulation and the Failure Modes of Robust Watermarking](https://mlsanity.com/paper/2603.12949/) (2026)
- [On the Controllability-Fidelity Frontier in Diffusion Editing](https://mlsanity.com/paper/2606.09901/) (2026)
- [VigilFormer: Deformable Attention for Video Anomaly Detection with Causal Risk Inference](https://mlsanity.com/paper/2606.14724/) (2026)

## Abstract (arXiv)

The success of CLIP has driven substantial progress in text-video retrieval. However, current methods often suffer from "blind" feature interaction, where the model struggles to discern key visual information from background noise due to the sparsity of textual queries. To bridge this gap, we draw inspiration from human cognitive behavior and propose the Human Vision-Driven (HVD) model. Our framework establishes a coarse-to-fine alignment mechanism comprising two key components: the Frame Features Selection Module (FFSM) and the Patch Features Compression Module (PFCM). FFSM mimics the human macro-perception ability by selecting key frames to eliminate temporal redundancy. Subsequently, PFCM simulates micro-perception by aggregating patch features into salient visual entities through an advanced attention mechanism, enabling precise entity-level matching. Extensive experiments on five benchmarks demonstrate that HVD not only captures human-like visual focus but also achieves state-of-the-art performance.