# R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding

arXiv: https://arxiv.org/abs/2404.00801 · HTML: https://mlsanity.com/paper/2404.00801/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 77 references that are in the map.
- First posted on arXiv 31 Mar 2024.

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding](https://mlsanity.com/paper/1604.01753/) (2016)
- [Expanding Language-Image Pretrained Models for General Video Recognition](https://mlsanity.com/paper/2208.02816/) (2022)
- [Prompting Visual-Language Models for Efficient Video Understanding](https://mlsanity.com/paper/2112.04478/) (2021)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning](https://mlsanity.com/paper/2206.06522/) (2022)
- [FractalNet: Ultra-Deep Neural Networks without Residuals](https://mlsanity.com/paper/1605.07648/) (2016)
- [TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval](https://mlsanity.com/paper/2001.09099/) (2020)
- [ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning](https://mlsanity.com/paper/2206.13559/) (2022)
- [UniVTG: Towards Unified Video-Language Temporal Grounding](https://mlsanity.com/paper/2307.16715/) (2023)
- [Frozen CLIP Models are Efficient Video Learners](https://mlsanity.com/paper/2208.03550/) (2022)
- [Fine-tuned CLIP Models are Efficient Video Learners](https://mlsanity.com/paper/2212.03640/) (2022)

## Built on by (most cited first, 30 of 34)

- [E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding](https://mlsanity.com/paper/2409.18111/) (2024)
- [VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning](https://mlsanity.com/paper/2503.13444/) (2025)
- [FlashVTG: Feature Layering and Adaptive Score Handling Network for Video Temporal Grounding](https://mlsanity.com/paper/2412.13441/) (2024)
- [Saliency-Guided DETR for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2410.01615/) (2024)
- [Moment of Untruth: Dealing with Negative Queries in Video Moment Retrieval](https://mlsanity.com/paper/2502.08544/) (2025)
- [LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2501.10787/) (2025)
- [TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2512.03963/) (2025)
- [VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotations](https://mlsanity.com/paper/2510.23397/) (2025)
- [Moment Quantization for Video Temporal Grounding](https://mlsanity.com/paper/2504.02286/) (2025)
- [MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering](https://mlsanity.com/paper/2506.18071/) (2025)
- [Sim-DETR: Unlock DETR for Temporal Sentence Grounding](https://mlsanity.com/paper/2509.23867/) (2025)
- [Empower Words: DualGround for Structured Phrase and Sentence-Level Temporal Grounding](https://mlsanity.com/paper/2510.20244/) (2025)
- [VideoLights: Feature Refinement and Cross-Task Alignment Transformer for Joint Video Highlight Detection and Moment Retrieval](https://mlsanity.com/paper/2412.01558/) (2024)
- [CFSum: A Transformer-Based Multi-Modal Video Summarization Framework With Coarse-Fine Fusion](https://mlsanity.com/paper/2503.00364/) (2025)
- [TimeLoc: A Unified End-to-End Framework for Precise Timestamp Localization in Long Videos](https://mlsanity.com/paper/2503.06526/) (2025)
- [SVAG-Bench: A Large-Scale Benchmark for Multi-Instance Spatio-temporal Video Action Grounding](https://mlsanity.com/paper/2510.13016/) (2025)
- [When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions](https://mlsanity.com/paper/2510.17218/) (2025)
- [ActPrompt: In-Domain Feature Adaptation via Action Cues for Video Temporal Grounding](https://mlsanity.com/paper/2408.06622/) (2024)
- [Sparse-Dense Side-Tuner for efficient Video Temporal Grounding](https://mlsanity.com/paper/2507.07744/) (2025)
- [Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2510.10671/) (2025)
- [Grounding Everything in Tokens for Multimodal Large Language Models](https://mlsanity.com/paper/2512.10554/) (2025)
- [MomentMix Augmentation with Length-Aware DETR for Temporally Robust Moment Retrieval](https://mlsanity.com/paper/2412.20816/) (2024)
- [Augmenting Moment Retrieval: Zero-Dependency Two-Stage Learning](https://mlsanity.com/paper/2510.19622/) (2025)
- [SMART: Shot-Aware Multimodal Video Moment Retrieval with Audio-Enhanced MLLM](https://mlsanity.com/paper/2511.14143/) (2025)
- [CoSTL: Comprehensive Spatial-Temporal Representation Learning for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2606.01149/) (2026)
- [D&M: Enriching E-commerce Videos with Sound Effects by Key Moment Detection and SFX Matching](https://mlsanity.com/paper/2408.13226/) (2024)
- [OVG-HQ: Online Video Grounding with Hybrid-modal Queries](https://mlsanity.com/paper/2508.11903/) (2025)
- [HERO: Hierarchical Embedding-Refinement for Open-Vocabulary Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/2603.06732/) (2026)
- [ClipTBP: Clip-Pair based Temporal Boundary Prediction with Boundary-Aware Learning for Moment Retrieval](https://mlsanity.com/paper/2604.27591/) (2026)
- [Static and Dynamic Graph Alignment Network for Temporal Video Grounding](https://mlsanity.com/paper/2605.00684/) (2026)

## Abstract (arXiv)

Video temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries. Most existing VTG models are built upon frame-wise final-layer CLIP features, aided by additional temporal backbones (e.g., SlowFast) with sophisticated temporal reasoning mechanisms. In this work, we claim that CLIP itself already shows great potential for fine-grained spatial-temporal modeling, as each layer offers distinct yet useful information under different granularity levels. Motivated by this, we propose Reversed Recurrent Tuning (R2-Tuning), a parameter- and memory-efficient transfer learning framework for video temporal grounding. Our method learns a lightweight R2 Block containing only 1.5% of the total parameters to perform progressive spatial-temporal modeling. Starting from the last layer of CLIP, R2 Block recurrently aggregates spatial features from earlier layers, then refines temporal correlation conditioning on the given query, resulting in a coarse-to-fine scheme. R2-Tuning achieves state-of-the-art performance across three VTG tasks (i.e., moment retrieval, highlight detection, and video summarization) on six public benchmarks (i.e., QVHighlights, Charades-STA, Ego4D-NLQ, TACoS, YouTube Highlights, and TVSum) even without the additional backbone, demonstrating the significance and effectiveness of the proposed scheme. Our code is available at https://github.com/yeliudev/R2-Tuning.