# VideoRoPE: What Makes for Good Video Rotary Position Embedding?

arXiv: https://arxiv.org/abs/2502.05173 · HTML: https://mlsanity.com/paper/2502.05173/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 80 references that are in the map.
- First posted on arXiv 7 Feb 2025.

## Builds on (most cited first, 30 of 58)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding](https://mlsanity.com/paper/2109.14084/) (2021)
- [YaRN: Efficient Context Window Extension of Large Language Models](https://mlsanity.com/paper/2309.00071/) (2023)
- [LLaVA-Video: Video Instruction Tuning With Synthetic Data](https://mlsanity.com/paper/2410.02713/) (2024)
- [LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models](https://mlsanity.com/paper/2311.17043/) (2023)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling](https://mlsanity.com/paper/2102.06183/) (2021)
- [OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation](https://mlsanity.com/paper/2311.17911/) (2023)
- [ShareGPT4Video: Improving Video Understanding and Generation with Better Captions](https://mlsanity.com/paper/2406.04325/) (2024)
- [Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding](https://mlsanity.com/paper/2311.08046/) (2023)
- [PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning](https://mlsanity.com/paper/2404.16994/) (2024)

## Built on by (most cited first, 30 of 65)

- [Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generation](https://mlsanity.com/paper/2505.05472/) (2025)
- [SIM-CoT: Supervised Implicit Chain-of-Thought](https://mlsanity.com/paper/2509.20317/) (2025)
- [MM-IFEngine: Towards Multimodal Instruction Following](https://mlsanity.com/paper/2504.07957/) (2025)
- [LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs](https://mlsanity.com/paper/2506.14429/) (2025)
- [CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning](https://mlsanity.com/paper/2509.22647/) (2025)
- [Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsification](https://mlsanity.com/paper/2506.19225/) (2025)
- [Advancing Complex Video Object Segmentation via Progressive Concept Construction](https://mlsanity.com/paper/2507.15852/) (2025)
- [From Foundation to Application: Improving VLA Models in Practice](https://mlsanity.com/paper/2607.06403/) (2026)
- [Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspective](https://mlsanity.com/paper/2507.08801/) (2025)
- [Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generation](https://mlsanity.com/paper/2510.24821/) (2025)
- [From Pixels to Words -- Towards Native Vision-Language Primitives at Scale](https://mlsanity.com/paper/2510.14979/) (2025)
- [Revisiting Multimodal Positional Encoding in Vision-Language Models](https://mlsanity.com/paper/2510.23095/) (2025)
- [ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality Debiasing](https://mlsanity.com/paper/2506.19848/) (2025)
- [Addressable Memory for Video World Models](https://mlsanity.com/paper/2608.07408/) (2026)
- [Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Models](https://mlsanity.com/paper/2505.16416/) (2025)
- [STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence](https://mlsanity.com/paper/2510.24693/) (2025)
- [SPARK: Synergistic Policy And Reward Co-Evolving Framework](https://mlsanity.com/paper/2509.22624/) (2025)
- [FASA: Frequency-aware Sparse Attention](https://mlsanity.com/paper/2602.03152/) (2026)
- [Enhancing Temporal Understanding in Video-LLMs through Stacked Temporal Attention in Vision Encoders](https://mlsanity.com/paper/2510.26027/) (2025)
- [EchoingPixels: Aliasing-Resistant Joint Token Reduction for Audio-Visual LLMs](https://mlsanity.com/paper/2512.10324/) (2025)
- [VFRTok: Variable Frame Rates Video Tokenizer with Duration-Proportional Information Assumption](https://mlsanity.com/paper/2505.12053/) (2025)
- [The Unseen Bias: How Norm Discrepancy in Pre-Norm MLLMs Leads to Visual Information Loss](https://mlsanity.com/paper/2512.08374/) (2025)
- [Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing](https://mlsanity.com/paper/2602.16455/) (2026)
- [HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models](https://mlsanity.com/paper/2505.20444/) (2025)
- [An Empirical Study on How Video-LLMs Answer Video Questions](https://mlsanity.com/paper/2508.15360/) (2025)
- [EVA02-AT: Egocentric Video-Language Understanding with Spatial-Temporal Rotary Positional Embeddings and Symmetric Optimization](https://mlsanity.com/paper/2506.14356/) (2025)
- [A Circular Argument : Does RoPE need to be Equivariant for Vision?](https://mlsanity.com/paper/2511.08368/) (2025)
- [ClinKD: Cross-Modal Clinical Knowledge Distiller For Multi-Task Medical Images](https://mlsanity.com/paper/2502.05928/) (2025)
- [Infinite Video Understanding](https://mlsanity.com/paper/2507.09068/) (2025)
- [Extending Audio Context for Long-Form Understanding in Large Audio-Language Models](https://mlsanity.com/paper/2510.15231/) (2025)

## Abstract (arXiv)

While Rotary Position Embedding (RoPE) and its variants are widely adopted for their long-context capabilities, the extension of the 1D RoPE to video, with its complex spatio-temporal structure, remains an open challenge. This work first introduces a comprehensive analysis that identifies four key characteristics essential for the effective adaptation of RoPE to video, which have not been fully considered in prior work. As part of our analysis, we introduce a challenging V-NIAH-D (Visual Needle-In-A-Haystack with Distractors) task, which adds periodic distractors into V-NIAH. The V-NIAH-D task demonstrates that previous RoPE variants, lacking appropriate temporal dimension allocation, are easily misled by distractors. Based on our analysis, we introduce VideoRoPE, with a textit3D structure designed to preserve spatio-temporal relationships. VideoRoPE features textitlow-frequency temporal allocation to mitigate periodic oscillations, a textitdiagonal layout to maintain spatial symmetry, and textitadjustable temporal spacing to decouple temporal and spatial indexing. VideoRoPE consistently surpasses previous RoPE variants, across diverse downstream tasks such as long video retrieval, video understanding, and video hallucination. Our code will be available at hrefhttps://github.com/Wiselnn570/VideoRoPEhttps://github.com/Wiselnn570/VideoRoPE.