mlsanityData to 14 Oct 2026

VideoRoPE: What Makes for Good Video Rotary Position Embedding?

arXiv 2502.05173 ↗First posted 7 Feb 2025cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

While Rotary Position Embedding (RoPE) and its variants are widely adopted for their long-context capabilities, the extension of the 1D RoPE to video, with its complex spatio-temporal structure, remains an open challenge. This work first introduces a comprehensive analysis that identifies four key characteristics essential for the effective adaptation of RoPE to video, which have not been fully considered in prior work. As part of our analysis, we introduce a challenging V-NIAH-D (Visual Needle-In-A-Haystack with Distractors) task, which adds periodic distractors into V-NIAH. The V-NIAH-D task demonstrates that previous RoPE variants, lacking appropriate temporal dimension allocation, are easily misled by distractors. Based on our analysis, we introduce VideoRoPE, with a textit3D structure designed to preserve spatio-temporal relationships. VideoRoPE features textitlow-frequency temporal allocation to mitigate periodic oscillations, a textitdiagonal layout to maintain spatial symmetry, and textitadjustable temporal spacing to decouple temporal and spatial indexing. VideoRoPE consistently surpasses previous RoPE variants, across diverse downstream tasks such as long video retrieval, video understanding, and video hallucination. Our code will be available at hrefhttps://github.com/Wiselnn570/VideoRoPEhttps://github.com/Wiselnn570/VideoRoPE.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 58 of its 80 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

58 of 80 references.

References · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  9. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  10. 2024Qwen2 Technical Reportcited by 2,120
  11. 2017The Kinetics Human Action Video Datasetcited by 1,909
  12. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
38 more
  1. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  2. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  3. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  4. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  5. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  6. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  7. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  8. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  9. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  10. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  11. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  12. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  13. 2024Long Context Transfer from Language to Visioncited by 515
  14. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  15. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 478
  16. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  17. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  18. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 322
  19. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
  20. 2024Long-CLIP: Unlocking the Long-Text Capability of CLIPcited by 300
  21. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  22. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 294
  23. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
  24. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  25. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  26. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  27. 2024Streaming Long Video Understanding with Large Language Modelscited by 167
  28. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 166
  29. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 140
  30. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
  31. 2023Scaling Laws of RoPE-based Extrapolationcited by 125
  32. 2023LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Modelscited by 124
  33. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 115
  34. 2024MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMscited by 90
  35. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 90
  36. 2024LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256Kcited by 81
  37. 2024SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Treecited by 80
  38. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Modelcited by 74

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 32; 2026: 33.

65 papers.

Sort:

By area:

  1. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 109
  2. 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 70
  3. 2025MM-IFEngine: Towards Multimodal Instruction Followingcited by 42
  4. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMscited by 40
  5. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learningcited by 38
  6. 2025Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsificationcited by 37
  7. 2025Advancing Complex Video Object Segmentation via Progressive Concept Constructioncited by 26
  8. 2026From Foundation to Application: Improving VLA Models in Practicecited by 26
  9. 2025Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspectivecited by 22
  10. 2025Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generationcited by 22
  11. 2025From Pixels to Words -- Towards Native Vision-Language Primitives at Scalecited by 18
  12. 2025Revisiting Multimodal Positional Encoding in Vision-Language Modelscited by 16
38 more
  1. 2025ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality Debiasingcited by 15
  2. 2026Addressable Memory for Video World Modelscited by 15
  3. 2025Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Modelscited by 14
  4. 2025STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligencecited by 11
  5. 2025SPARK: Synergistic Policy And Reward Co-Evolving Frameworkcited by 10
  6. 2026FASA: Frequency-aware Sparse Attentioncited by 8
  7. 2025Enhancing Temporal Understanding in Video-LLMs through Stacked Temporal Attention in Vision Encoderscited by 6
  8. 2025EchoingPixels: Aliasing-Resistant Joint Token Reduction for Audio-Visual LLMscited by 6
  9. 2025VFRTok: Variable Frame Rates Video Tokenizer with Duration-Proportional Information Assumptioncited by 5
  10. 2025The Unseen Bias: How Norm Discrepancy in Pre-Norm MLLMs Leads to Visual Information Losscited by 5
  11. 2026Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsingcited by 5
  12. 2025HoPE: Hybrid of Position Embedding for Long Context Vision-Language Modelscited by 4
  13. 2025An Empirical Study on How Video-LLMs Answer Video Questionscited by 4
  14. 2025EVA02-AT: Egocentric Video-Language Understanding with Spatial-Temporal Rotary Positional Embeddings and Symmetric Optimizationcited by 3
  15. 2025A Circular Argument : Does RoPE need to be Equivariant for Vision?cited by 3
  16. 2025ClinKD: Cross-Modal Clinical Knowledge Distiller For Multi-Task Medical Imagescited by 2
  17. 2025Infinite Video Understandingcited by 2
  18. 2025Extending Audio Context for Long-Form Understanding in Large Audio-Language Modelscited by 2
  19. 2025Beyond Real: Imaginary Extension of Rotary Position Embeddings for Long-Context LLMscited by 2
  20. 2025Flowception: Temporally Expansive Flow Matching for Video Generationcited by 2
  21. 2026C^2ROPE: Causal Continuous Rotary Positional Encoding for 3D Large Multimodal-Models Reasoningcited by 2
  22. 2026SigVLP: Sigmoid Volume-Language Pre-Training for Self-Supervised CT-Volume Adaptive Representation Learningcited by 2
  23. 2026Anchoring and Rescaling Attention for Semantically Coherent Inbetweeningcited by 2
  24. 2026Adaptive 3D-RoPE: Physics-Aligned Rotary Positional Encoding for Wireless Foundation Modelscited by 2
  25. 2026ZAYA1-VL-8B Technical Reportcited by 2
  26. 2026AnchorEdit: Maintaining Temporal Consistency in Multi-turn Image Editing via Causal Memorycited by 2
  27. 2026nD-RoPE: A Generalized RoPE for n-Dimensional Position Embeddingcited by 2
  28. 2025Task-Aware KV Compression For Cost-Effective Long Video Understandingcited by 1
  29. 2025Head-wise Adaptive Rotary Positional Encoding for Fine-Grained Image Generationcited by 1
  30. 2025PAS: A Training-Free Stabilizer for Temporal Encoding in Video LLMscited by 1
  31. 2026T2SGrid: Temporal-to-Spatial Gridification for Video Temporal Groundingcited by 1
  32. 2026Beyond Sequential Distance: Inter-Modal Distance Invariant Position Encodingcited by 1
  33. 2026Scalable Object Relation Encoding for Better 3D Spatial Reasoning in Large Language Modelscited by 1
  34. 2026Video Analysis and Generation via a Semantic Progress Functioncited by 1
  35. 2026PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documentscited by 1
  36. 2025Minimal Clips, Maximum Salience: Long Video Summarization via Key Moment Extractioncited by 0
  37. 2026PlaneCycle: Training-Free 2D-to-3D Lifting of Foundation Models Without Adapterscited by 0
  38. 2026Diffusion MRI Transformer with a Diffusion Space Rotary Positional Embedding (D-RoPE)cited by 0

50 most cited of 65 All 65 (JSON)