mlsanityData to 14 Oct 2026

ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Models

arXiv 2505.21465 ↗First posted 27 May 2025cs.CV cs.CLcited by 2

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Currently, a prevalent approach for enhancing Vision-Language Models (VLMs) performance is to encode both the high-resolution version and the thumbnail of an image simultaneously. While effective, this method generates a large number of image tokens. When combined with the widely used Rotary Position Embedding (RoPE), its long-term decay property hinders the interaction between high-resolution tokens and thumbnail tokens, as well as between text and image. To address these issues, we propose ID-Align, which alleviates these problems by reordering position IDs. In this method, high-resolution tokens inherit IDs from their corresponding thumbnail token while constraining the overexpansion of positional indices. Our experiments conducted within the LLaVA-Next framework demonstrate that ID-Align achieves significant improvements, including a 6.09% enhancement on MMBench's relation reasoning tasks and notable gains across multiple benchmarks. Our code is available at the following link: https://github.com/zooblastlbz/ID-Align.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 50 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

35 of 50 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encoding19 Jan
  2. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding13 Dec
  3. 2024V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding12 Dec
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  6. 2024NVLM: Open Frontier-Class Multimodal LLMs17 Sep
  7. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspective26 May
  8. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites25 Apr
  9. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension25 Apr
  10. 2023Honeybee: Locality-enhanced Projector for Multimodal LLM11 Dec
  11. 2023Improved Baselines with Visual Instruction Tuning5 Oct

Other references · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  9. 2016Pointer Sentinel Mixture Modelscited by 3,096
  10. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  12. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
12 more
  1. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  2. 2018Self-Attention with Relative Position Representationscited by 1,120
  3. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  4. 2016A Diagram Is Worth A Dozen Imagescited by 804
  5. 2017Convolutional Sequence to Sequence Learningcited by 552
  6. 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
  7. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 115
  8. 2020Learning to Encode Position for Transformer with Continuous Dynamical Modelcited by 61
  9. 2019Encoding word order in complex embeddingscited by 53
  10. 2024Base of RoPE Bounds Context Lengthcited by 52
  11. 2024Mitigating Object Hallucination via Concentric Causal Attentioncited by 49
  12. 2024SynthVLM: Towards High-Quality and Efficient Synthesis of Image-Caption Datasets for Vision-Language Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025The Unseen Bias: How Norm Discrepancy in Pre-Norm MLLMs Leads to Visual Information Losscited by 5
  2. 2026Human-Centric Image Captioning with Subject-Centered Spatial Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Revisiting Multimodal Positional Encoding in Vision-Language Models
  2. 2025Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Models
  3. 2026Same Semantics, Different Paths: Self-Improving Alignment for Vision-Text Compression
  4. 2025TinyAlign: Boosting Lightweight Vision-Language Models by Mitigating Modal Alignment Bottlenecks
  5. 2025Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization
  6. 2025Delta-LLaVA: Base-then-Specialize Alignment for Token-Efficient Vision-Language Models
  7. 2025HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models
  8. 2025OMEGA: Optimized Multimodal Position Encoding Index Derivation with Global Adaptive Scaling for Vision-Language Models