RIG-RoPE: Relation-Stratified Multimodal Attention with Instance-Local Rotary Geometry and Representation-Aware Traversal Coordinates
Not on the map. Why?
Abstract (arXiv)
Multimodal rotary positional encodings apply temporal, height, and width phases to interleaved text, image, and video tokens. This creates two ambiguities: cross-instance spatial displacement depends on preprocessing chart choices unless registration is declared, and scalar advance across visual blocks is often inherited from coordinate extrema rather than defined at the representation level. We introduce RIG-RoPE, combining instance-local rotary geometry, relation-stratified attention, and representation-aware traversal coordinates. RIG-RoPE normalizes relation-homogeneous scores separately, allocates mass with a common H/W-neutral LogSumExp statistic, and uses traversal extent that is additive over ordered slices and sublinear over parallel spatial scale. Text advances by unit increments, image patches are simultaneous, and video accumulates over tokenizer temporal tokens. In a matched, inference-only Qwen2-VL-2B checkpoint experiment, native and RIG text-only paths were exactly equal. RIG was exactly invariant to a whole-chart single-image translation and to translating only the second instance of an unregistered image pair. Native attention remained sensitive to the latter, while an H/W-collapse control confirmed that RIG retained same-instance spatial effects; visual embeddings and all parameters were unchanged. Across three seeds of a frozen tiny task, RIG also had zero clean-to-Gauge logit change, whereas the raw-H/W baseline changed in every seed. Gauge-accuracy differences were +2/72, 0, and 0, failing the preregistered stability gate. These results support the specified activation and Gauge mechanisms, not stable task improvement, universality, or empirical superiority.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 30 references that are in the map. First posted on arXiv 21 May 2026.
Builds on
15 of 30 references.
References · 15
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2025Qwen3-Omni Technical Reportcited by 497
- 2026Qwen3.5-Omni Technical Reportcited by 174
- 2025VideoRoPE: What Makes for Good Video Rotary Position Embedding?cited by 65
- 2024V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encodingcited by 28
- 2025VRoPE: Rotary Position Embedding for Video Large Language Modelscited by 18
- 2025Revisiting Multimodal Positional Encoding in Vision-Language Modelscited by 16
3 more
- 2025Circle-RoPE: Cone-like Decoupled Rotary Positional Embedding for Large Vision-Language Modelscited by 14
- 2026MODIX: A Training-Free Multimodal Information-Driven Positional Index Scaling for Vision-Language Modelscited by 2
- 2026Beyond Sequential Distance: Inter-Modal Distance Invariant Position Encodingcited by 1
Built on by
Not cited yet.