mlsanityData to 14 Oct 2026

ExtraVAR: Stage-Aware RoPE Remapping for Resolution Extrapolation in Visual Autoregressive Models

arXiv 2605.10045 ↗First posted 11 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Visual Autoregressive (VAR) models have emerged as a strong alternative to diffusion for image synthesis, yet their fixed training resolution prevents direct generation at higher resolutions. Naively transferring training-free extrapolation methods from LLMs or diffusion models to VAR yields three characteristic failure modes: global repetition, local repetition, and detail degradation. We trace them to a unified band-stage mismatch: VAR generates images in a coarse-to-fine, scale-wise process where each stage is driven by a distinct dominant RoPE frequency band, and each failure mode emerges when the dominant band of a particular stage is disrupted. Building on this insight, we propose Stage-Aware RoPE Remapping, a training-free strategy that assigns each frequency band a stage-specific remapping rule, jointly suppressing all three failure modes. We further observe that attention becomes systematically dispersed as the image resolution increases. Existing methods typically depend on predefined attention scaling factors, which are neither adaptive to the target resolution nor capable of faithfully capturing the actual extent of attention dispersion. We therefore propose Entropy-Driven Adaptive Attention Calibration, which quantifies dispersion via a resolution-invariant normalized entropy and yields a closed-form per-head scaling factor that realigns the extrapolated-resolution attention entropy with its training-resolution counterpart. Extensive experiments show that our method consistently outperforms prior resolution-extrapolation methods in both structural coherence and fine-detail fidelity. Our code is available at https://github.com/feihongyan1/ExtraVAR.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 53 of its 54 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

53 of 54 references.

References · 53

  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  3. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  4. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  5. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  6. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  7. 2016Pixel Recurrent Neural Networkscited by 816
  8. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  9. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  10. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  11. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  12. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
38 more
  1. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  2. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  3. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 422
  4. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  5. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
  6. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 327
  7. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  8. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 132
  9. 2023ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Modelscited by 128
  10. 2023SyncDiffusion: Coherent Montage via Synchronized Joint Diffusionscited by 110
  11. 2023Relay Diffusion: Unifying diffusion process across resolutions for image synthesiscited by 79
  12. 2024Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encodingcited by 72
  13. 2025RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformerscited by 66
  14. 2024UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peakscited by 65
  15. 2025Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Modelscited by 65
  16. 2024FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesiscited by 59
  17. 2023Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesiscited by 56
  18. 2024Make a Cheap Scaling: A Self-Cascade Diffusion Model for Higher-Resolution Adaptationcited by 51
  19. 2025Scalable-Softmax Is Superior for Attentioncited by 45
  20. 2023ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separationcited by 37
  21. 2023Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Imagescited by 36
  22. 2024DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidancecited by 34
  23. 2025FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruningcited by 34
  24. 2024FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free Scale Fusioncited by 30
  25. 2023HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Modelscited by 29
  26. 2024I-Max: Maximize the Resolution Potential of Pre-trained Rectified Flow Transformers with Projected Flowcited by 23
  27. 2025HiFlow: Training-free High-Resolution Image Generation with Flow-Aligned Guidancecited by 22
  28. 2024Extending LLMs' Context Window with 100 Samplescited by 21
  29. 2025Critical attention scaling in long-context transformerscited by 20
  30. 2024FreCaS: Efficient Higher-Resolution Image Generation via Frequency-aware Cascaded Samplingcited by 16
  31. 2024Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspectivecited by 15
  32. 2025Native-Resolution Image Synthesiscited by 15
  33. 2025DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusioncited by 14
  34. 2025Ultra-Resolution Adaptation with Easecited by 13
  35. 2025UltraImage: Rethinking Resolution Extrapolation in Image Diffusion Transformerscited by 12
  36. 2025Entropy Rectifying Guidance for Diffusion and Flow Modelscited by 11
  37. 2025FreePCA: Integrating Consistency Information across Long-short Frames in Training-free Long Video Generation via Principal Component Analysiscited by 8
  38. 2025ScaleDiff: Higher-Resolution Image Synthesis via Efficient and Model-Agnostic Diffusioncited by 6

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Scale-Wise VAR is Secretly Discrete Diffusion
  2. 2025RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restoration
  3. 2026SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers
  4. 2023Fast Sampling Through The Reuse Of Attention Maps In Diffusion Models
  5. 2026Drift-AR: Single-Step Visual Autoregressive Generation via Anti-Symmetric Drifting
  6. 2025Learning to Expand Images for Efficient Visual Autoregressive Modeling
  7. 2024LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantization
  8. 2025Efficient Conditional Generation on Scale-based Visual Autoregressive Models