Rethinking Structure Preservation in Text-Guided Image Editing with Visual Autoregressive Models
Not on the map. Why?
Abstract (arXiv)
Visual autoregressive (VAR) models have recently emerged as a promising family of generative models, enabling a wide range of downstream vision tasks such as text-guided image editing. By shifting the editing paradigm from noise manipulation in diffusion-based methods to token-level operations, VAR-based approaches achieve better background preservation and significantly faster inference. However, existing VAR-based editing methods still face two key challenges: accurately localizing editable tokens and maintaining structural consistency in the edited results. In this work, we propose a novel text-guided image editing framework rooted in an analysis of intermediate feature distributions within VAR models. First, we introduce a coarse-to-fine token localization strategy that can refine editable regions, balancing editing fidelity and background preservation. Second, we analyze the intermediate representations of VAR models and identify structure-related features, by which we design a simple yet effective feature injection mechanism to enhance structural consistency between the edited and source images. Third, we develop a reinforcement learning-based adaptive feature injection scheme that automatically learns scale- and layer-specific injection ratios to jointly optimize editing fidelity and structure preservation. Extensive experiments demonstrate that our method achieves superior structural consistency and editing quality compared with state-of-the-art approaches, across both local and global editing scenarios.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 45 of its 55 references that are in the map. First posted on arXiv 30 Mar 2026.
Builds on
45 of 55 references.
References · 45
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
33 more
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
- 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
- 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
- 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
- 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
- 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
- 2023Zero-shot Image-to-Image Translationcited by 505
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
- 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 256
- 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
- 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 196
- 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editingcited by 183
- 2024Semantic Image Inversion and Editing using Rectified Stochastic Differential Equationscited by 165
- 2023LEDITS++: Limitless Image Editing using Text-to-Image Modelscited by 148
- 2024Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editingcited by 135
- 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 132
- 2024DiT4Edit: Diffusion Transformer for Image Editingcited by 101
- 2023Prompt Tuning Inversion for Text-Driven Image Editing Using Diffusion Modelscited by 99
- 2024Get What You Want, Not What You Don't: Image Content Suppression for Text-to-Image Diffusion Modelscited by 63
- 2025EditAR: Unified Conditional Generation with Autoregressive Modelscited by 40
- 2025Visual Autoregressive Modeling for Instruction-Guided Image Editingcited by 22
- 2025Training-Free Text-Guided Image Editing with Visual Autoregressive Modelcited by 21
- 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generationcited by 13
- 2024LocInv: Localization-aware Inversion for Text-Guided Image Editingcited by 11
- 2025Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editingcited by 10
- 2025Consistent Image Layout Editing with Diffusion Modelscited by 9
- 2025Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editingcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing
- 2025The Promise of RL for Autoregressive Image Editing
- 2026Prompt-Guided Image Editing with Masked Logit Nudging in Visual Autoregressive Models
- 2025Semantic Context Matters: Improving Conditioning for Autoregressive Models
- 2026EditTransfer++: Toward Faithful and Efficient Visual-Prompt-Guided Image Editing
- 2025NEP: Autoregressive Image Editing via Next Editing Token Prediction
- 2026ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations
- 2026Diffusion Image Editing via Asynchronous Token Decoding