# EditAR: Unified Conditional Generation with Autoregressive Models

arXiv: https://arxiv.org/abs/2501.04699 · HTML: https://mlsanity.com/paper/2501.04699/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 76 references that are in the map.
- First posted on arXiv 8 Jan 2025.

## Builds on (most cited first, 30 of 69)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)

## Built on by (most cited first, 30 of 40)

- [Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning](https://mlsanity.com/paper/2505.07538/) (2025)
- [Visual Autoregressive Modeling for Instruction-Guided Image Editing](https://mlsanity.com/paper/2508.15772/) (2025)
- [Fine-Tuning Visual Autoregressive Models for Subject-Driven Generation](https://mlsanity.com/paper/2504.02612/) (2025)
- [EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editing](https://mlsanity.com/paper/2512.11715/) (2025)
- [Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editing](https://mlsanity.com/paper/2504.10434/) (2025)
- [Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning](https://mlsanity.com/paper/2506.01480/) (2025)
- [PixelPonder: Dynamic Patch Adaptation for Enhanced Multi-Conditional Text-to-Image Generation](https://mlsanity.com/paper/2503.06684/) (2025)
- [Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editing](https://mlsanity.com/paper/2509.01984/) (2025)
- [Semantic Context Matters: Improving Conditioning for Autoregressive Models](https://mlsanity.com/paper/2511.14063/) (2025)
- [ICAS: Detecting Training Data from Autoregressive Image Generative Models](https://mlsanity.com/paper/2507.05068/) (2025)
- [The Promise of RL for Autoregressive Image Editing](https://mlsanity.com/paper/2508.01119/) (2025)
- [LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation](https://mlsanity.com/paper/2604.11789/) (2026)
- [Context-Aware Autoregressive Models for Multi-Conditional Image Generation](https://mlsanity.com/paper/2505.12274/) (2025)
- [NEP: Autoregressive Image Editing via Next Editing Token Prediction](https://mlsanity.com/paper/2508.06044/) (2025)
- [MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cues](https://mlsanity.com/paper/2512.03046/) (2025)
- [MRI Contrast Enhancement Kinetics World Model](https://mlsanity.com/paper/2602.19285/) (2026)
- [OmniGen-AR: AutoRegressive Any-to-Image Generation](https://mlsanity.com/paper/2606.09156/) (2026)
- [Layout-Conditioned Autoregressive Text-to-Image Generation via Structured Masking](https://mlsanity.com/paper/2509.12046/) (2025)
- [Autoregressive Styled Text Image Generation, but Make it Reliable](https://mlsanity.com/paper/2510.23240/) (2025)
- [Referring Layer Decomposition](https://mlsanity.com/paper/2602.19358/) (2026)
- [Masked Generative Transformer Is What You Need for Image Editing](https://mlsanity.com/paper/2605.10859/) (2026)
- [EditTrack: Detecting and Attributing AI-assisted Image Editing](https://mlsanity.com/paper/2510.01173/) (2025)
- [Your VAR Model is Secretly an Efficient and Explainable Generative Classifier](https://mlsanity.com/paper/2510.12060/) (2025)
- [Progress by Pieces: Test-Time Scaling for Autoregressive Image Generation](https://mlsanity.com/paper/2511.21185/) (2025)
- [EditGarment: An Instruction-Based Garment Editing Dataset Constructed with Automated MLLM Synthesis and Semantic-Aware Evaluation](https://mlsanity.com/paper/2508.03497/) (2025)
- [DCoAR: Deep Concept Injection into Unified Autoregressive Models for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2508.07341/) (2025)
- [SSD: Spatial-Semantic Head Decoupling for Efficient Autoregressive Image Generation](https://mlsanity.com/paper/2510.18716/) (2025)
- [Towards Generalized Multi-Image Editing for Unified Multimodal Models](https://mlsanity.com/paper/2601.05572/) (2026)
- [Rethinking Structure Preservation in Text-Guided Image Editing with Visual Autoregressive Models](https://mlsanity.com/paper/2603.28367/) (2026)
- [TokenAR: Multiple Subject Generation via Autoregressive Token-level enhancement](https://mlsanity.com/paper/2510.16332/) (2025)

## Abstract (arXiv)

Recent progress in controllable image generation and editing is largely driven by diffusion-based methods. Although diffusion models perform exceptionally well in specific tasks with tailored designs, establishing a unified model is still challenging. In contrast, autoregressive models inherently feature a unified tokenized representation, which simplifies the creation of a single foundational model for various tasks. In this work, we propose EditAR, a single unified autoregressive framework for a variety of conditional image generation tasks, e.g., image editing, depth-to-image, edge-to-image, segmentation-to-image. The model takes both images and instructions as inputs, and predicts the edited images tokens in a vanilla next-token paradigm. To enhance the text-to-image alignment, we further propose to distill the knowledge from foundation models into the autoregressive modeling process. We evaluate its effectiveness across diverse tasks on established benchmarks, showing competitive performance to various state-of-the-art task-specific methods. Project page: https://jitengmu.github.io/EditAR/