mlsanityData to 7 Oct 2026

EditAR: Unified Conditional Generation with Autoregressive Models

arXiv 2501.04699 ↗First posted 8 Jan 2025cs.CVcited by 40

Not on the map. Why?

Abstract (arXiv)

Recent progress in controllable image generation and editing is largely driven by diffusion-based methods. Although diffusion models perform exceptionally well in specific tasks with tailored designs, establishing a unified model is still challenging. In contrast, autoregressive models inherently feature a unified tokenized representation, which simplifies the creation of a single foundational model for various tasks. In this work, we propose EditAR, a single unified autoregressive framework for a variety of conditional image generation tasks, e.g., image editing, depth-to-image, edge-to-image, segmentation-to-image. The model takes both images and instructions as inputs, and predicts the edited images tokens in a vanilla next-token paradigm. To enhance the text-to-image alignment, we further propose to distill the knowledge from foundation models into the autoregressive modeling process. We evaluate its effectiveness across diverse tasks on established benchmarks, showing competitive performance to various state-of-the-art task-specific methods. Project page: https://jitengmu.github.io/EditAR/

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 69 of its 76 references that are in the map. First posted on arXiv 8 Jan 2025.

Builds on

69 of 76 references.

References · 69

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2020Denoising Diffusion Implicit Modelscited by 9,655
  11. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  12. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
38 more
  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  13. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  14. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  15. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  16. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  17. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  18. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  19. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  20. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  21. 2023Consistency Modelscited by 1,778
  22. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  23. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  24. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  25. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  26. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  27. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  28. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  29. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  30. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  31. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  32. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  33. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  34. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  35. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  36. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  37. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  38. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 23; 2026: 17.

40 papers.

Sort:

By area:

  1. 2025Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoningcited by 31
  2. 2025Visual Autoregressive Modeling for Instruction-Guided Image Editingcited by 21
  3. 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generationcited by 13
  4. 2025EditMGT: Unleashing Potentials of Masked Generative Transformers in Image Editingcited by 12
  5. 2025Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editingcited by 10
  6. 2025Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learningcited by 8
  7. 2025PixelPonder: Dynamic Patch Adaptation for Enhanced Multi-Conditional Text-to-Image Generationcited by 7
  8. 2025Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editingcited by 7
  9. 2025Semantic Context Matters: Improving Conditioning for Autoregressive Modelscited by 7
  10. 2025ICAS: Detecting Training Data from Autoregressive Image Generative Modelscited by 6
  11. 2025The Promise of RL for Autoregressive Image Editingcited by 6
  12. 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generationcited by 6
28 more
  1. 2025Context-Aware Autoregressive Models for Multi-Conditional Image Generationcited by 5
  2. 2025NEP: Autoregressive Image Editing via Next Editing Token Predictioncited by 5
  3. 2025MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cuescited by 4
  4. 2026MRI Contrast Enhancement Kinetics World Modelcited by 4
  5. 2026OmniGen-AR: AutoRegressive Any-to-Image Generationcited by 4
  6. 2025Layout-Conditioned Autoregressive Text-to-Image Generation via Structured Maskingcited by 3
  7. 2025Autoregressive Styled Text Image Generation, but Make it Reliablecited by 3
  8. 2026Referring Layer Decompositioncited by 3
  9. 2026Masked Generative Transformer Is What You Need for Image Editingcited by 3
  10. 2025EditTrack: Detecting and Attributing AI-assisted Image Editingcited by 2
  11. 2025Your VAR Model is Secretly an Efficient and Explainable Generative Classifiercited by 2
  12. 2025Progress by Pieces: Test-Time Scaling for Autoregressive Image Generationcited by 2
  13. 2025EditGarment: An Instruction-Based Garment Editing Dataset Constructed with Automated MLLM Synthesis and Semantic-Aware Evaluationcited by 1
  14. 2025DCoAR: Deep Concept Injection into Unified Autoregressive Models for Personalized Text-to-Image Generationcited by 1
  15. 2025SSD: Spatial-Semantic Head Decoupling for Efficient Autoregressive Image Generationcited by 1
  16. 2026Towards Generalized Multi-Image Editing for Unified Multimodal Modelscited by 1
  17. 2026Rethinking Structure Preservation in Text-Guided Image Editing with Visual Autoregressive Modelscited by 1
  18. 2025TokenAR: Multiple Subject Generation via Autoregressive Token-level enhancementcited by 0
  19. 2026InfoTok: Information-Theoretic Regularization for Capacity-Constrained Shared Visual Tokenization in Unified MLLMscited by 0
  20. 2026TerraDiT: Point-Conditioned Diffusion Transformer for Satellite Image Synthesiscited by 0
  21. 2026UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMscited by 0
  22. 2026Imagine Before You Draw: Visual Prompt Engineering for Image Generationcited by 0
  23. 2026Edit the Bits, Diff the Codes: Bitwise Residual Editing for Visual Autoregressive Modelscited by 0
  24. 2026Next-Dense-Stride Prediction for Multimodal Autoregressive Visual Modelingcited by 0
  25. 2026Improving Sample Diversity in Autoregressive Text-to-Image Generation via Cluster Truncationcited by 0
  26. 2026ARDepth: Auto-regressive Monocular Depth Estimation with Progressive Visual Conditioningcited by 0
  27. 2026Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applicationscited by 0
  28. 2026Attribute Token Arithmetic: Disentangled and Continuous Semantic Control for Visual Autoregressive Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale
  2. 2025UniVG: A Generalist Diffusion Model for Unified Image Generation and Editing
  3. 2025Nexus-Gen: Unified Image Understanding, Generation, and Editing via Prefilled Autoregression in Shared Embedding Space
  4. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  5. 2022UniTune: Text-Driven Image Editing by Fine Tuning a Diffusion Model on a Single Image
  6. 2024Diffusion Model-Based Image Editing: A Survey
  7. 2025SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RL
  8. 2024ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformer