mlsanityData to 14 Oct 2026

C2P-VAR: Continual and Compositional Personalization in Visual Autoregressive Models

arXiv 2605.19750 ↗First posted 19 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Visual autoregressive (VAR) models have recently emerged as an efficient paradigm for text-to-image generation, yet their personalization capabilities remain largely limited to static, single-concept settings. In practice, users may continuously introduce new concepts and wish to compose multiple personalized concepts within a single image. Such scenarios pose two fundamental challenges: catastrophic forgetting during sequential personalization and feature interference during multi-concept composition. In this work, we study continual and compositional personalization in VAR models and propose C2P-VAR, a unified framework addressing both challenges. For continual personalization, we introduce C2PVAR-S, which identifies concept-relevant parameters from gradient magnitudes and dynamically updates their selection during training. To preserve previously learned concepts, C2PVAR-S applies regularization only to parameters shared by the current and historical concepts, thereby reducing unnecessary interference without introducing additional model components. For multi-concept personalization, we further propose C2PVAR-M, which employs parallel global and concept-specific branches with spatially localized feature fusion and logit aggregation to achieve controllable concept placement and reduce feature entanglement. Extensive experiments on continual and multi-concept personalization demonstrate that C2P-VAR consistently outperforms existing baselines in subject fidelity, while maintaining competitive text alignment and introducing negligible storage and inference overhead. Our results establish a unified framework for scalable and controllable personalization of visual autoregressive models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 49 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

44 of 49 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2017Neural Discrete Representation Learningcited by 4,712
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  8. 2016Learning without Forgettingcited by 2,496
  9. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  10. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
32 more
  1. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  2. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  3. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  4. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  5. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  6. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  7. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  8. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
  9. 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
  10. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
  11. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
  12. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 386
  13. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  14. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  15. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  16. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
  17. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 173
  18. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 141
  19. 2022Federated Class-Incremental Learningcited by 127
  20. 2024Image and Video Tokenization with Binary Spherical Quantizationcited by 112
  21. 2023Orthogonal Adaptation for Modular Customization of Diffusion Modelscited by 70
  22. 2023Directed Diffusion: Direct Control of Object Placement through Attention Guidancecited by 69
  23. 2024DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalizationcited by 60
  24. 2024LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Modelscited by 55
  25. 2024A Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Modelscited by 52
  26. 2024Identity Decoupling for Multi-Subject Personalization of Text-to-Image Modelscited by 33
  27. 2024Attention Calibration for Disentangled Text-to-Image Personalizationcited by 31
  28. 2023Lego: Learning to Disentangle and Invert Personalized Concepts Beyond Object Appearance in Text-to-Image Diffusion Modelscited by 19
  29. 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generationcited by 13
  30. 2024Selectively Informative Description can Reduce Undesired Embedding Entanglements in Text-to-Image Personalizationcited by 10
  31. 2025Closing the Safety Gap: Surgical Concept Erasure in Visual Autoregressive Modelscited by 9
  32. 2025Continual Personalization for Diffusion Modelscited by 3

Built on by

Not cited yet.