C2P-VAR: Continual and Compositional Personalization in Visual Autoregressive Models
Not on the map. Why?
Abstract (arXiv)
Visual autoregressive (VAR) models have recently emerged as an efficient paradigm for text-to-image generation, yet their personalization capabilities remain largely limited to static, single-concept settings. In practice, users may continuously introduce new concepts and wish to compose multiple personalized concepts within a single image. Such scenarios pose two fundamental challenges: catastrophic forgetting during sequential personalization and feature interference during multi-concept composition. In this work, we study continual and compositional personalization in VAR models and propose C2P-VAR, a unified framework addressing both challenges. For continual personalization, we introduce C2PVAR-S, which identifies concept-relevant parameters from gradient magnitudes and dynamically updates their selection during training. To preserve previously learned concepts, C2PVAR-S applies regularization only to parameters shared by the current and historical concepts, thereby reducing unnecessary interference without introducing additional model components. For multi-concept personalization, we further propose C2PVAR-M, which employs parallel global and concept-specific branches with spatially localized feature fusion and logit aggregation to achieve controllable concept placement and reduce feature entanglement. Extensive experiments on continual and multi-concept personalization demonstrate that C2P-VAR consistently outperforms existing baselines in subject fidelity, while maintaining competitive text alignment and introducing negligible storage and inference overhead. Our results establish a unified framework for scalable and controllable personalization of visual autoregressive models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 49 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
44 of 49 references.
References · 44
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2017Neural Discrete Representation Learningcited by 4,712
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2016Learning without Forgettingcited by 2,496
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
- 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
32 more
- 2017SmoothGrad: removing noise by adding noisecited by 1,109
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
- 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
- 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
- 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
- 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
- 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 386
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
- 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
- 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
- 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
- 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 173
- 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 141
- 2022Federated Class-Incremental Learningcited by 127
- 2024Image and Video Tokenization with Binary Spherical Quantizationcited by 112
- 2023Orthogonal Adaptation for Modular Customization of Diffusion Modelscited by 70
- 2023Directed Diffusion: Direct Control of Object Placement through Attention Guidancecited by 69
- 2024DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalizationcited by 60
- 2024LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Modelscited by 55
- 2024A Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Modelscited by 52
- 2024Identity Decoupling for Multi-Subject Personalization of Text-to-Image Modelscited by 33
- 2024Attention Calibration for Disentangled Text-to-Image Personalizationcited by 31
- 2023Lego: Learning to Disentangle and Invert Personalized Concepts Beyond Object Appearance in Text-to-Image Diffusion Modelscited by 19
- 2025Fine-Tuning Visual Autoregressive Models for Subject-Driven Generationcited by 13
- 2024Selectively Informative Description can Reduce Undesired Embedding Entanglements in Text-to-Image Personalizationcited by 10
- 2025Closing the Safety Gap: Surgical Concept Erasure in Visual Autoregressive Modelscited by 9
- 2025Continual Personalization for Diffusion Modelscited by 3
Built on by
Not cited yet.