mlsanityData to 7 Oct 2026

TokenVerse: Versatile Multi-concept Personalization in Token Modulation Space

arXiv 2501.12224 ↗First posted 21 Jan 2025cs.CVcited by 51

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We present TokenVerse -- a method for multi-concept personalization, leveraging a pre-trained text-to-image diffusion model. Our framework can disentangle complex visual elements and attributes from as little as a single image, while enabling seamless plug-and-play generation of combinations of concepts extracted from multiple images. As opposed to existing works, TokenVerse can handle multiple images with multiple concepts each, and supports a wide-range of concepts, including objects, accessories, materials, pose, and lighting. Our work exploits a DiT-based text-to-image model, in which the input text affects the generation through both attention and modulation (shift and scale). We observe that the modulation space is semantic and enables localized control over complex concepts. Building on this insight, we devise an optimization-based framework that takes as input an image and a text description, and finds for each word a distinct direction in the modulation space. These directions can then be used to generate new images that combine the learned concepts in a desired configuration. We demonstrate the effectiveness of TokenVerse in challenging personalization settings, and showcase its advantages over existing methods. project's webpage in https://token-verse.github.io/

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 35 of its 43 references that are in the map. First posted on arXiv 21 Jan 2025.

Builds on

35 of 43 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024ConceptExpress: Harnessing Diffusion Models for Single-image Unsupervised Concept Extraction9 Jul
  2. 2024LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models18 Mar
  3. 2024OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models16 Mar
  4. 2023Orthogonal Adaptation for Modular Customization of Diffusion Models5 Dec
  5. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 28

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  8. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  9. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  10. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  11. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  12. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
16 more
  1. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  2. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  3. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  4. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  5. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  6. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  7. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  8. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  9. 2021ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinementcited by 248
  10. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  11. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  12. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  13. 2024Implicit Style-Content Separation using B-LoRAcited by 135
  14. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  15. 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
  16. 2023Concept Decomposition for Visual Exploration and Inspirationcited by 61

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 25; 2026: 26.

51 papers.

Sort:

By area:

  1. 2025XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulationcited by 55
  2. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  3. 2025T-LoRA: Single Image Diffusion Model Customization Without Overfittingcited by 15
  4. 2025Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editingcited by 15
  5. 2025Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adaptercited by 13
  6. 2026Physical Simulator In-the-Loop Video Generationcited by 12
  7. 2025UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collectionscited by 10
  8. 2025SpaceControl: Introducing Test-Time Spatial Control to 3D Generative Modelingcited by 9
  9. 2025SAEdit: Token-level control for continuous image editing via Sparse AutoEncodercited by 8
  10. 2025BlenderFusion: 3D-Grounded Visual Editing and Generative Compositingcited by 7
  11. 2025ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generationcited by 7
  12. 2025Piece it Together: Part-Based Concepting with IP-Priorscited by 6
38 more
  1. 2025CSD-VAR: Content-Style Decomposition in Visual Autoregressive Modelscited by 6
  2. 2025Canvas-to-Image: Compositional Image Generation with Multimodal Controlscited by 5
  3. 2025Inverse Virtual Try-On: Generating Multi-Category Product-Style Images from Clothed Individualscited by 4
  4. 2025Omni-Attribute: Open-vocabulary Attribute Encoder for Visual Concept Personalizationcited by 4
  5. 2025CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvascited by 3
  6. 2026Versatile Editing of Video Content, Actions, and Dynamics without Trainingcited by 3
  7. 2026On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformerscited by 3
  8. 2025PartComposer: Learning and Composing Part-Level Concepts from Single-Image Examplescited by 2
  9. 2025ShowFlow: From Robust Single Concept to Condition-Free Multi-Concept Generationcited by 2
  10. 2025Neural USD: An object-centric framework for iterative editing and controlcited by 2
  11. 2025Composing Concepts from Images and Videos via Concept-prompt Bindingcited by 2
  12. 2026ELROND: Exploring and decomposing intrinsic capabilities of diffusion modelscited by 2
  13. 2026Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generationcited by 2
  14. 2026ConceptWeaver: Weaving Disentangled Concepts with Flowcited by 2
  15. 2025Griffin: Generative Reference and Layout Guided Image Compositioncited by 1
  16. 2025Adversarial Concept Distillation for One-Step Diffusion Personalizationcited by 1
  17. 2026Inspiration Seeds: Learning Non-Literal Visual Combinations for Generative Explorationcited by 1
  18. 2026ChArtist: Generating Pictorial Charts with Unified Spatial and Subject Controlcited by 1
  19. 2026DreamShot: Personalized Storyboard Synthesis with Video Diffusion Priorcited by 1
  20. 2026UniCustom: Unified Visual Conditioning for Multi-Reference Image Generationcited by 1
  21. 2026MULTI: Disentangling Camera Lens, Sensor, View, and Domain for Novel Image Generationcited by 1
  22. 2026Few Channels Draw The Whole Picture: Revealing Massive Activations in Diffusion Transformerscited by 1
  23. 2026VT-DUDA: Visual Token Conditioning for Diffusion-guided Unsupervised Domain Adaptationcited by 1
  24. 2025FreeFuse: Multi-Subject LoRA Fusion via Adaptive Token-Level Routing at Test Timecited by 0
  25. 2025MIFO: Learning and Synthesizing Multi-Instance from One Imagecited by 0
  26. 2025Match-and-Fuse: Consistent Generation from Unstructured Image Setscited by 0
  27. 2026Muses: Designing, Composing, Generating Nonexistent Fantasy 3D Creatures without Trainingcited by 0
  28. 2026Rethinking Global Text Conditioning in Diffusion Transformerscited by 0
  29. 2026ConceptPrism: Concept Disentanglement in Personalized Diffusion Models via Residual Token Optimizationcited by 0
  30. 2026AnyPhoto: Multi-Person Identity Preserving Image Generation with ID Adaptive Modulation on Location Canvascited by 0
  31. 2026PixelSmile: Toward Fine-Grained Facial Expression Editingcited by 0
  32. 2026TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Spacecited by 0
  33. 2026UniVerse: A Unified Modulation Framework for Segmentation-Free,Disentangled Multi-Concept Personalizationcited by 0
  34. 2026AlbedoEdit: Unified Instance-Level Video Editing with Albedo Guidancecited by 0
  35. 2026Thinking in Boxes: 3D Editing in Real Images Made Easycited by 0
  36. 2026Token-to-Token Alignment of Text Embeddings for Semantic Blendingcited by 0
  37. 2026Latent-Identity Tuning in Text-to-Image Personalization Modelscited by 0
  38. 2026X-MULTI: VLM-based Imaging Factor Disentanglement for Factor-Aware Image Synthesiscited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Visual Concept-driven Image Generation with Text-to-Image Diffusion Model
  2. 2025TARA: Token-Aware LoRA for Composable Personalization in Diffusion Models
  3. 2024Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models
  4. 2025ConceptSplit: Decoupled Multi-Concept Personalization of Diffusion Models via Token-wise Adaptation and Attention Disentanglement
  5. 2025ACCORD: Alleviating Concept Coupling through Dependence Regularization for Text-to-Image Diffusion Personalization
  6. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models
  7. 2024Learning to Customize Text-to-Image Diffusion In Diverse Context
  8. 2022Multi-Concept Customization of Text-to-Image Diffusion