# TokenVerse: Versatile Multi-concept Personalization in Token Modulation Space

arXiv: https://arxiv.org/abs/2501.12224 · HTML: https://mlsanity.com/paper/2501.12224/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 35 of its 43 references that are in the map.
- First posted on arXiv 21 Jan 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 35)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models](https://mlsanity.com/paper/2305.18292/) (2023)
- [ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement](https://mlsanity.com/paper/2104.02699/) (2021)
- [Break-A-Scene: Extracting Multiple Concepts from a Single Image](https://mlsanity.com/paper/2305.16311/) (2023)
- [HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing](https://mlsanity.com/paper/2111.15666/) (2021)
- [ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs](https://mlsanity.com/paper/2311.13600/) (2023)
- [Implicit Style-Content Separation using B-LoRA](https://mlsanity.com/paper/2403.14572/) (2024)
- [DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation](https://mlsanity.com/paper/2406.16855/) (2024)
- [A Neural Space-Time Representation for Text-to-Image Personalization](https://mlsanity.com/paper/2305.15391/) (2023)

## Built on by (most cited first, 30 of 52)

- [XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation](https://mlsanity.com/paper/2506.21416/) (2025)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editing](https://mlsanity.com/paper/2510.08532/) (2025)
- [T-LoRA: Single Image Diffusion Model Customization Without Overfitting](https://mlsanity.com/paper/2507.05964/) (2025)
- [Physical Simulator In-the-Loop Video Generation](https://mlsanity.com/paper/2603.06408/) (2026)
- [Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adapter](https://mlsanity.com/paper/2505.18612/) (2025)
- [UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collections](https://mlsanity.com/paper/2509.24817/) (2025)
- [SAEdit: Token-level control for continuous image editing via Sparse AutoEncoder](https://mlsanity.com/paper/2510.05081/) (2025)
- [SpaceControl: Introducing Test-Time Spatial Control to 3D Generative Modeling](https://mlsanity.com/paper/2512.05343/) (2025)
- [BlenderFusion: 3D-Grounded Visual Editing and Generative Compositing](https://mlsanity.com/paper/2506.17450/) (2025)
- [ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generation](https://mlsanity.com/paper/2509.18092/) (2025)
- [Piece it Together: Part-Based Concepting with IP-Priors](https://mlsanity.com/paper/2503.10365/) (2025)
- [CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models](https://mlsanity.com/paper/2507.13984/) (2025)
- [Canvas-to-Image: Compositional Image Generation with Multimodal Controls](https://mlsanity.com/paper/2511.21691/) (2025)
- [Inverse Virtual Try-On: Generating Multi-Category Product-Style Images from Clothed Individuals](https://mlsanity.com/paper/2505.21062/) (2025)
- [Omni-Attribute: Open-vocabulary Attribute Encoder for Visual Concept Personalization](https://mlsanity.com/paper/2512.10955/) (2025)
- [Versatile Editing of Video Content, Actions, and Dynamics without Training](https://mlsanity.com/paper/2603.17989/) (2026)
- [CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvas](https://mlsanity.com/paper/2510.20820/) (2025)
- [Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation](https://mlsanity.com/paper/2603.20725/) (2026)
- [On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers](https://mlsanity.com/paper/2603.28762/) (2026)
- [PartComposer: Learning and Composing Part-Level Concepts from Single-Image Examples](https://mlsanity.com/paper/2506.03004/) (2025)
- [ShowFlow: From Robust Single Concept to Condition-Free Multi-Concept Generation](https://mlsanity.com/paper/2506.18493/) (2025)
- [Neural USD: An object-centric framework for iterative editing and control](https://mlsanity.com/paper/2510.23956/) (2025)
- [Composing Concepts from Images and Videos via Concept-prompt Binding](https://mlsanity.com/paper/2512.09824/) (2025)
- [ELROND: Exploring and decomposing intrinsic capabilities of diffusion models](https://mlsanity.com/paper/2602.10216/) (2026)
- [ChArtist: Generating Pictorial Charts with Unified Spatial and Subject Control](https://mlsanity.com/paper/2603.14209/) (2026)
- [ConceptWeaver: Weaving Disentangled Concepts with Flow](https://mlsanity.com/paper/2603.28493/) (2026)
- [UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation](https://mlsanity.com/paper/2605.12088/) (2026)
- [Few Channels Draw The Whole Picture: Revealing Massive Activations in Diffusion Transformers](https://mlsanity.com/paper/2605.13974/) (2026)
- [Griffin: Generative Reference and Layout Guided Image Composition](https://mlsanity.com/paper/2509.23643/) (2025)

## Abstract (arXiv)

We present TokenVerse -- a method for multi-concept personalization, leveraging a pre-trained text-to-image diffusion model. Our framework can disentangle complex visual elements and attributes from as little as a single image, while enabling seamless plug-and-play generation of combinations of concepts extracted from multiple images. As opposed to existing works, TokenVerse can handle multiple images with multiple concepts each, and supports a wide-range of concepts, including objects, accessories, materials, pose, and lighting. Our work exploits a DiT-based text-to-image model, in which the input text affects the generation through both attention and modulation (shift and scale). We observe that the modulation space is semantic and enables localized control over complex concepts. Building on this insight, we devise an optimization-based framework that takes as input an image and a text description, and finds for each word a distinct direction in the modulation space. These directions can then be used to generate new images that combine the learned concepts in a desired configuration. We demonstrate the effectiveness of TokenVerse in challenging personalization settings, and showcase its advantages over existing methods. project's webpage in https://token-verse.github.io/