UniVerse: A Unified Modulation Framework for Segmentation-Free,Disentangled Multi-Concept Personalization
Not on the map. Why?
Abstract (arXiv)
Personalized visual understanding has advanced significantly, yet existing approaches struggle to localize and extract specific concepts when input images contain multiple objects. Many prior methods rely heavily on segmentation-based supervision or exhibit poor compositional generalization, limiting their ability to accurately disentangle and manipulate individual concepts. In this work, we propose UniVerse, a Unified Modulation Framework for segmentation-free, disentangled multi-concept personalization in diffusion transformers. Our method allows for composable and decomposable concept extraction, enabling fine-grained localization and representation of target objects without explicit segmentation masks. UniVerse learns to decompose complex scenes into concept-specific representations and then compose them in a unified manner, enabling robust personalization across diverse visual contexts. Through extensive experiments on multiple benchmarks, we demonstrate that UniVerse significantly outperforms state-of-the-art baselines in both localization accuracy and visual fidelity. Qualitative and quantitative results show that our approach can precisely extract target concepts in cluttered scenes, paving the way for more flexible, interpretable, and personalized visual generation and understanding.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 40 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
35 of 40 references.
References · 35
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2024GPT-4o System Cardcited by 4,588
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
23 more
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
- 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
- 2021Image Segmentation Using Text and Image Promptscited by 512
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
- 2024OmniGen: Unified Image Generationcited by 394
- 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
- 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 231
- 2019Understanding and Improving Layer Normalizationcited by 217
- 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 154
- 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 144
- 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
- 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
- 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
- 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
- 2025DreamO: A Unified Framework for Image Customizationcited by 83
- 2024SeedEdit: Align Image Re-Generation to Image Editingcited by 61
- 2025XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulationcited by 56
- 2025TokenVerse: Versatile Multi-concept Personalization in Token Modulation Spacecited by 52
- 2024Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generationcited by 43
- 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
- 2025Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adaptercited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features
- 2026ConceptPrism: Concept Disentanglement in Personalized Diffusion Models via Residual Token Optimization
- 2024Visual Concept-driven Image Generation with Text-to-Image Diffusion Model
- 2025ConceptSplit: Decoupled Multi-Concept Personalization of Diffusion Models via Token-wise Adaptation and Attention Disentanglement
- 2026MMDiff: Extending Diffusion Transformers for Multi-Modal Generation
- 2024ConceptExpress: Harnessing Diffusion Models for Single-image Unsupervised Concept Extraction
- 2025Bridging the gap to real-world language-grounded visual concept learning
- 2026AnchorDiff: Training-Free Concept Grounding for MM-DiTs via Anchor-Based Graph Propagation