DVI: Disentangling Semantic and Visual Identity for Training-Free Personalized Generation
Not on the map. Why?
Abstract (arXiv)
Recent tuning-free identity customization methods achieve high facial fidelity but often overlook visual context, such as lighting, skin texture, and environmental tone. This limitation leads to ``Semantic-Visual Dissonance,'' where accurate facial geometry clashes with the input's unique atmosphere, causing an unnatural ``sticker-like'' effect. We propose **DVI (Disentangled Visual-Identity)**, a zero-shot framework that orthogonally disentangles identity into fine-grained semantic and coarse-grained visual streams. Unlike methods relying solely on semantic vectors, DVI exploits the inherent statistical properties of the VAE latent space, utilizing mean and variance as lightweight descriptors for global visual atmosphere. We introduce a **Parameter-Free Feature Modulation** mechanism that adaptively modulates semantic embeddings with these visual statistics, effectively injecting the reference's ``visual soul'' without training. Furthermore, a **Dynamic Temporal Granularity Scheduler** aligns with the diffusion process, prioritizing visual atmosphere in early denoising stages while refining semantic details later. Extensive experiments demonstrate that DVI significantly enhances visual consistency and atmospheric fidelity without parameter fine-tuning, maintaining robust identity preservation and outperforming state-of-the-art methods in IBench evaluations.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 25 of its 29 references that are in the map. First posted on arXiv 22 Dec 2025.
Builds on
25 of 29 references.
References · 25
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
- 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
- 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
- 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
13 more
- 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 154
- 2024PuLID: Pure and Lightning ID Customization via Contrastive Alignmentcited by 144
- 2025DreamO: A Unified Framework for Image Customizationcited by 83
- 2025XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulationcited by 56
- 2024UniPortrait: A Unified Framework for Identity-Preserving Single- and Multi-Human Image Personalizationcited by 47
- 2025InfiniteYou: Flexible Photo Recrafting While Preserving Your Identitycited by 35
- 2025UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Rewardcited by 19
- 2023FaceChain: A Playground for Human-centric Artificial Intelligence Generated Contentcited by 10
- 2025EditID: Training-Free Editable ID Customization for Text-to-Image Generationcited by 8
- 2025EditIDv2: Editable ID Customization with Data-Lubricated ID Feature Integration for Text-to-Image Generationcited by 8
- 2023Smartbanner: Intelligent banner design framework that strikes a balance between creative freedom and design rulescited by 6
- 2024Layout Control and Semantic Guidance with Attention Loss Backward for T2I Diffusion Modelcited by 5
- 2025Learning Joint ID-Textual Representation for ID-Preserving Image Synthesiscited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026FlexID: Training-Free Flexible Identity Injection via Intent-Aware Modulation for Text-to-Image Generationcited by 3
- 2026Inject Where It Matters: Training-Free Spatially-Adaptive Identity Preservation for Text-to-Image Personalizationcited by 3
- 2026Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generationcited by 0
- 2026Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generation25 May 2026
- 2026Inject Where It Matters: Training-Free Spatially-Adaptive Identity Preservation for Text-to-Image Personalization15 Feb 2026
- 2026FlexID: Training-Free Flexible Identity Injection via Intent-Aware Modulation for Text-to-Image Generation7 Feb 2026