# DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition

arXiv: https://arxiv.org/abs/2504.17349 · HTML: https://mlsanity.com/paper/2504.17349/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 23 references that are in the map.
- First posted on arXiv 24 Apr 2025.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [OmniGen: Unified Image Generation](https://mlsanity.com/paper/2409.11340/) (2024)
- [DreamBooth3D: Subject-Driven Text-to-3D Generation](https://mlsanity.com/paper/2303.13508/) (2023)
- [Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning](https://mlsanity.com/paper/2004.11660/) (2020)
- [DreamVideo: Composing Your Dream Videos with Customized Subject and Motion](https://mlsanity.com/paper/2312.04433/) (2023)
- [DisCo: Disentangled Control for Realistic Human Dance Generation](https://mlsanity.com/paper/2307.00040/) (2023)
- [Disentangled Person Image Generation](https://mlsanity.com/paper/1712.02621/) (2017)
- [DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations](https://mlsanity.com/paper/2403.06951/) (2024)
- [Learning Attention as Disentangler for Compositional Zero-shot Learning](https://mlsanity.com/paper/2303.15111/) (2023)
- [Diffusion Models for Generative Outfit Recommendation](https://mlsanity.com/paper/2402.17279/) (2024)
- [A New Creative Generation Pipeline for Click-Through Rate with Stable Diffusion Model](https://mlsanity.com/paper/2401.10934/) (2024)
- [Learning Disentangled Identifiers for Action-Customized Text-to-Image Generation](https://mlsanity.com/paper/2311.15841/) (2023)
- [Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting](https://mlsanity.com/paper/2310.08129/) (2023)
- [PMG : Personalized Multimodal Generation with Large Language Models](https://mlsanity.com/paper/2404.08677/) (2024)
- [CoRe: Context-Regularized Text Embedding Learning for Text-to-Image Personalization](https://mlsanity.com/paper/2408.15914/) (2024)
- [Investigating Personalization Methods in Text to Music Generation](https://mlsanity.com/paper/2309.11140/) (2023)
- [Chaining text-to-image and large language model: A novel approach for generating personalized e-commerce banners](https://mlsanity.com/paper/2403.05578/) (2024)

## Built on by (most cited first, 8 of 8)

- [Latent Inter-User Difference Modeling for LLM Personalization](https://mlsanity.com/paper/2507.20849/) (2025)
- [SteerX: Disentangled Steering for LLM Personalization](https://mlsanity.com/paper/2510.22256/) (2025)
- [HLLM-Creator: Hierarchical LLM-based Personalized Creative Generation](https://mlsanity.com/paper/2508.18118/) (2025)
- [Thinking with Frames: Generative Video Distortion Evaluation via Frame Reward Model](https://mlsanity.com/paper/2601.04033/) (2026)
- [Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation](https://mlsanity.com/paper/2603.20725/) (2026)
- [Generate, Not Recommend: Personalized Multimodal Content Generation](https://mlsanity.com/paper/2506.01704/) (2025)
- [GSE: Evaluating Sticker Visual Semantic Similarity via a General Sticker Encoder](https://mlsanity.com/paper/2511.04977/) (2025)
- [NextAds: Towards Next-generation Personalized Video Advertising](https://mlsanity.com/paper/2603.02137/) (2026)

## Abstract (arXiv)

Personalized image generation has emerged as a promising direction in multimodal content creation. It aims to synthesize images tailored to individual style preferences (e.g., color schemes, character appearances, layout) and semantic intentions (e.g., emotion, action, scene contexts) by leveraging user-interacted history images and multimodal instructions. Despite notable progress, existing methods -- whether based on diffusion models, large language models, or Large Multimodal Models (LMMs) -- struggle to accurately capture and fuse user style preferences and semantic intentions. In particular, the state-of-the-art LMM-based method suffers from the entanglement of visual features, leading to Guidance Collapse, where the generated images fail to preserve user-preferred styles or reflect the specified semantics. To address these limitations, we introduce DRC, a novel personalized image generation framework that enhances LMMs through Disentangled Representation Composition. DRC explicitly extracts user style preferences and semantic intentions from history images and the reference image, respectively, to form user-specific latent instructions that guide image generation within LMMs. Specifically, it involves two critical learning stages: 1) Disentanglement learning, which employs a dual-tower disentangler to explicitly separate style and semantic features, optimized via a reconstruction-driven paradigm with difficulty-aware importance sampling; and 2) Personalized modeling, which applies semantic-preserving augmentations to effectively adapt the disentangled representations for robust personalized generation. Extensive experiments on two benchmarks demonstrate that DRC shows competitive performance while effectively mitigating the guidance collapse issue, underscoring the importance of disentangled representation learning for controllable and effective personalized image generation.