# CopyCat: Improving Fine-Grained Subject Consistency in Subject-to-Image Models within Seconds

arXiv: https://arxiv.org/abs/2608.00674 · HTML: https://mlsanity.com/paper/2608.00674/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 1 Aug 2026.

## Builds on (most cited first, 26 of 26)

- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](https://mlsanity.com/paper/2305.14720/) (2023)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [OminiControl: Minimal and Universal Control for Diffusion Transformer](https://mlsanity.com/paper/2411.15098/) (2024)
- [PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding](https://mlsanity.com/paper/2312.04461/) (2023)
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](https://mlsanity.com/paper/2305.10431/) (2023)
- [Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models](https://mlsanity.com/paper/2305.18292/) (2023)
- [Subject-driven Text-to-Image Generation via Apprenticeship Learning](https://mlsanity.com/paper/2304.00186/) (2023)
- [Key-Locked Rank One Editing for Text-to-Image Personalization](https://mlsanity.com/paper/2305.01644/) (2023)
- [ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs](https://mlsanity.com/paper/2311.13600/) (2023)
- [Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning](https://mlsanity.com/paper/2307.11410/) (2023)
- [Less-to-More Generalization: Unlocking More Controllability by In-Context Generation](https://mlsanity.com/paper/2504.02160/) (2025)
- [Cones: Concept Neurons in Diffusion Models for Customized Generation](https://mlsanity.com/paper/2303.05125/) (2023)
- [PuLID: Pure and Lightning ID Customization via Contrastive Alignment](https://mlsanity.com/paper/2404.16022/) (2024)
- [MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidance](https://mlsanity.com/paper/2406.07209/) (2024)
- [DreamO: A Unified Framework for Image Customization](https://mlsanity.com/paper/2504.16915/) (2025)
- [DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization](https://mlsanity.com/paper/2402.09812/) (2024)
- [XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation](https://mlsanity.com/paper/2506.21416/) (2025)
- [MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generation](https://mlsanity.com/paper/2404.11565/) (2024)
- [Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generation](https://mlsanity.com/paper/2409.17920/) (2024)
- [USO: Unified Style and Subject-Driven Generation via Disentangled and Reward Learning](https://mlsanity.com/paper/2508.18966/) (2025)
- [Contrastive Test-Time Composition of Multiple LoRA Models for Image Generation](https://mlsanity.com/paper/2403.19776/) (2024)
- [PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewards](https://mlsanity.com/paper/2512.01236/) (2025)
- [FreeLoRA: Enabling Training-Free LoRA Fusion for Autoregressive Multi-Subject Personalization](https://mlsanity.com/paper/2507.01792/) (2025)

## Abstract (arXiv)

Recent subject-to-image models have achieved impressive progress in personalized image generation, yet they still struggle to preserve fine-grained subject-specific details. A major reason is the lack of high-quality fine-grained identity supervision: real paired data are expensive to collect, while synthesized training pairs often preserve only coarse subject appearance and fail to capture subtle subject-specific details. In this work, we propose CopyCat, a lightweight model-refinement framework that improves fine-grained subject consistency within only a few seconds. CopyCat performs a one-time refinement of a pretrained subject-to-image model by attaching a lightweight Fine-grained Consistency LoRA (FCLoRA) and optimizing it using a single proxy image, which is used as both the conditioning image and the reconstruction target. This exact self-reconstruction objective substantially simplifies the optimization task, enabling effective fine-grained refinement within only a few seconds. The refinement is performed only once; the resulting model can be directly applied to diverse unseen reference subjects and prompts without further subject-specific optimization. We further revisit subject-to-image LoRA training in double-stream diffusion transformers and find that adapting only the visual stream consistently improves subject consistency. Extensive experiments on DreamBench and XVerseBench demonstrate consistent improvements in fine-grained subject consistency across representative subject-to-image models under both single- and multi-subject settings.