# RealisID: Scale-Robust and Fine-Controllable Identity Customization via Local and Global Complementation

arXiv: https://arxiv.org/abs/2412.16832 · HTML: https://mlsanity.com/paper/2412.16832/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 36 references that are in the map.
- First posted on arXiv 22 Dec 2024.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Paint by Example: Exemplar-based Image Editing with Diffusion Models](https://mlsanity.com/paper/2211.13227/) (2022)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding](https://mlsanity.com/paper/2312.04461/) (2023)
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](https://mlsanity.com/paper/2305.10431/) (2023)
- [AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation](https://mlsanity.com/paper/2403.17694/) (2024)
- [StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On](https://mlsanity.com/paper/2312.01725/) (2023)
- [Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2406.01900/) (2024)
- [PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization](https://mlsanity.com/paper/2312.06354/) (2023)
- [StableIdentity: Inserting Anybody into Anywhere at First Sight](https://mlsanity.com/paper/2401.15975/) (2024)
- [IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models](https://mlsanity.com/paper/2403.13535/) (2024)
- [FlashFace: Human Image Personalization with High-fidelity Identity Preservation](https://mlsanity.com/paper/2403.17008/) (2024)
- [CosmicMan: A Text-to-Image Foundation Model for Humans](https://mlsanity.com/paper/2404.01294/) (2024)
- [InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generation](https://mlsanity.com/paper/2404.19427/) (2024)
- [CapHuman: Capture Your Moments in Parallel Universes](https://mlsanity.com/paper/2402.00627/) (2024)
- [MM-Diff: High-Fidelity Image Personalization via Multi-Modal Condition Integration](https://mlsanity.com/paper/2403.15059/) (2024)

## Built on by (most cited first, 1 of 1)

- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)

## Abstract (arXiv)

Recently, the success of text-to-image synthesis has greatly advanced the development of identity customization techniques, whose main goal is to produce realistic identity-specific photographs based on text prompts and reference face images. However, it is difficult for existing identity customization methods to simultaneously meet the various requirements of different real-world applications, including the identity fidelity of small face, the control of face location, pose and expression, as well as the customization of multiple persons. To this end, we propose a scale-robust and fine-controllable method, namely RealisID, which learns different control capabilities through the cooperation between a pair of local and global branches. Specifically, by using cropping and up-sampling operations to filter out face-irrelevant information, the local branch concentrates the fine control of facial details and the scale-robust identity fidelity within the face region. Meanwhile, the global branch manages the overall harmony of the entire image. It also controls the face location by taking the location guidance as input. As a result, RealisID can benefit from the complementarity of these two branches. Finally, by implementing our branches with two different variants of ControlNet, our method can be easily extended to handle multi-person customization, even only trained on single-person datasets. Extensive experiments and ablation studies indicate the effectiveness of RealisID and verify its ability in fulfilling all the requirements mentioned above.