# Omni-ID: Holistic Identity Representation Designed for Generative Tasks

arXiv: https://arxiv.org/abs/2412.09694 · HTML: https://mlsanity.com/paper/2412.09694/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 53 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 30 of 41)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning](https://mlsanity.com/paper/2304.03411/) (2023)
- [PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding](https://mlsanity.com/paper/2312.04461/) (2023)
- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [P+: Extended Textual Conditioning in Text-to-Image Generation](https://mlsanity.com/paper/2303.09522/) (2023)
- [TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment](https://mlsanity.com/paper/2308.03060/) (2023)
- [HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models](https://mlsanity.com/paper/2307.06949/) (2023)

## Built on by (most cited first, 18 of 18)

- [Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generation](https://mlsanity.com/paper/2508.07901/) (2025)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [Dynamic Concepts Personalization from Single Videos](https://mlsanity.com/paper/2502.14844/) (2025)
- [WithAnyone: Towards Controllable and ID Consistent Image Generation](https://mlsanity.com/paper/2510.14975/) (2025)
- [UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collections](https://mlsanity.com/paper/2509.24817/) (2025)
- [Zero-Shot Dynamic Concept Personalization with Grid-Based LoRA](https://mlsanity.com/paper/2507.17963/) (2025)
- [Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation](https://mlsanity.com/paper/2509.21989/) (2025)
- [Canvas-to-Image: Compositional Image Generation with Multimodal Controls](https://mlsanity.com/paper/2511.21691/) (2025)
- [Preventing Shortcuts in Adapter Training via Providing the Shortcuts](https://mlsanity.com/paper/2510.20887/) (2025)
- [Omni-Attribute: Open-vocabulary Attribute Encoder for Visual Concept Personalization](https://mlsanity.com/paper/2512.10955/) (2025)
- [CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvas](https://mlsanity.com/paper/2510.20820/) (2025)
- [FlexID: Training-Free Flexible Identity Injection via Intent-Aware Modulation for Text-to-Image Generation](https://mlsanity.com/paper/2602.07554/) (2026)
- [HY-WU (Part I): An Extensible Functional Neural Memory Framework and An Instantiation in Text-Guided Image Editing](https://mlsanity.com/paper/2603.07236/) (2026)
- [Learning Joint ID-Textual Representation for ID-Preserving Image Synthesis](https://mlsanity.com/paper/2504.14202/) (2025)
- [ImageGem: In-the-wild Generative Image Interaction Dataset for Generative Model Personalization](https://mlsanity.com/paper/2510.18433/) (2025)
- [FFAvatar: Few-Shot, Feed-Forward, and Generalizable Avatar Reconstruction](https://mlsanity.com/paper/2605.15320/) (2026)
- [NearID: Identity Representation Learning via Near-identity Distractors](https://mlsanity.com/paper/2604.01973/) (2026)
- [Latent-Identity Tuning in Text-to-Image Personalization Models](https://mlsanity.com/paper/2607.11885/) (2026)

## Abstract (arXiv)

We introduce Omni-ID, a novel facial representation designed specifically for generative tasks. Omni-ID encodes holistic information about an individual's appearance across diverse expressions and poses within a fixed-size representation. It consolidates information from a varied number of unstructured input images into a structured representation, where each entry represents certain global or local identity features. Our approach uses a few-to-many identity reconstruction training paradigm, where a limited set of input images is used to reconstruct multiple target images of the same individual in various poses and expressions. A multi-decoder framework is further employed to leverage the complementary strengths of diverse decoders during training. Unlike conventional representations, such as CLIP and ArcFace, which are typically learned through discriminative or contrastive objectives, Omni-ID is optimized with a generative objective, resulting in a more comprehensive and nuanced identity capture for generative tasks. Trained on our MFHQ dataset -- a multi-view facial image collection, Omni-ID demonstrates substantial improvements over conventional representations across various generative tasks.