# WithAnyone: Towards Controllable and ID Consistent Image Generation

arXiv: https://arxiv.org/abs/2510.14975 · HTML: https://mlsanity.com/paper/2510.14975/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 72 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Builds on (most cited first, 30 of 58)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [OmniGen: Unified Image Generation](https://mlsanity.com/paper/2409.11340/) (2024)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](https://mlsanity.com/paper/2305.10431/) (2023)
- [Recognize Anything: A Strong Image Tagging Model](https://mlsanity.com/paper/2306.03514/) (2023)
- [OmniGen2: Towards Instruction-Aligned Multimodal Generation](https://mlsanity.com/paper/2506.18871/) (2025)
- [Less-to-More Generalization: Unlocking More Controllability by In-Context Generation](https://mlsanity.com/paper/2504.02160/) (2025)
- [PuLID: Pure and Lightning ID Customization via Contrastive Alignment](https://mlsanity.com/paper/2404.16022/) (2024)
- [DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering](https://mlsanity.com/paper/2307.10173/) (2023)
- [Arc2Face: A Foundation Model for ID-Consistent Human Faces](https://mlsanity.com/paper/2403.11641/) (2024)
- [DreamO: A Unified Framework for Image Customization](https://mlsanity.com/paper/2504.16915/) (2025)
- [Face0: Instantaneously Conditioning a Text-to-Image Model on a Face](https://mlsanity.com/paper/2306.06638/) (2023)
- [PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization](https://mlsanity.com/paper/2312.06354/) (2023)

## Built on by (most cited first, 17 of 17)

- [MICo-150K: A Comprehensive Dataset Advancing Multi-Image Composition](https://mlsanity.com/paper/2512.07348/) (2025)
- [iMontage: Unified, Versatile, Highly Dynamic Many-to-many Image Generation](https://mlsanity.com/paper/2511.20635/) (2025)
- [OpenSubject: Leveraging Video-Derived Identity and Diversity Priors for Subject-driven Image Generation and Manipulation](https://mlsanity.com/paper/2512.08294/) (2025)
- [MACRO: Advancing Multi-Reference Image Generation with Structured Long-Context Data](https://mlsanity.com/paper/2603.25319/) (2026)
- [OmniLottie: Generating Vector Animations via Parameterized Lottie Tokens](https://mlsanity.com/paper/2603.02138/) (2026)
- [Ar2Can: An Architect and an Artist Leveraging a Canvas for Multi-Human Generation](https://mlsanity.com/paper/2511.22690/) (2025)
- [UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation](https://mlsanity.com/paper/2605.12088/) (2026)
- [OmniDiT: Extending Diffusion Transformer to Omni-VTON Framework](https://mlsanity.com/paper/2603.19643/) (2026)
- [PixelSmile: Toward Fine-Grained Facial Expression Editing](https://mlsanity.com/paper/2603.25728/) (2026)
- [MPIE-Bench: Benchmarking Anatomically Plausible Multi-Person Interaction Editing](https://mlsanity.com/paper/2607.27616/) (2026)
- [Illuminating Visual Identity in Universal Multimodal Embeddings](https://mlsanity.com/paper/2608.01794/) (2026)
- [Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalization](https://mlsanity.com/paper/2512.03964/) (2025)
- [AnyPhoto: Multi-Person Identity Preserving Image Generation with ID Adaptive Modulation on Location Canvas](https://mlsanity.com/paper/2603.14770/) (2026)
- [Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generation](https://mlsanity.com/paper/2605.26111/) (2026)
- [Mixed-Modality Dual Face-Hair Retrieval](https://mlsanity.com/paper/2606.03470/) (2026)
- [WithEveryone: Unified Planning and Identity Grounding for Group Image Generation](https://mlsanity.com/paper/2608.20336/) (2026)
- [Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning](https://mlsanity.com/paper/2608.26993/) (2026)

## Abstract (arXiv)

Identity-consistent generation has become an important focus in text-to-image research, with recent models achieving notable success in producing images aligned with a reference identity. Yet, the scarcity of large-scale paired datasets containing multiple images of the same individual forces most approaches to adopt reconstruction-based training. This reliance often leads to a failure mode we term copy-paste, where the model directly replicates the reference face rather than preserving identity across natural variations in pose, expression, or lighting. Such over-similarity undermines controllability and limits the expressive power of generation. To address these limitations, we (1) construct a large-scale paired dataset MultiID-2M, tailored for multi-person scenarios, providing diverse references for each identity; (2) introduce a benchmark that quantifies both copy-paste artifacts and the trade-off between identity fidelity and variation; and (3) propose a novel training paradigm with a contrastive identity loss that leverages paired data to balance fidelity with diversity. These contributions culminate in WithAnyone, a diffusion-based model that effectively mitigates copy-paste while preserving high identity similarity. Extensive qualitative and quantitative experiments demonstrate that WithAnyone significantly reduces copy-paste artifacts, improves controllability over pose and expression, and maintains strong perceptual quality. User studies further validate that our method achieves high identity fidelity while enabling expressive controllable generation.