# ExpertGen: Training-Free Expert Guidance for Controllable Text-to-Face Generation

arXiv: https://arxiv.org/abs/2505.17256 · HTML: https://mlsanity.com/paper/2505.17256/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 27 of its 32 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 27 of 27)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model](https://mlsanity.com/paper/2303.09833/) (2023)
- [LCM-LoRA: A Universal Stable-Diffusion Acceleration Module](https://mlsanity.com/paper/2311.05556/) (2023)
- [General Facial Representation Learning in a Visual-Linguistic Manner](https://mlsanity.com/paper/2112.03109/) (2021)
- [Manifold Preserving Guided Diffusion](https://mlsanity.com/paper/2311.16424/) (2023)
- [TFG: Unified Training-Free Guidance for Diffusion Models](https://mlsanity.com/paper/2409.15761/) (2024)
- [Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control](https://mlsanity.com/paper/2405.12970/) (2024)
- [LCM-Lookahead for Encoder-based Text-to-Image Personalization](https://mlsanity.com/paper/2404.03620/) (2024)
- [MiVOLO: Multi-input Transformer for Age and Gender Estimation](https://mlsanity.com/paper/2307.04616/) (2023)
- [Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Models](https://mlsanity.com/paper/2309.02218/) (2023)
- [Enhancing Diffusion-Based Image Synthesis with Robust Classifier Guidance](https://mlsanity.com/paper/2208.08664/) (2022)
- [PreciseControl: Enhancing Text-To-Image Diffusion Models with Fine-Grained Attribute Control](https://mlsanity.com/paper/2408.05083/) (2024)
- [SegFace: Face Segmentation of Long-Tail Classes](https://mlsanity.com/paper/2412.08647/) (2024)
- [Face2Diffusion for Fast and Editable Face Personalization](https://mlsanity.com/paper/2403.05094/) (2024)
- [RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance](https://mlsanity.com/paper/2405.14677/) (2024)
- [FaceScore: Benchmarking and Enhancing Face Quality in Human Generation](https://mlsanity.com/paper/2406.17100/) (2024)

## Built on by (most cited first, 1 of 1)

- [Identity-Conditioned Latent Consistency Distillation for Face Synthesis](https://mlsanity.com/paper/2608.31053/) (2026)

## Abstract (arXiv)

Recent advances in diffusion models have significantly improved text-to-face generation, but achieving fine-grained control over facial features remains a challenge. Existing methods often require training additional modules to handle specific controls such as identity, attributes, or age, making them inflexible and resource-intensive. We propose ExpertGen, a training-free framework that leverages pre-trained expert models such as face recognition, facial attribute recognition, and age estimation networks to guide generation with fine control. Our approach uses a latent consistency model to ensure realistic and in-distribution predictions at each diffusion step, enabling accurate guidance signals to effectively steer the diffusion process. We show qualitatively and quantitatively that expert models can guide the generation process with high precision, and multiple experts can collaborate to enable simultaneous control over diverse facial aspects. By allowing direct integration of off-the-shelf expert models, our method transforms any such model into a plug-and-play component for controllable face generation.