# Generating Synthetic Data via Augmentations for Improved Facial Resemblance in DreamBooth and InstantID

arXiv: https://arxiv.org/abs/2505.03557 · HTML: https://mlsanity.com/paper/2505.03557/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 6 May 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 18 of 18)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [Effective Data Augmentation With Diffusion Models](https://mlsanity.com/paper/2302.07944/) (2023)
- [FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping](https://mlsanity.com/paper/1912.13457/) (2019)
- [Bridging Composite and Real: Towards End-to-end Deep Image Matting](https://mlsanity.com/paper/2010.16188/) (2020)
- [DiffuseMix: Label-Preserving Data Augmentation with Diffusion Models](https://mlsanity.com/paper/2405.14881/) (2024)
- [MagicMix: Semantic Mixing with Diffusion Models](https://mlsanity.com/paper/2210.16056/) (2022)
- [Context-guided Responsible Data Augmentation with Diffusion Models](https://mlsanity.com/paper/2503.10687/) (2025)
- [FaceOff: A Video-to-Video Face Swapping System](https://mlsanity.com/paper/2208.09788/) (2022)
- [GlamTry: Advancing Virtual Try-On for High-End Accessories](https://mlsanity.com/paper/2409.14553/) (2024)

## Abstract (arXiv)

Personalizing Stable Diffusion for professional portrait generation from amateur photos faces challenges in maintaining facial resemblance. This paper evaluates the impact of augmentation strategies on two personalization methods: DreamBooth and InstantID. We compare classical augmentations (flipping, cropping, color adjustments) with generative augmentation using InstantID's synthetic images to enrich training data. Using SDXL and a new FaceDistance metric based on FaceNet, we quantitatively assess facial similarity. Results show classical augmentations can cause artifacts harming identity retention, while InstantID improves fidelity when balanced with real images to avoid overfitting. A user study with 97 participants confirms high photorealism and preferences for InstantID's polished look versus DreamBooth's identity accuracy. Our findings inform effective augmentation strategies for personalized text-to-image generation.