mlsanityData to 14 Oct 2026

Generating Synthetic Data via Augmentations for Improved Facial Resemblance in DreamBooth and InstantID

arXiv 2505.03557 ↗First posted 6 May 2025cs.CV cs.AIcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Personalizing Stable Diffusion for professional portrait generation from amateur photos faces challenges in maintaining facial resemblance. This paper evaluates the impact of augmentation strategies on two personalization methods: DreamBooth and InstantID. We compare classical augmentations (flipping, cropping, color adjustments) with generative augmentation using InstantID's synthetic images to enrich training data. Using SDXL and a new FaceDistance metric based on FaceNet, we quantitatively assess facial similarity. Results show classical augmentations can cause artifacts harming identity retention, while InstantID improves fidelity when balanced with real images to avoid overfitting. A user study with 97 participants confirms high photorealism and preferences for InstantID's polished look versus DreamBooth's identity accuracy. Our findings inform effective augmentation strategies for personalized text-to-image generation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 18 of its 34 references that are in the map. First posted on arXiv 6 May 2025.

Builds on

18 of 34 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Context-guided Responsible Data Augmentation with Diffusion Models12 Mar
  2. 2024DiffuseMix: Label-Preserving Data Augmentation with Diffusion Models5 Apr
  3. 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
  4. 2023Effective Data Augmentation With Diffusion Models7 Feb
  5. 2022MagicMix: Semantic Mixing with Diffusion Models28 Oct
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug

Other references · 12

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  6. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  7. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,254
  8. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 903
  9. 2019FaceShifter: Towards High Fidelity And Occlusion Aware Face Swappingcited by 221
  10. 2020Bridging Composite and Real: Towards End-to-end Deep Image Mattingcited by 89
  11. 2022FaceOff: A Video-to-Video Face Swapping Systemcited by 3
  12. 2024GlamTry: Advancing Virtual Try-On for High-End Accessoriescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023MagiCapture: High-Resolution Multi-Concept Portrait Customization
  2. 2023PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization
  3. 2023Limitations of Face Image Generation
  4. 2024MasterWeaver: Taming Editability and Face Identity for Personalized Text-to-Image Generation
  5. 2024LCM-Lookahead for Encoder-based Text-to-Image Personalization
  6. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models
  7. 2026FaceSnap: Enhanced ID-fidelity Network for Tuning-free Portrait Customization
  8. 2024Beyond Inserting: Learning Identity Embedding for Semantic-Fidelity Personalized Diffusion Generation