mlsanityData to 14 Oct 2026

From Synthetic to Real: Toward Identity-Consistent Makeup Transfer with Synthetic and Real Data

arXiv 2605.07861 ↗First posted 8 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Makeup transfer aims to apply the makeup style of a reference portrait to a source portrait while preserving identity and background. Early methods formulate this task as unsupervised image-to-image translation, relying on surrogate objectives and often yielding limited performance. Recent diffusion- and flow-based approaches instead exploit synthetic data for supervised training, leading to significant improvements. However, these methods still face two critical challenges: synthetic supervision frequently fails to faithfully preserve identity, and the domain gap between synthetic and real data limits generalization, resulting in degraded performance in complex real-world scenarios. To address these issues, this paper first proposes ConsistentBeauty, a novel data curation pipeline that ensures makeup fidelity and strict identity consistency within the synthesized data. Second, we propose RealBeauty, a synthetic-to-real post-training framework. Beyond supervised learning on curated synthetic data, we further adapt the model to real-world scenarios through reinforcement learning and design novel verifiable rewards tailored to the makeup transfer task. It allows the model to further benefit from real makeup patterns beyond synthetic supervision. In addition, we establish a new diverse benchmark for makeup transfer, covering a wide range of skin tones, ages, genders, poses, and makeup styles, thereby enabling a more comprehensive evaluation of model performance under diverse real-world conditions. Extensive experiments show that our method achieves state-of-the-art performance on multiple benchmarks and demonstrates clear advantages in identity preservation and performance on complex real-world cases.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 50 of its 52 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

50 of 52 references.

References · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  11. 2022Scalable Diffusion Models with Transformerscited by 6,054
  12. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
38 more
  1. 2022Flow Matching for Generative Modelingcited by 5,702
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  10. 2014Conditional Generative Adversarial Netscited by 2,162
  11. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  12. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  13. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  14. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  15. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  16. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
  17. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  18. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  19. 2023Aligning Text-to-Image Models using Human Feedbackcited by 422
  20. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 413
  21. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  22. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 178
  23. 2019PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfercited by 66
  24. 2019LADN: Local Adversarial Disentangling Network for Facial Makeup and De-Makeupcited by 64
  25. 2025SeedEdit 3.0: Fast and High-Quality Generative Image Editingcited by 53
  26. 2024Stable-Makeup: When Real-World Makeup Transfer Meets Diffusion Modelcited by 45
  27. 2017Perceptual Adversarial Networks for Image-to-Image Transformationcited by 42
  28. 2021Lipstick ain't enough: Beyond Color Matching for In-the-Wild Makeup Transfercited by 35
  29. 2022EleGANt: Exquisite and Locally Editable GAN for Makeup Transfercited by 28
  30. 2021SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removalcited by 22
  31. 2022BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfercited by 16
  32. 2024SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Modelscited by 10
  33. 2025Consistent Image Layout Editing with Diffusion Modelscited by 9
  34. 2025CycleDiff: Cycle Diffusion Models for Unpaired Image-to-image Translationcited by 7
  35. 2025FLUX-Makeup: High-Fidelity, Identity-Consistent, and Robust Makeup Transfer via Diffusion Transformercited by 5
  36. 2025From Competition to Synergy: Unlocking Reinforcement Learning for Subject-Driven Image Generationcited by 4
  37. 2025EvoMakeup: High-Fidelity and Controllable Makeup Editing with MakeupQuadcited by 3
  38. 2025FFHQ-Makeup: Paired Synthetic Makeup Dataset with Facial Consistency Across Multiple Stylescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Supervised makeup transfer with a curated dataset: Decoupling identity and makeup features for enhanced transformation
  2. 2026MakeupMirror: Improving Facial Attribute Preservation in Diffusion Models for Makeup Transfer
  3. 2026MagicMakeup: A Region-Controllable Diffusion Transformer for High-Fidelity Makeup-Transfer
  4. 2026AdaptDiff: Adaptive Guidance in Diffusion Models for Diverse and Identity-Consistent Face Synthesis (Student Abstract)
  5. 2023Real-Fake: Effective Training Data Synthesis Through Distribution Matching
  6. 2025Training-Free Diffusion Framework for Stylized Image Generation with Identity Preservation
  7. 2025DreamMakeup: Face Makeup Customization using Latent Diffusion Models
  8. 2026Diff-ID: Identity Consistent Facial Image Generation and Morphing via Diffusion Models