# Toward Tiny and High-quality Facial Makeup with Data Amplify Learning

arXiv: https://arxiv.org/abs/2403.15033 · HTML: https://mlsanity.com/paper/2403.15033/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 31 of its 42 references that are in the map.
- First posted on arXiv 22 Mar 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [SimSwap: An Efficient Framework For High Fidelity Face Swapping](https://mlsanity.com/paper/2106.06340/) (2021)
- [General Facial Representation Learning in a Visual-Linguistic Manner](https://mlsanity.com/paper/2112.03109/) (2021)
- [PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer](https://mlsanity.com/paper/1909.06956/) (2019)
- [EleGANt: Exquisite and Locally Editable GAN for Makeup Transfer](https://mlsanity.com/paper/2207.09840/) (2022)
- [SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer](https://mlsanity.com/paper/2104.10567/) (2021)
- [Facial Attribute Transformers for Precise and Robust Makeup Transfer](https://mlsanity.com/paper/2104.02894/) (2021)
- [BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfer](https://mlsanity.com/paper/2212.05855/) (2022)
- [CooGAN: A Memory-Efficient Framework for High-Resolution Facial Attribute Editing](https://mlsanity.com/paper/2011.01563/) (2020)
- [Face Beautification: Beyond Makeup Transfer](https://mlsanity.com/paper/1912.03630/) (2019)

## Built on by (most cited first, 10 of 10)

- [BeautyBank: Encoding Facial Makeup in Latent Space](https://mlsanity.com/paper/2411.11231/) (2024)
- [AvatarMakeup: Realistic Makeup Transfer for 3D Animatable Head Avatars](https://mlsanity.com/paper/2507.02419/) (2025)
- [Flux-Sculptor: Text-Driven Rich-Attribute Portrait Editing through Decomposed Spatial Flow Control](https://mlsanity.com/paper/2507.03979/) (2025)
- [FFHQ-Makeup: Paired Synthetic Makeup Dataset with Facial Consistency Across Multiple Styles](https://mlsanity.com/paper/2508.03241/) (2025)
- [Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas](https://mlsanity.com/paper/2404.13944/) (2024)
- [DreamMakeup: Face Makeup Customization using Latent Diffusion Models](https://mlsanity.com/paper/2510.10918/) (2025)
- [Supervised makeup transfer with a curated dataset: Decoupling identity and makeup features for enhanced transformation](https://mlsanity.com/paper/2602.00729/) (2026)
- [AesFormer: Transform Everyday Photos into Beautiful Memories](https://mlsanity.com/paper/2605.22126/) (2026)
- [Towards High-Fidelity, Identity-Preserving Real-Time Makeup Transfer: Decoupling Style Generation](https://mlsanity.com/paper/2509.02445/) (2025)
- [Diffusion-Based Makeup Transfer with Facial Region-Aware Makeup Features](https://mlsanity.com/paper/2603.20012/) (2026)

## Abstract (arXiv)

Contemporary makeup approaches primarily hinge on unpaired learning paradigms, yet they grapple with the challenges of inaccurate supervision (e.g., face misalignment) and sophisticated facial prompts (including face parsing, and landmark detection). These challenges prohibit low-cost deployment of facial makeup models, especially on mobile devices. To solve above problems, we propose a brand-new learning paradigm, termed "Data Amplify Learning (DAL)," alongside a compact makeup model named "TinyBeauty." The core idea of DAL lies in employing a Diffusion-based Data Amplifier (DDA) to "amplify" limited images for the model training, thereby enabling accurate pixel-to-pixel supervision with merely a handful of annotations. Two pivotal innovations in DDA facilitate the above training approach: (1) A Residual Diffusion Model (RDM) is designed to generate high-fidelity detail and circumvent the detail vanishing problem in the vanilla diffusion models; (2) A Fine-Grained Makeup Module (FGMM) is proposed to achieve precise makeup control and combination while retaining face identity. Coupled with DAL, TinyBeauty necessitates merely 80K parameters to achieve a state-of-the-art performance without intricate face prompts. Meanwhile, TinyBeauty achieves a remarkable inference speed of up to 460 fps on the iPhone 13. Extensive experiments show that DAL can produce highly competitive makeup models using only 5 image pairs.