# Adv-CPG: A Customized Portrait Generation Framework with Facial Adversarial Attacks

arXiv: https://arxiv.org/abs/2503.08269 · HTML: https://mlsanity.com/paper/2503.08269/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 60 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding](https://mlsanity.com/paper/2312.04461/) (2023)
- [FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention](https://mlsanity.com/paper/2305.10431/) (2023)
- [HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models](https://mlsanity.com/paper/2307.06949/) (2023)
- [AdvHat: Real-world adversarial attack on ArcFace Face ID system](https://mlsanity.com/paper/1908.08705/) (2019)
- [PuLID: Pure and Lightning ID Customization via Contrastive Alignment](https://mlsanity.com/paper/2404.16022/) (2024)
- [Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognition](https://mlsanity.com/paper/2105.03162/) (2021)
- [SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing](https://mlsanity.com/paper/1906.07927/) (2019)
- [Improving Transferability of Adversarial Patches on Face Recognition with Generative Models](https://mlsanity.com/paper/2106.15058/) (2021)
- [Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer](https://mlsanity.com/paper/2203.03121/) (2022)

## Built on by (most cited first, 14 of 14)

- [Explore How to Inject Beneficial Noise in MLLMs](https://mlsanity.com/paper/2511.12917/) (2025)
- [Rectified Noise: A Generative Model Using Positive-incentive Noise](https://mlsanity.com/paper/2511.07911/) (2025)
- [Laytrol: Preserving Pretrained Knowledge in Layout Control for Multimodal Diffusion Transformers](https://mlsanity.com/paper/2511.07934/) (2025)
- [Affordance Benchmark for MLLMs](https://mlsanity.com/paper/2506.00893/) (2025)
- [AnchorFormer: Differentiable Anchor Attention for Efficient Vision Transformer](https://mlsanity.com/paper/2505.16463/) (2025)
- [The Ever-Evolving Science Exam](https://mlsanity.com/paper/2507.16514/) (2025)
- [Machine Pareidolia: Protecting Facial Image with Emotional Editing](https://mlsanity.com/paper/2603.03665/) (2026)
- [Adv-TGD: Adversarial Text-Guided Diffusion for Face Recognition Impersonation Attacks](https://mlsanity.com/paper/2606.11615/) (2026)
- [CoLM: Collaborative Large Models via A Client-Server Paradigm](https://mlsanity.com/paper/2511.06991/) (2025)
- [SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy](https://mlsanity.com/paper/2602.04994/) (2026)
- [Personalized Face Privacy Protection From a Single Image](https://mlsanity.com/paper/2605.19032/) (2026)
- [Face De-Identification: A Domain-Centric Survey from Capture to Processing](https://mlsanity.com/paper/2607.25926/) (2026)
- [DiffAttack: Evasion Attacks Against Face Recognition via Latent Diffusion Models](https://mlsanity.com/paper/2607.28936/) (2026)
- [SSM-Attack: Learning Where to Perturb for Black-Box Face Privacy](https://mlsanity.com/paper/2610.17323/) (2026)

## Abstract (arXiv)

Recent Customized Portrait Generation (CPG) methods, taking a facial image and a textual prompt as inputs, have attracted substantial attention. Although these methods generate high-fidelity portraits, they fail to prevent the generated portraits from being tracked and misused by malicious face recognition systems. To address this, this paper proposes a Customized Portrait Generation framework with facial Adversarial attacks (Adv-CPG). Specifically, to achieve facial privacy protection, we devise a lightweight local ID encryptor and an encryption enhancer. They implement progressive double-layer encryption protection by directly injecting the target identity and adding additional identity guidance, respectively. Furthermore, to accomplish fine-grained and personalized portrait generation, we develop a multi-modal image customizer capable of generating controlled fine-grained facial features. To the best of our knowledge, Adv-CPG is the first study that introduces facial adversarial attacks into CPG. Extensive experiments demonstrate the superiority of Adv-CPG, e.g., the average attack success rate of the proposed Adv-CPG is 28.1% and 2.86% higher compared to the SOTA noise-based attack methods and unconstrained attack methods, respectively.