# Hierarchical Anti-Aesthetics: Protecting Facial Privacy against Customized Diffusion Models

arXiv: https://arxiv.org/abs/2607.02038 · HTML: https://mlsanity.com/paper/2607.02038/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 13 of its 57 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 13 of 13)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples](https://mlsanity.com/paper/2302.04578/) (2023)
- [Targeted Attack Improves Protection against Unauthorized Diffusion Customization](https://mlsanity.com/paper/2310.04687/) (2023)
- [Personalizing Text-to-Image Generation via Aesthetic Gradients](https://mlsanity.com/paper/2209.12330/) (2022)
- [AgentCanary: A Security Evaluation Framework for Autonomous AI Agents in Real Executable Environments](https://mlsanity.com/paper/2606.10484/) (2026)
- [VMix: Improving Text-to-Image Diffusion Model with Cross-Attention Mixing Control](https://mlsanity.com/paper/2412.20800/) (2024)
- [Exploring Adversarial Transferability between Kolmogorov-arnold Networks](https://mlsanity.com/paper/2503.06276/) (2025)
- [Exposing and Defending the Achilles' Heel of Video Mixture-of-Experts](https://mlsanity.com/paper/2602.01369/) (2026)
- [Anti-Aesthetics: Protecting Facial Privacy against Customized Text-to-Image Synthesis](https://mlsanity.com/paper/2504.12129/) (2025)

## Abstract (arXiv)

The rise of customized diffusion models has fueled a boom in personalized visual content creation, but it also introduces serious risks of malicious misuse, thereby posing threats to personal privacy. Image aesthetics are strongly correlated with human perception of image quality. Motivated by this observation, we address facial privacy protection from a novel aesthetic perspective by degrading the generation quality of maliciously customized models, thus reducing facial identity leakage. Specifically, we propose a Hierarchical Anti-Aesthetics (HAA) framework that exploits aesthetic cues at multiple perceptual levels. HAA consists of two key branches: (1) Global Anti-Aesthetics, which degrades overall aesthetics and generation quality by constructing a global anti-aesthetic reward mechanism and a corresponding loss; and (2) Local Anti-Aesthetics, which disrupts facial identity by using a local anti-aesthetic reward mechanism and loss to guide adversarial perturbations toward facial regions. By integrating both branches, HAA achieves anti-aesthetic degradation from a global to a local level during customized generation. Extensive experiments show that HAA outperforms existing methods in identity removal, providing an effective tool for protecting facial privacy.