# Adversarial Relighting Against Face Recognition

arXiv: https://arxiv.org/abs/2108.07920 · HTML: https://mlsanity.com/paper/2108.07920/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 69 references that are in the map.
- First posted on arXiv 18 Aug 2021.

## Builds on (most cited first, 30 of 35)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection](https://mlsanity.com/paper/2001.00179/) (2020)
- [AdvHat: Real-world adversarial attack on ArcFace Face ID system](https://mlsanity.com/paper/1908.08705/) (2019)
- [Single Image Portrait Relighting](https://mlsanity.com/paper/1905.00824/) (2019)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wild](https://mlsanity.com/paper/1712.01261/) (2017)
- [RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation](https://mlsanity.com/paper/1910.11515/) (2019)
- [SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing](https://mlsanity.com/paper/1906.07927/) (2019)
- [Unrestricted Adversarial Examples via Semantic Manipulation](https://mlsanity.com/paper/1904.06347/) (2019)
- [Towards Transferable Adversarial Attack against Deep Face Recognition](https://mlsanity.com/paper/2004.05790/) (2020)
- [Improving Transferability of Adversarial Patches on Face Recognition with Generative Models](https://mlsanity.com/paper/2106.15058/) (2021)
- [DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms](https://mlsanity.com/paper/2006.07634/) (2020)
- [AdvFaces: Adversarial Face Synthesis](https://mlsanity.com/paper/1908.05008/) (2019)
- [Adv-watermark: A Novel Watermark Perturbation for Adversarial Examples](https://mlsanity.com/paper/2008.01919/) (2020)
- [Towards High Fidelity Face Relighting with Realistic Shadows](https://mlsanity.com/paper/2104.00825/) (2021)
- [Unravelling Robustness of Deep Learning based Face Recognition Against Adversarial Attacks](https://mlsanity.com/paper/1803.00401/) (2018)
- [AdvHaze: Adversarial Haze Attack](https://mlsanity.com/paper/2104.13673/) (2021)
- [Robust Face Recognition via Multimodal Deep Face Representation](https://mlsanity.com/paper/1509.00244/) (2015)
- [On the Robustness of Face Recognition Algorithms Against Attacks and Bias](https://mlsanity.com/paper/2002.02942/) (2020)
- [Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack](https://mlsanity.com/paper/2007.07097/) (2020)

## Built on by (most cited first, 16 of 16)

- [SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments](https://mlsanity.com/paper/2412.00114/) (2024)
- [ALA: Naturalness-aware Adversarial Lightness Attack](https://mlsanity.com/paper/2201.06070/) (2022)
- [ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks](https://mlsanity.com/paper/2111.13330/) (2021)
- [A Survey on Physical Adversarial Attacks against Face Recognition Systems](https://mlsanity.com/paper/2410.16317/) (2024)
- [Benchmarking Shadow Removal for Facial Landmark Detection and Beyond](https://mlsanity.com/paper/2111.13790/) (2021)
- [Benchmarking Out-of-Distribution Generalization Capabilities of DNN-based Encoding Models for the Ventral Visual Cortex](https://mlsanity.com/paper/2406.16935/) (2024)
- [Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machines](https://mlsanity.com/paper/2511.14440/) (2025)
- [Masked Faces with Faced Masks](https://mlsanity.com/paper/2201.06427/) (2022)
- [Improving generalization by mimicking the human visual diet](https://mlsanity.com/paper/2206.07802/) (2022)
- [Non-Uniform Illumination Attack for Fooling Convolutional Neural Networks](https://mlsanity.com/paper/2409.03458/) (2024)
- [Take Fake as Real: Realistic-like Robust Black-box Adversarial Attack to Evade AIGC Detection](https://mlsanity.com/paper/2412.06727/) (2024)
- [Light as Deception: GPT-driven Natural Relighting Against Vision-Language Pre-training Models](https://mlsanity.com/paper/2505.24227/) (2025)
- [RetouchUAA: Unconstrained Adversarial Attack via Image Retouching](https://mlsanity.com/paper/2311.16478/) (2023)
- [AngleRoCL: Angle-Robust Concept Learning for Physically View-Invariant T2I Adversarial Patches](https://mlsanity.com/paper/2506.09538/) (2025)
- [Investigating Adversarial Robustness against Preprocessing used in Blackbox Face Recognition](https://mlsanity.com/paper/2510.17169/) (2025)
- [Face De-Identification: A Domain-Centric Survey from Capture to Processing](https://mlsanity.com/paper/2607.25926/) (2026)

## Abstract (arXiv)

Deep face recognition (FR) has achieved significantly high accuracy on several challenging datasets and fosters successful real-world applications, even showing high robustness to the illumination variation that is usually regarded as a main threat to the FR system. However, in the real world, illumination variation caused by diverse lighting conditions cannot be fully covered by the limited face dataset. In this paper, we study the threat of lighting against FR from a new angle, i.e., adversarial attack, and identify a new task, i.e., adversarial relighting. Given a face image, adversarial relighting aims to produce a naturally relighted counterpart while fooling the state-of-the-art deep FR methods. To this end, we first propose the physical modelbased adversarial relighting attack (ARA) denoted as albedoquotient-based adversarial relighting attack (AQ-ARA). It generates natural adversarial light under the physical lighting model and guidance of FR systems and synthesizes adversarially relighted face images. Moreover, we propose the auto-predictive adversarial relighting attack (AP-ARA) by training an adversarial relighting network (ARNet) to automatically predict the adversarial light in a one-step manner according to different input faces, allowing efficiency-sensitive applications. More importantly, we propose to transfer the above digital attacks to physical ARA (PhyARA) through a precise relighting device, making the estimated adversarial lighting condition reproducible in the real world. We validate our methods on three state-of-the-art deep FR methods, i.e., FaceNet, ArcFace, and CosFace, on two public datasets. The extensive and insightful results demonstrate our work can generate realistic adversarial relighted face images fooling face recognition tasks easily, revealing the threat of specific light directions and strengths.