# Sibling-Attack: Rethinking Transferable Adversarial Attacks against Face Recognition

arXiv: https://arxiv.org/abs/2303.12512 · HTML: https://mlsanity.com/paper/2303.12512/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 77 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 30 of 51)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks](https://mlsanity.com/paper/1908.06281/) (2019)
- [Enhancing the Transferability of Adversarial Attacks through Variance Tuning](https://mlsanity.com/paper/2103.15571/) (2021)
- [MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices](https://mlsanity.com/paper/1804.07573/) (2018)
- [Efficient Decision-based Black-box Adversarial Attacks on Face Recognition](https://mlsanity.com/paper/1904.04433/) (2019)
- [AdvHat: Real-world adversarial attack on ArcFace Face ID system](https://mlsanity.com/paper/1908.08705/) (2019)
- [Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognition](https://mlsanity.com/paper/2105.03162/) (2021)
- [SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistency](https://mlsanity.com/paper/2302.03251/) (2023)
- [Improving Adversarial Transferability via Neuron Attribution-Based Attacks](https://mlsanity.com/paper/2204.00008/) (2022)
- [Regularized Fine-grained Meta Face Anti-spoofing](https://mlsanity.com/paper/1911.10771/) (2019)

## Built on by (most cited first, 30 of 34)

- [SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation](https://mlsanity.com/paper/2312.04913/) (2023)
- [Analysing the Robustness of Vision-Language-Models to Common Corruptions](https://mlsanity.com/paper/2504.13690/) (2025)
- [White-Box Multi-Objective Adversarial Attack on Dialogue Generation](https://mlsanity.com/paper/2305.03655/) (2023)
- [Dynamic Transformers Provide a False Sense of Efficiency](https://mlsanity.com/paper/2305.12228/) (2023)
- [Artwork Protection Against Neural Style Transfer Using Locally Adaptive Adversarial Color Attack](https://mlsanity.com/paper/2401.09673/) (2024)
- [Research on the Application of Computer Vision Based on Deep Learning in Autonomous Driving Technology](https://mlsanity.com/paper/2406.00490/) (2024)
- [Rethinking Impersonation and Dodging Attacks on Face Recognition Systems](https://mlsanity.com/paper/2401.08903/) (2024)
- [RED: A Systematic Real-Time Scheduling Approach for Robotic Environmental Dynamics](https://mlsanity.com/paper/2308.15368/) (2023)
- [R^3: On-device Real-Time Deep Reinforcement Learning for Autonomous Robotics](https://mlsanity.com/paper/2308.15039/) (2023)
- [Improving Visual Quality and Transferability of Adversarial Attacks on Face Recognition Simultaneously with Adversarial Restoration](https://mlsanity.com/paper/2309.01582/) (2023)
- [Adversarial Attacks on Both Face Recognition and Face Anti-spoofing Models](https://mlsanity.com/paper/2405.16940/) (2024)
- [Transferable Adversarial Attacks against ASR](https://mlsanity.com/paper/2411.09220/) (2024)
- [Improving the Transferability of Adversarial Attacks on Face Recognition with Diverse Parameters Augmentation](https://mlsanity.com/paper/2411.15555/) (2024)
- [Diagnosing Corruption-Induced Reliability Failures in Vision-Language Models](https://mlsanity.com/paper/2511.19032/) (2025)
- [DIP-Watermark: A Double Identity Protection Method Based on Robust Adversarial Watermark](https://mlsanity.com/paper/2404.14693/) (2024)
- [A Multi-task Adversarial Attack Against Face Authentication](https://mlsanity.com/paper/2408.08205/) (2024)
- [Artificial Immune System of Secure Face Recognition Against Adversarial Attacks](https://mlsanity.com/paper/2406.18144/) (2024)
- [Rethinking the Threat and Accessibility of Adversarial Attacks against Face Recognition Systems](https://mlsanity.com/paper/2407.08514/) (2024)
- [Non-Adaptive Adversarial Face Generation](https://mlsanity.com/paper/2507.12107/) (2025)
- [Attacking All Tasks at Once Using Adversarial Examples in Multi-Task Learning](https://mlsanity.com/paper/2305.12066/) (2023)
- [Latent Code Augmentation Based on Stable Diffusion for Data-free Substitute Attacks](https://mlsanity.com/paper/2307.12872/) (2023)
- [Imperceptible Physical Attack against Face Recognition Systems via LED Illumination Modulation](https://mlsanity.com/paper/2307.13294/) (2023)
- [Adversarially Robust Deepfake Detection via Adversarial Feature Similarity Learning](https://mlsanity.com/paper/2403.08806/) (2024)
- [Adversarial Identity Injection for Semantic Face Image Synthesis](https://mlsanity.com/paper/2404.10408/) (2024)
- [Learning to Learn Transferable Generative Attack for Person Re-Identification](https://mlsanity.com/paper/2409.04208/) (2024)
- [ErasableMask: A Robust and Erasable Privacy Protection Scheme against Black-box Face Recognition Models](https://mlsanity.com/paper/2412.17038/) (2024)
- [Undermining Image and Text Classification Algorithms Using Adversarial Attacks](https://mlsanity.com/paper/2411.03348/) (2024)
- [MORE: Multi-Objective Adversarial Attacks on Speech Recognition](https://mlsanity.com/paper/2601.01852/) (2026)
- [Devling into Adversarial Transferability on Image Classification: Review, Benchmark, and Evaluation](https://mlsanity.com/paper/2602.23117/) (2026)
- [From Pixels to Reality: Physical-Digital Patch Attacks on Real-World Camera](https://mlsanity.com/paper/2603.28425/) (2026)

## Abstract (arXiv)

A hard challenge in developing practical face recognition (FR) attacks is due to the black-box nature of the target FR model, i.e., inaccessible gradient and parameter information to attackers. While recent research took an important step towards attacking black-box FR models through leveraging transferability, their performance is still limited, especially against online commercial FR systems that can be pessimistic (e.g., a less than 50% ASR--attack success rate on average). Motivated by this, we present Sibling-Attack, a new FR attack technique for the first time explores a novel multi-task perspective (i.e., leveraging extra information from multi-correlated tasks to boost attacking transferability). Intuitively, Sibling-Attack selects a set of tasks correlated with FR and picks the Attribute Recognition (AR) task as the task used in Sibling-Attack based on theoretical and quantitative analysis. Sibling-Attack then develops an optimization framework that fuses adversarial gradient information through (1) constraining the cross-task features to be under the same space, (2) a joint-task meta optimization framework that enhances the gradient compatibility among tasks, and (3) a cross-task gradient stabilization method which mitigates the oscillation effect during attacking. Extensive experiments demonstrate that Sibling-Attack outperforms state-of-the-art FR attack techniques by a non-trivial margin, boosting ASR by 12.61% and 55.77% on average on state-of-the-art pre-trained FR models and two well-known, widely used commercial FR systems.