# Towards an End-to-End (E2E) Adversarial Learning and Application in the Physical World

arXiv: https://arxiv.org/abs/2501.08258 · HTML: https://mlsanity.com/paper/2501.08258/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 66 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Builds on (most cited first, 30 of 38)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models](https://mlsanity.com/paper/1712.04248/) (2017)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [Simple Black-box Adversarial Attacks](https://mlsanity.com/paper/1905.07121/) (2019)
- [Adversarial Robustness Toolbox v1.0.0](https://mlsanity.com/paper/1807.01069/) (2018)
- [Physical Adversarial Examples for Object Detectors](https://mlsanity.com/paper/1807.07769/) (2018)
- [AdvHat: Real-world adversarial attack on ArcFace Face ID system](https://mlsanity.com/paper/1908.08705/) (2019)
- [ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector](https://mlsanity.com/paper/1804.05810/) (2018)
- [Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors](https://mlsanity.com/paper/1910.14667/) (2019)
- [SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems](https://mlsanity.com/paper/1812.00292/) (2018)
- [DPatch: An Adversarial Patch Attack on Object Detectors](https://mlsanity.com/paper/1806.02299/) (2018)
- [Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognition](https://mlsanity.com/paper/2105.03162/) (2021)
- [Adversarial Texture for Fooling Person Detectors in the Physical World](https://mlsanity.com/paper/2203.03373/) (2022)
- [On Physical Adversarial Patches for Object Detection](https://mlsanity.com/paper/1906.11897/) (2019)
- [Universal Physical Camouflage Attacks on Object Detectors](https://mlsanity.com/paper/1909.04326/) (2019)
- [Adversarial Laser Beam: Effective Physical-World Attack to DNNs in a Blink](https://mlsanity.com/paper/2103.06504/) (2021)
- [Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection](https://mlsanity.com/paper/2112.04532/) (2021)
- [The Translucent Patch: A Physical and Universal Attack on Object Detectors](https://mlsanity.com/paper/2012.12528/) (2020)
- [T-SEA: Transfer-based Self-Ensemble Attack on Object Detection](https://mlsanity.com/paper/2211.09773/) (2022)
- [Fooling thermal infrared pedestrian detectors in real world using small bulbs](https://mlsanity.com/paper/2101.08154/) (2021)
- [DTA: Physical Camouflage Attacks using Differentiable Transformation Network](https://mlsanity.com/paper/2203.09831/) (2022)
- [SLAP: Improving Physical Adversarial Examples with Short-Lived Adversarial Perturbations](https://mlsanity.com/paper/2007.04137/) (2020)
- [Infrared Invisible Clothing:Hiding from Infrared Detectors at Multiple Angles in Real World](https://mlsanity.com/paper/2205.05909/) (2022)
- [Physically Realizable Natural-Looking Clothing Textures Evade Person Detectors via 3D Modeling](https://mlsanity.com/paper/2307.01778/) (2023)
- [DAP: A Dynamic Adversarial Patch for Evading Person Detectors](https://mlsanity.com/paper/2305.11618/) (2023)
- [On adversarial patches: real-world attack on ArcFace-100 face recognition system](https://mlsanity.com/paper/1910.07067/) (2019)

## Abstract (arXiv)

The traditional learning process of patch-based adversarial attacks, conducted in the digital domain and then applied in the physical domain (e.g., via printed stickers), may suffer from reduced performance due to adversarial patches' limited transferability from the digital domain to the physical domain. Given that previous studies have considered using projectors to apply adversarial attacks, we raise the following question: can adversarial learning (i.e., patch generation) be performed entirely in the physical domain with a projector? In this work, we propose the Physical-domain Adversarial Patch Learning Augmentation (PAPLA) framework, a novel end-to-end (E2E) framework that converts adversarial learning from the digital domain to the physical domain using a projector. We evaluate PAPLA across multiple scenarios, including controlled laboratory settings and realistic outdoor environments, demonstrating its ability to ensure attack success compared to conventional digital learning-physical application (DL-PA) methods. We also analyze the impact of environmental factors, such as projection surface color, projector strength, ambient light, distance, and angle of the target object relative to the camera, on the effectiveness of projected patches. Finally, we demonstrate the feasibility of the attack against a parked car and a stop sign in a real-world outdoor environment. Our results show that under specific conditions, E2E adversarial learning in the physical domain eliminates the transferability issue and ensures evasion by object detectors. Finally, we provide insights into the challenges and opportunities of applying adversarial learning in the physical domain and explain where such an approach is more effective than using a sticker.