URET: Universal Robustness Evaluation Toolkit (for Evasion)
Not on the map. Why?
Abstract (arXiv)
Machine learning models are known to be vulnerable to adversarial evasion attacks as illustrated by image classification models. Thoroughly understanding such attacks is critical in order to ensure the safety and robustness of critical AI tasks. However, most evasion attacks are difficult to deploy against a majority of AI systems because they have focused on image domain with only few constraints. An image is composed of homogeneous, numerical, continuous, and independent features, unlike many other input types to AI systems used in practice. Furthermore, some input types include additional semantic and functional constraints that must be observed to generate realistic adversarial inputs. In this work, we propose a new framework to enable the generation of adversarial inputs irrespective of the input type and task domain. Given an input and a set of pre-defined input transformations, our framework discovers a sequence of transformations that result in a semantically correct and functional adversarial input. We demonstrate the generality of our approach on several diverse machine learning tasks with various input representations. We also show the importance of generating adversarial examples as they enable the deployment of mitigation techniques.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 33 of its 50 references that are in the map. First posted on arXiv 3 Aug 2023.
Builds on
33 of 50 references.
References · 33
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2013Intriguing properties of neural networkscited by 5,155
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
- 2015The Limitations of Deep Learning in Adversarial Settingscited by 790
- 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 678
- 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 666
- 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
21 more
- 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
- 2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 333
- 2018Adversarial Robustness Toolbox v1.0.0cited by 215
- 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 205
- 2018EMBER: An Open Dataset for Training Static PE Malware Machine Learning Modelscited by 165
- 2018ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detectorcited by 159
- 2016Technical Report on the CleverHans v2.1.0 Adversarial Examples Librarycited by 124
- 2020Adversarial Machine Learning -- Industry Perspectivescited by 122
- 2021Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attackscited by 101
- 2017Generating Adversarial Malware Examples for Black-Box Attacks Based on GANcited by 75
- 2019advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorchcited by 69
- 2018Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learningcited by 61
- 2018Exploring Adversarial Examples in Malware Detectioncited by 42
- 2022AugLy: Data Augmentations for Robustnesscited by 34
- 2018Invisible Mask: Practical Attacks on Face Recognition with Infraredcited by 33
- 2018Adversarial Deep Learning for Robust Detection of Binary Encoded Malwarecited by 32
- 2019Malware Makeover: Breaking ML-based Static Analysis by Modifying Executable Bytescited by 31
- 2020Advbox: a toolbox to generate adversarial examples that fool neural networkscited by 17
- 2019ALOHA: Auxiliary Loss Optimization for Hypothesis Augmentationcited by 9
- 2020Adversarial Evaluation of Multimodal Models under Realistic Gray Box Assumptioncited by 8
- 2018Evading classifiers in discrete domains with provable optimality guaranteescited by 7
Built on by
Citing papers per year: 2024: 2; 2025: 1.
3 papers.