Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
Not on the map. Why?
Abstract (arXiv)
CleverHans is a software library that provides standardized reference implementations of adversarial example construction techniques and adversarial training. The library may be used to develop more robust machine learning models and to provide standardized benchmarks of models' performance in the adversarial setting. Benchmarks constructed without a standardized implementation of adversarial example construction are not comparable to each other, because a good result may indicate a robust model or it may merely indicate a weak implementation of the adversarial example construction procedure. This technical report is structured as follows. Section 1 provides an overview of adversarial examples in machine learning and of the CleverHans software. Section 2 presents the core functionalities of the library: namely the attacks based on adversarial examples and defenses to improve the robustness of machine learning models to these attacks. Section 3 describes how to report benchmark results using the library. Section 4 describes the versioning system.
Key facts. Cited by 124 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Oct 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 36; 2022: 34; 2023: 20; 2024: 16; 2025: 13; 2026: 5.
124 papers.
Sort:
By area:
- 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
- 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 85
- 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
- 2021Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Imagescited by 58
- 2021Person Re-identification Method Based on Color Attack and Joint Defencecited by 56
- 2022CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Modelscited by 53
- 2022Assaying Out-Of-Distribution Generalization in Transfer Learningcited by 53
- 2023Fast Decision Boundary based Out-of-Distribution Detectorcited by 49
- 2021Adversarial Attacks for Tabular Data: Application to Fraud Detection and Imbalanced Datacited by 45
- 2021The Threat of Offensive AI to Organizationscited by 38
- 2022LoMar: A Local Defense Against Poisoning Attack on Federated Learningcited by 31
- 2021FoveaTer: Foveated Transformer for Image Classificationcited by 28
38 more
- 2023Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanationscited by 27
- 2023Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspectivecited by 20
- 2022Efficiently Training Low-Curvature Neural Networkscited by 19
- 2021Adversarial Attacks against Windows PE Malware Detection: A Survey of the State-of-the-Artcited by 18
- 2025LLM-Safety Evaluations Lack Robustnesscited by 18
- 2022Can we achieve robustness from data alone?cited by 16
- 2024AttackBench: Evaluating Gradient-based Attacks for Adversarial Examplescited by 14
- 2021Building Reliable Explanations of Unreliable Neural Networks: Locally Smoothing Perspective of Model Interpretationcited by 12
- 2022Topology and geometry of data manifold in deep learningcited by 12
- 2022How Does Frequency Bias Affect the Robustness of Neural Image Classifiers against Common Corruption and Adversarial Perturbations?cited by 11
- 2021Detecting AutoAttack Perturbations in the Frequency Domaincited by 10
- 2021Launching Adversarial Attacks against Network Intrusion Detection Systems for IoTcited by 9
- 2022Tools and Practices for Responsible AI Engineeringcited by 9
- 2021Evaluating Document Coherence Modellingcited by 8
- 2021PDPGD: Primal-Dual Proximal Gradient Descent Adversarial Attackcited by 8
- 2021Adversarial robustness for latent models: Revisiting the robust-standard accuracies tradeoffcited by 8
- 2021Detecting Localized Adversarial Examples: A Generic Approach using Critical Region Analysiscited by 7
- 2021Perceptually Constrained Adversarial Attackscited by 7
- 2021Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutionscited by 7
- 2022A Survey on AI Sustainability: Emerging Trends on Learning Algorithms and Research Challengescited by 7
- 2022Adversarial Framework with Certified Robustness for Time-Series Domain via Statistical Featurescited by 7
- 2022SoK: Modeling Explainability in Security Analytics for Interpretability, Trustworthiness, and Usabilitycited by 7
- 2024Unveiling the Hidden Structure of Self-Attention via Kernel Principal Component Analysiscited by 7
- 2021Evaluating the Cybersecurity Risk of Real World, Machine Learning Production Systemscited by 6
- 2023Optimization and Optimizers for Adversarial Robustnesscited by 6
- 2023Towards Evaluating Transfer-based Attacks Systematically, Practically, and Fairlycited by 6
- 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Researchcited by 6
- 2021On managing vulnerabilities in AI/ML systemscited by 5
- 2021Adversarial Information Bottleneckcited by 5
- 2022Subverting Fair Image Search with Generative Adversarial Perturbationscited by 5
- 2022Dynamic Time Warping based Adversarial Framework for Time-Series Domaincited by 5
- 2022Towards an Awareness of Time Series Anomaly Detection Models' Adversarial Vulnerabilitycited by 5
- 2021Robust Vision-Based Cheat Detection in Competitive Gamingcited by 4
- 2021Attribution of Gradient Based Adversarial Attacks for Reverse Engineering of Deceptionscited by 4
- 2022On Certifying and Improving Generalization to Unseen Domainscited by 4
- 2023SecurityNet: Assessing Machine Learning Vulnerabilities on Public Modelscited by 4
- 2021ExAD: An Ensemble Approach for Explanation-based Adversarial Detectioncited by 3
- 2021Making Generated Images Hard To Spot: A Transferable Attack On Synthetic Image Detectorscited by 3
50 most cited of 124 All 124 (JSON)
- 2026VCR-Bench: A Modular Open-Source Benchmark for Video Classification Robustness6 Oct 2026
- 2026Trustworthy Data- and ML-Ops for Intelligent Transportation Systems and Logistics1 Oct 2026
- 2026Imperceptible and Reversible Adversarial Examples against Vision-Language Models for Privacy Protection11 Jul 2026
- 2026RoAd-RL: A Unified Library and Benchmark for Robust Adversarial Reinforcement Learning29 Jun 2026
- 2026AI-Driven Cybersecurity Threats: A Survey of Emerging Risks and Defensive Strategies6 Jan 2026
- 2025Toward Secure and Compliant AI: Organizational Standards and Protocols for NLP Model Lifecycle Management26 Dec 2025
- 2025Adversarial Attacks Against Deep Learning-Based Radio Frequency Fingerprint Identification12 Dec 2025
- 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research6 Nov 2025
- 2025PRBench: A Standardized Probabilistic Robustness Benchmark3 Nov 2025
- 2025Amulet: a Python Library for Assessing Interactions Among ML Defenses and Risks15 Sep 2025
- 2025Securing AI Systems: A Guide to Known Attacks and Impacts29 Jun 2025
- 2025Towards Secure MLOps: Surveying Attacks, Mitigation Strategies, and Research Challenges30 May 2025
38 more
- 2025Black-box Adversarial Attacks on CNN-based SLAM Algorithms30 May 2025
- 2025Anomaly Detection Based on Critical Paths for Deep Neural Networks20 May 2025
- 2025Leveraging Generalizability of Image-to-Image Translation for Enhanced Adversarial Defense2 Apr 2025
- 2025Securing Virtual Reality Experiences: Unveiling and Tackling Cybersickness Attacks with Explainable AI17 Mar 2025
- 2025LLM-Safety Evaluations Lack Robustness4 Mar 2025
- 2025PAR-AdvGAN: Improving Adversarial Attack Capability with Progressive Auto-Regression AdvGAN16 Feb 2025
- 2024Neural Fingerprints for Adversarial Attack Detection7 Nov 2024
- 2024A constrained optimization approach to improve robustness of neural networks18 Sep 2024
- 2024Golden Ratio Search: A Low-Power Adversarial Attack for Deep Learning based Modulation Classification17 Sep 2024
- 2024AdvSecureNet: A Python Toolkit for Adversarial Machine Learning4 Sep 2024
- 2024Label Augmentation for Neural Networks Robustness4 Aug 2024
- 2024Bayesian Low-Rank LeArning (Bella): A Practical Approach to Bayesian Neural Networks30 Jul 2024
- 2024NeuSemSlice: Towards Effective DNN Model Maintenance via Neuron-level Semantic Slicing26 Jul 2024
- 2024A Survey on the Application of Generative Adversarial Networks in Cybersecurity: Prospective, Direction and Open Research Scopes11 Jul 2024
- 2024Unveiling the Hidden Structure of Self-Attention via Kernel Principal Component Analysis19 Jun 2024
- 2024VQUNet: Vector Quantization U-Net for Defending Adversarial Atacks by Regularizing Unwanted Noise5 Jun 2024
- 2024AttackBench: Evaluating Gradient-based Attacks for Adversarial Examples30 Apr 2024
- 2024On adversarial training and the 1 Nearest Neighbor classifier9 Apr 2024
- 2024David and Goliath: An Empirical Evaluation of Attacks and Defenses for QNNs at the Deep Edge8 Apr 2024
- 2024Versatile Defense Against Adversarial Attacks on Image Recognition13 Mar 2024
- 2024Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness21 Feb 2024
- 2024Black-Box Access is Insufficient for Rigorous AI Audits25 Jan 2024
- 2023Fast Decision Boundary based Out-of-Distribution Detector15 Dec 2023
- 2023Adversarial Purification of Information Masking26 Nov 2023
- 2023When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence23 Nov 2023
- 2023On The Relationship Between Universal Adversarial Attacks And Sparse Representations14 Nov 2023
- 2023Towards Evaluating Transfer-based Attacks Systematically, Practically, and Fairly2 Nov 2023
- 2023SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models19 Oct 2023
- 2023On Continuity of Robust and Accurate Classifiers29 Sep 2023
- 2023What Learned Representations and Influence Functions Can Tell Us About Adversarial Examples19 Sep 2023
- 2023RecRec: Algorithmic Recourse for Recommender Systems28 Aug 2023
- 2023Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations16 Aug 2023
- 2023A reading survey on adversarial machine learning: Adversarial attacks and their understanding7 Aug 2023
- 2023URET: Universal Robustness Evaluation Toolkit (for Evasion)3 Aug 2023
- 2023Scaling Model Checking for DNN Analysis via State-Space Reduction and Input Segmentation (Extended Version)29 Jun 2023
- 2023Global-Local Processing in Convolutional Neural Networks14 Jun 2023
- 2023Evaluating Adversarial Robustness on Document Image Classification24 Apr 2023
- 2023Optimization and Optimizers for Adversarial Robustness23 Mar 2023
50 newest of 124 All 124 (JSON)
- 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 85
- 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Researchcited by 6
- 2023SecurityNet: Assessing Machine Learning Vulnerabilities on Public Modelscited by 4
- 2021ActiveGuard: An Active DNN IP Protection Technique via Adversarial Examplescited by 1
- 2025AdversariaLLM: A Unified and Modular Toolbox for LLM Robustness Research6 Nov 2025
- 2023SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models19 Oct 2023
- 2021ActiveGuard: An Active DNN IP Protection Technique via Adversarial Examples2 Mar 2021
- 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models4 Feb 2021
- 2025LLM-Safety Evaluations Lack Robustnesscited by 18
- 2025LLM-Safety Evaluations Lack Robustness4 Mar 2025
- 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
- 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
- 2021Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Imagescited by 58
- 2021Person Re-identification Method Based on Color Attack and Joint Defencecited by 56
- 2022CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Modelscited by 53
- 2022Assaying Out-Of-Distribution Generalization in Transfer Learningcited by 53
- 2023Fast Decision Boundary based Out-of-Distribution Detectorcited by 49
- 2021Adversarial Attacks for Tabular Data: Application to Fraud Detection and Imbalanced Datacited by 45
- 2021The Threat of Offensive AI to Organizationscited by 38
- 2022LoMar: A Local Defense Against Poisoning Attack on Federated Learningcited by 31
- 2021FoveaTer: Foveated Transformer for Image Classificationcited by 28
- 2023Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanationscited by 27
12 most cited of 117 All 117 (JSON)
- 2026VCR-Bench: A Modular Open-Source Benchmark for Video Classification Robustness6 Oct 2026
- 2026Trustworthy Data- and ML-Ops for Intelligent Transportation Systems and Logistics1 Oct 2026
- 2026AI-Driven Cybersecurity Threats: A Survey of Emerging Risks and Defensive Strategies6 Jan 2026
- 2025Toward Secure and Compliant AI: Organizational Standards and Protocols for NLP Model Lifecycle Management26 Dec 2025
- 2025Adversarial Attacks Against Deep Learning-Based Radio Frequency Fingerprint Identification12 Dec 2025
- 2025PRBench: A Standardized Probabilistic Robustness Benchmark3 Nov 2025
- 2025Amulet: a Python Library for Assessing Interactions Among ML Defenses and Risks15 Sep 2025
- 2025Securing AI Systems: A Guide to Known Attacks and Impacts29 Jun 2025
- 2025Towards Secure MLOps: Surveying Attacks, Mitigation Strategies, and Research Challenges30 May 2025
- 2025Black-box Adversarial Attacks on CNN-based SLAM Algorithms30 May 2025
- 2025Anomaly Detection Based on Critical Paths for Deep Neural Networks20 May 2025
- 2025Leveraging Generalizability of Image-to-Image Translation for Enhanced Adversarial Defense2 Apr 2025
12 newest of 117 All 117 (JSON)