Foolbox: A Python toolbox to benchmark the robustness of machine learning models
Not on the map. Why?
Abstract (arXiv)
Even todays most advanced machine learning models are easily fooled by almost imperceptible perturbations of their inputs. Foolbox is a new Python package to generate such adversarial perturbations and to quantify and compare the robustness of machine learning models. It is build around the idea that the most comparable robustness measure is the minimum perturbation needed to craft an adversarial example. To this end, Foolbox provides reference implementations of most published adversarial attack methods alongside some new ones, all of which perform internal hyperparameter tuning to find the minimum adversarial perturbation. Additionally, Foolbox interfaces with most popular deep learning frameworks such as PyTorch, Keras, TensorFlow, Theano and MXNet and allows different adversarial criteria such as targeted misclassification and top-k misclassification as well as different distance measures. The code is licensed under the MIT license and is openly available at https://github.com/bethgelab/foolbox . The most up-to-date documentation can be found at http://foolbox.readthedocs.io .
Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jul 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 5; 2022: 9; 2023: 2; 2024: 2; 2025: 1.
19 papers.
Sort:
- 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
- 2021Demystifying the Transferability of Adversarial Attacks in Computer Networkscited by 9
- 2021An Orthogonal Classifier for Improving the Adversarial Robustness of Neural Networkscited by 7
- 2022ROOM: Adversarial Machine Learning Attacks Under Real-Time Constraintscited by 5
- 2022Software Testing for Machine Learningcited by 5
- 2022Resisting Deep Learning Models Against Adversarial Attack Transferability via Feature Randomizationcited by 5
- 2024Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbationcited by 5
- 2021SEC4SR: A Security Analysis Platform for Speaker Recognitioncited by 3
- 2022Robust Sensible Adversarial Learning of Deep Neural Networks for Image Classificationcited by 2
- 2021Fortify Machine Learning Production Systems: Detect and Classify Adversarial Attackscited by 1
- 2021Defensive Tensorizationcited by 1
- 2022From Environmental Sound Representation to Robustness of 2D CNN Models Against Adversarial Attackscited by 1
7 more
- 2022Trace and Detect Adversarial Attacks on CNNs using Feature Response Mapscited by 1
- 2024CARE: Ensemble Adversarial Robustness Evaluation Against Adaptive Attackers for Security Applicationscited by 1
- 2025PRBench: A Standardized Probabilistic Robustness Benchmarkcited by 1
- 2022Understanding Adversarial Robustness from Feature Maps of Convolutional Layerscited by 0
- 2022Attacking Compressed Vision Transformerscited by 0
- 2022AccelAT: A Framework for Accelerating the Adversarial Training of Deep Neural Networks through Accuracy Gradientcited by 0
- 2023Unscrambling the Rectification of Adversarial Attacks Transferability across Computer Networkscited by 0
- 2025PRBench: A Standardized Probabilistic Robustness Benchmark3 Nov 2025
- 2024Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation12 Oct 2024
- 2024CARE: Ensemble Adversarial Robustness Evaluation Against Adaptive Attackers for Security Applications20 Jan 2024
- 2023Unscrambling the Rectification of Adversarial Attacks Transferability across Computer Networks26 Oct 2023
- 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking28 Feb 2023
- 2022AccelAT: A Framework for Accelerating the Adversarial Training of Deep Neural Networks through Accuracy Gradient13 Oct 2022
- 2022Attacking Compressed Vision Transformers28 Sep 2022
- 2022Resisting Deep Learning Models Against Adversarial Attack Transferability via Feature Randomization11 Sep 2022
- 2022Trace and Detect Adversarial Attacks on CNNs using Feature Response Maps24 Aug 2022
- 2022Robust Sensible Adversarial Learning of Deep Neural Networks for Image Classification20 May 2022
- 2022Software Testing for Machine Learning30 Apr 2022
- 2022From Environmental Sound Representation to Robustness of 2D CNN Models Against Adversarial Attacks14 Apr 2022
7 more
- 2022Understanding Adversarial Robustness from Feature Maps of Convolutional Layers25 Feb 2022
- 2022ROOM: Adversarial Machine Learning Attacks Under Real-Time Constraints5 Jan 2022
- 2021Defensive Tensorization26 Oct 2021
- 2021Demystifying the Transferability of Adversarial Attacks in Computer Networks9 Oct 2021
- 2021SEC4SR: A Security Analysis Platform for Speaker Recognition4 Sep 2021
- 2021An Orthogonal Classifier for Improving the Adversarial Robustness of Neural Networks19 May 2021
- 2021Fortify Machine Learning Production Systems: Detect and Classify Adversarial Attacks19 Feb 2021
- 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
- 2021Demystifying the Transferability of Adversarial Attacks in Computer Networkscited by 9
- 2021An Orthogonal Classifier for Improving the Adversarial Robustness of Neural Networkscited by 7
- 2022ROOM: Adversarial Machine Learning Attacks Under Real-Time Constraintscited by 5
- 2022Software Testing for Machine Learningcited by 5
- 2022Resisting Deep Learning Models Against Adversarial Attack Transferability via Feature Randomizationcited by 5
- 2024Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbationcited by 5
- 2021SEC4SR: A Security Analysis Platform for Speaker Recognitioncited by 3
- 2022Robust Sensible Adversarial Learning of Deep Neural Networks for Image Classificationcited by 2
- 2021Fortify Machine Learning Production Systems: Detect and Classify Adversarial Attackscited by 1
- 2021Defensive Tensorizationcited by 1
- 2022From Environmental Sound Representation to Robustness of 2D CNN Models Against Adversarial Attackscited by 1
12 most cited of 19 All 19 (JSON)
- 2025PRBench: A Standardized Probabilistic Robustness Benchmark3 Nov 2025
- 2024Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation12 Oct 2024
- 2024CARE: Ensemble Adversarial Robustness Evaluation Against Adaptive Attackers for Security Applications20 Jan 2024
- 2023Unscrambling the Rectification of Adversarial Attacks Transferability across Computer Networks26 Oct 2023
- 2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking28 Feb 2023
- 2022AccelAT: A Framework for Accelerating the Adversarial Training of Deep Neural Networks through Accuracy Gradient13 Oct 2022
- 2022Attacking Compressed Vision Transformers28 Sep 2022
- 2022Resisting Deep Learning Models Against Adversarial Attack Transferability via Feature Randomization11 Sep 2022
- 2022Trace and Detect Adversarial Attacks on CNNs using Feature Response Maps24 Aug 2022
- 2022Robust Sensible Adversarial Learning of Deep Neural Networks for Image Classification20 May 2022
- 2022Software Testing for Machine Learning30 Apr 2022
- 2022From Environmental Sound Representation to Robustness of 2D CNN Models Against Adversarial Attacks14 Apr 2022
12 newest of 19 All 19 (JSON)