mlsanityData to 7 Oct 2026

Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead

arXiv 2101.02559 ↗First posted 4 Jan 2021cs.CR cs.AI cs.ARcited by 40

Not on the map. Why?

Abstract (arXiv)

Machine Learning (ML) techniques have been rapidly adopted by smart Cyber-Physical Systems (CPS) and Internet-of-Things (IoT) due to their powerful decision-making capabilities. However, they are vulnerable to various security and reliability threats, at both hardware and software levels, that compromise their accuracy. These threats get aggravated in emerging edge ML devices that have stringent constraints in terms of resources (e.g., compute, memory, power/energy), and that therefore cannot employ costly security and reliability measures. Security, reliability, and vulnerability mitigation techniques span from network security measures to hardware protection, with an increased interest towards formal verification of trained ML models. This paper summarizes the prominent vulnerabilities of modern ML systems, highlights successful defenses and mitigation techniques against these vulnerabilities, both at the cloud (i.e., during the ML training phase) and edge (i.e., during the ML inference stage), discusses the implications of a resource-constrained design on the reliability and security of the system, identifies verification methodologies to ensure correct system behavior, and describes open research challenges for building secure and reliable ML systems at both the edge and the cloud.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 74 of its 188 references that are in the map. First posted on arXiv 4 Jan 2021.

Builds on

74 of 188 references.

References · 74

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2016Adversarial examples in the physical worldcited by 1,762
  8. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  9. 2018Sanity Checks for Saliency Mapscited by 1,149
  10. 2016End to End Learning for Self-Driving Carscited by 1,000
  11. 2016Adversarial Machine Learning at Scalecited by 934
  12. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
38 more
  1. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  2. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  3. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  4. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  5. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  6. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 459
  7. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 427
  8. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  9. 2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 328
  10. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 307
  11. 2017Certified Defenses for Data Poisoning Attackscited by 288
  12. 2019Fooling automated surveillance cameras: adversarial patches to attack person detectioncited by 268
  13. 2016Safety Verification of Deep Neural Networkscited by 262
  14. 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 249
  15. 2019Efficient Decision-based Black-box Adversarial Attacks on Face Recognitioncited by 173
  16. 2018Efficient Formal Safety Analysis of Neural Networkscited by 165
  17. 2017Neural Trojanscited by 151
  18. 2018Query-Efficient Hard-label Black-box Attack:An Optimization-based Approachcited by 130
  19. 2017An approach to reachability analysis for feed-forward ReLU neural networkscited by 126
  20. 2018Stealing Hyperparameters in Machine Learningcited by 115
  21. 2017Maximum Resilience of Artificial Neural Networkscited by 97
  22. 2016Measuring Neural Net Robustness with Constraintscited by 90
  23. 2016Towards the Science of Security and Privacy in Machine Learningcited by 89
  24. 2017Output Reachable Set Estimation and Verification for Multi-Layer Neural Networkscited by 79
  25. 2017Verifying Properties of Binarized Deep Neural Networkscited by 77
  26. 2017Natural and Effective Obfuscation by Head Inpaintingcited by 60
  27. 2017CryptoDL: Deep Neural Networks over Encrypted Datacited by 56
  28. 2017SafetyNets: Verifiable Execution of Deep Neural Networks on an Untrusted Cloudcited by 50
  29. 2017NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehiclescited by 48
  30. 2018Maximal Jacobian-based Saliency Map Attackcited by 46
  31. 2018Stealing Neural Networks via Timing Side Channelscited by 42
  32. 2018Chiron: Privacy-preserving Machine Learning as a Servicecited by 38
  33. 2019Certifiably Robust Interpretation in Deep Learningcited by 35
  34. 2019FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving their Fault Tolerance using Clipped Activationcited by 31
  35. 2018I Know What You See: Power Side-Channel Attack on Convolutional Neural Network Acceleratorscited by 26
  36. 2018Analyzing and Mitigating the Impact of Permanent Faults on a Systolic Array Based Neural Network Acceleratorcited by 25
  37. 2019EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAMcited by 24
  38. 2019A framework for the extraction of Deep Neural Networks by leveraging public datacited by 22

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 11; 2023: 12; 2024: 3; 2025: 2; 2026: 1.

40 papers.

Sort:

  1. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  2. 2022Hardware Approximate Techniques for Deep Neural Network Accelerators: A Surveycited by 39
  3. 2023A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacycited by 24
  4. 2023A Systematic Literature Review on Hardware Reliability Assessment Methods for Deep Neural Networkscited by 23
  5. 2021DVS-Attacks: Adversarial Attacks on Dynamic Vision Sensors for Spiking Neural Networkscited by 21
  6. 2021ReSpawn: Energy-Efficient Fault-Tolerance for Spiking Neural Networks considering Unreliable Memoriescited by 20
  7. 2021Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Frameworkcited by 17
  8. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  9. 2021MLDemon: Deployment Monitoring for Machine Learning Systemscited by 14
  10. 2021FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functionscited by 12
  11. 2023DeepAxe: A Framework for Exploration of Approximation and Reliability Trade-offs in DNN Acceleratorscited by 11
  12. 2021R-SNN: An Analysis and Design Methodology for Robustifying Spiking Neural Networks against Adversarial Attacks through Noise Filters for Dynamic Vision Sensorscited by 9
28 more
  1. 2022fakeWeather: Adversarial Attacks for Deep Neural Networks Emulating Weather Conditions on the Camera Lens of Autonomous Systemscited by 8
  2. 2022Isotropic Representation Can Improve Dense Retrievalcited by 8
  3. 2023DeepVigor: Vulnerability Value Ranges and Factors for DNNs' Reliability Assessmentcited by 7
  4. 2021R2F: A Remote Retraining Framework for AIoT Processors with Computing Errorscited by 6
  5. 2022Winograd Convolution: A Perspective from Fault Tolerancecited by 6
  6. 2022Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systemscited by 6
  7. 2023Enhancing Fault Resilience of QNNs by Selective Neuron Splittingcited by 6
  8. 2023Data-Free Hard-Label Robustness Stealing Attackcited by 6
  9. 2022Fault-Tolerant Deep Learning: A Hierarchical Perspectivecited by 5
  10. 2023Special Session: Approximation and Fault Resiliency of DNN Acceleratorscited by 5
  11. 2021Exploiting Vulnerabilities in Deep Neural Networks: Adversarial and Fault-Injection Attackscited by 4
  12. 2022RoHNAS: A Neural Architecture Search Framework with Conjoint Optimization for Adversarial Robustness and Hardware Efficiency of Convolutional and Capsule Networkscited by 4
  13. 2024SSAP: A Shape-Sensitive Adversarial Patch for Comprehensive Disruption of Monocular Depth Estimation in Autonomous Navigation Applicationscited by 4
  14. 2021Bit Error Tolerance Metrics for Binarized Neural Networkscited by 2
  15. 2022Software Engineering Approaches for TinyML based IoT Embedded Vision: A Systematic Literature Reviewcited by 2
  16. 2022Statistical Modeling of Soft Error Influence on Neural Networkscited by 2
  17. 2023Implicit Bilevel Optimization: Differentiating through Bilevel Optimization Programmingcited by 2
  18. 2023Probabilistic Safety Regions Via Finite Families of Scalable Classifierscited by 2
  19. 2021Machine Learning on the COVID-19 Pandemic, Human Mobility and Air Quality: A Reviewcited by 1
  20. 2021On the Robustness of Controlled Deep Reinforcement Learning for Slice Placementcited by 1
  21. 2022EcoFlow: Efficient Convolutional Dataflows for Low-Power Neural Network Acceleratorscited by 1
  22. 2023The Chai Platform's AI Safety Frameworkcited by 1
  23. 2023Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerancecited by 1
  24. 2024DeepVigor+: Scalable and Accurate Semi-Analytical Fault Resilience Analysis for Deep Neural Networkcited by 1
  25. 2026PhD Thesis Summary: Methods for Reliability Assessment and Enhancement of Deep Neural Network Hardware Acceleratorscited by 1
  26. 2023Cross-Layer Optimization for Fault-Tolerant Deep Learningcited by 0
  27. 2025Experimental robustness benchmarking of quantum neural networks on a superconducting quantum processorcited by 0
  28. 2025Ralts: Robust Aggregation for Enhancing Graph Neural Network Resilience on Bit-flip Errorscited by 0