mlsanityData to 7 Oct 2026

NeuralSentinel: Safeguarding Neural Network Reliability and Trustworthiness

arXiv 2402.07506 ↗First posted 12 Feb 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The usage of Artificial Intelligence (AI) systems has increased exponentially, thanks to their ability to reduce the amount of data to be analyzed, the user efforts and preserving a high rate of accuracy. However, introducing this new element in the loop has converted them into attacked points that can compromise the reliability of the systems. This new scenario has raised crucial challenges regarding the reliability and trustworthiness of the AI models, as well as about the uncertainties in their response decisions, becoming even more crucial when applied in critical domains such as healthcare, chemical, electrical plants, etc. To contain these issues, in this paper, we present NeuralSentinel (NS), a tool able to validate the reliability and trustworthiness of AI models. This tool combines attack and defence strategies and explainability concepts to stress an AI model and help non-expert staff increase their confidence in this new system by understanding the model decisions. NS provide a simple and easy-to-use interface for helping humans in the loop dealing with all the needed information. This tool was deployed and used in a Hackathon event to evaluate the reliability of a skin cancer image detector. During the event, experts and non-experts attacked and defended the detector, learning which factors were the most important for model misclassification and which techniques were the most efficient. The event was also used to detect NS's limitations and gather feedback for further improvements.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 12 Feb 2024.

Builds on

20 of 27 references.

References · 20

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  7. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  8. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  9. 2016Adversarial examples in the physical worldcited by 1,762
  10. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  11. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  12. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
8 more
  1. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  2. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
  3. 2017Adversarial Attacks on Neural Network Policiescited by 314
  4. 2018Adversarial Attacks Against Medical Deep Learning Systemscited by 61
  5. 2016Investigating the influence of noise and distractors on the interpretation of neural networkscited by 54
  6. 2018Maximal Jacobian-based Saliency Map Attackcited by 46
  7. 2021Cybersecurity Knowledge and Skills Taught in Capture the Flag Challengescited by 6
  8. 2020Deep Learning Defenses Against Adversarial Examples for Dynamic Risk Assessmentcited by 3

Built on by

Not cited yet.