mlsanityData to 7 Oct 2026

Learning Minimal Neural Specifications

arXiv 2404.04662 ↗First posted 6 Apr 2024cs.LG cs.PLcited by 3

Not on the map. Why?

Abstract (arXiv)

Formal verification is only as good as the specification of a system, which is also true for neural network verification. Existing specifications follow the paradigm of data as specification, where the local neighborhood around a reference data point is considered correct or robust. While these specifications provide a fair testbed for assessing model robustness, they are too restrictive for verifying any unseen test data points, a challenging task with significant real-world implications. Recent work shows great promise through a new paradigm, neural representation as specification, which uses neural activation patterns (NAPs) for this purpose. However, it computes the most refined NAPs, which include many redundant neurons. In this paper, we study the following problem: Given a neural network, find a minimal (general) NAP specification that is sufficient for formal verification of its robustness properties. Finding the minimal NAP specification not only expands verifiable bounds but also provides insights into which set of neurons contributes to the model's robustness. To address this problem, we propose three approaches: conservative, statistical, and optimistic. Each of these methods offers distinct strengths and trade-offs in terms of minimality and computational speed, making them suitable for scenarios with different priorities. Notably, the optimistic approach can probe potential causal links between neurons and the robustness of large vision neural networks without relying on verification tools, a task existing methods struggle to scale. Our experiments show that minimal NAP specifications use far fewer neurons than those from previous work while expanding verifiable boundaries by several orders of magnitude.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 58 references that are in the map. First posted on arXiv 6 Apr 2024.

Builds on

34 of 58 references.

References · 34

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  8. 2018Sanity Checks for Saliency Mapscited by 1,149
  9. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  10. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  11. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  12. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
22 more
  1. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  2. 2020A Survey on Neural Network Interpretabilitycited by 291
  3. 2016Safety Verification of Deep Neural Networkscited by 262
  4. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  5. 2017Formal Verification of Piece-Wise Linear Feed-Forward Neural Networkscited by 241
  6. 2019Adversarial Attacks and Defenses in Images, Graphs and Text: A Reviewcited by 234
  7. 2021Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verificationcited by 224
  8. 2020Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifierscited by 183
  9. 2019Complexity of Linear Regions in Deep Networkscited by 166
  10. 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 160
  11. 2017Maximum Resilience of Artificial Neural Networkscited by 97
  12. 2021Convolutional Neural Network Pruning with Structural Redundancy Reductioncited by 73
  13. 2019Neural Network Branching for Neural Network Verificationcited by 45
  14. 2020Proper Network Interpretability Helps Adversarial Robustness in Classificationcited by 41
  15. 2023The Fourth International Verification of Neural Networks Competition (VNN-COMP 2023): Summary and Resultscited by 38
  16. 2017Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examplescited by 27
  17. 2020Into the Unknown: Active Monitoring of Neural Networkscited by 19
  18. 2022Towards Reliable Neural Specificationscited by 12
  19. 2018Symbolic Execution for Deep Neural Networkscited by 8
  20. 2022Neural Activation Patterns (NAPs): Visual Explainability of Learned Conceptscited by 6
  21. 2023Learning Reliable Logical Rules with SATNetcited by 3
  22. 2022Scalar Invariant Networks with Zero Biascited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guaranteescited by 10
  2. 2026Event-B Agent: Towards LLM Agent for Formal Model Synthesis and Repaircited by 3
  3. 2025Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exitscited by 1