mlsanityData to 7 Oct 2026

Leveraging Sparse Linear Layers for Debuggable Deep Networks

arXiv 2105.04857 ↗First posted 11 May 2021cs.LG stat.MLcited by 82

Not on the map. Why?

Abstract (arXiv)

We show how fitting sparse linear models over learned deep feature representations can lead to more debuggable neural networks. These networks remain highly accurate while also being more amenable to human interpretation, as we demonstrate quantiatively via numerical and human experiments. We further illustrate how the resulting sparse explanations can help to identify spurious correlations, explain misclassifications, and diagnose model biases in vision and language tasks. The code for our toolkit can be found at https://github.com/madrylab/debuggabledeepnetworks.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 40 of its 64 references that are in the map. First posted on arXiv 11 May 2021.

Builds on

40 of 64 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  9. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  11. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  12. 2018Sanity Checks for Saliency Mapscited by 1,149
28 more
  1. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  2. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  3. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  4. 2018Robustness May Be at Odds with Accuracycited by 873
  5. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  6. 2018Recognition in Terra Incognitacited by 655
  7. 2020What is the State of Neural Network Pruning?cited by 578
  8. 2017Interpretation of Neural Networks is Fragilecited by 443
  9. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
  10. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 401
  11. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  12. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  13. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  14. 2018Actionable Recourse in Linear Classificationcited by 327
  15. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
  16. 2019Counterfactual Visual Explanationscited by 312
  17. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  18. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  19. 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
  20. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  21. 2020Debugging Tests for Model Explanationscited by 146
  22. 2019Explaining Classifiers with Causal Concept Effect (CaCE)cited by 135
  23. 2019The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasonscited by 115
  24. 2020From ImageNet to Image Classification: Contextualizing Progress on Benchmarkscited by 90
  25. 2020Towards falsifiable interpretability researchcited by 53
  26. 2020NBDT: Neural-Backed Decision Treescited by 46
  27. 2020Debiasing Skin Lesion Datasets and Models? Not So Fastcited by 43
  28. 2019Optimal mini-batch and step sizes for SAGAcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 22; 2023: 22; 2024: 11; 2025: 14; 2026: 7.

82 papers.

Sort:

By area:

  1. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 376
  2. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 369
  3. 2023Label-Free Concept Bottleneck Modelscited by 282
  4. 2023Model Sparsity Can Simplify Machine Unlearningcited by 206
  5. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  6. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  7. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  8. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  9. 2021Salient ImageNet: How to discover spurious features in Deep Learning?cited by 114
  10. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  11. 2024VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidancecited by 65
  12. 2021Editing a classifier by rewriting its prediction rulescited by 64
38 more
  1. 2022A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributescited by 56
  2. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  3. 2023FFCV: Accelerating Training by Removing Data Bottleneckscited by 49
  4. 2023LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Imagescited by 46
  5. 2023Codebook Features: Sparse and Discrete Interpretability for Neural Networkscited by 45
  6. 2022The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanationscited by 41
  7. 2022Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNetcited by 41
  8. 2025Weight-sparse transformers have interpretable circuitscited by 40
  9. 2022Missingness Bias in Model Debuggingcited by 36
  10. 2022MultiViz: Towards Visualizing and Understanding Multimodal Modelscited by 36
  11. 2022DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanationscited by 34
  12. 20213DB: A Framework for Debugging Computer Vision Modelscited by 31
  13. 2023Coarse-to-Fine Concept Bottleneck Modelscited by 31
  14. 2022ModelDiff: A Framework for Comparing Learning Algorithmscited by 30
  15. 2021How Well do Feature Visualizations Support Causal Understanding of CNN Activations?cited by 28
  16. 2025Language Guided Concept Bottleneck Models for Interpretable Continual Learningcited by 28
  17. 2023FACTS: First Amplify Correlations and Then Slice to Discover Biascited by 26
  18. 2023Sparse Linear Concept Discovery Modelscited by 22
  19. 2022Explicit Tradeoffs between Adversarial and Natural Distributional Robustnesscited by 20
  20. 2024Linear Explanations for Individual Neuronscited by 19
  21. 2023Scale Alone Does not Improve Mechanistic Interpretability in Vision Modelscited by 17
  22. 2024Concept Bottleneck Models Without Predefined Conceptscited by 17
  23. 2022Core Risk Minimization using Salient ImageNetcited by 15
  24. 2023How to Overcome Curse-of-Dimensionality for Out-of-Distribution Detection?cited by 15
  25. 2025If Concept Bottlenecks are the Question, are Foundation Models the Answer?cited by 15
  26. 2022Spuriosity Rankings: Sorting Data to Measure and Mitigate Biasescited by 13
  27. 2022Understanding Robust Learning through the Lens of Representation Similaritiescited by 12
  28. 2025Show and Tell: Visually Explainable Deep Neural Nets via Spatially-Aware Concept Bottleneck Modelscited by 12
  29. 2024Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learningcited by 11
  30. 2022Invariant Learning via Diffusion Dreamed Distribution Shiftscited by 10
  31. 2023Q-SENN: Quantized Self-Explaining Neural Networkscited by 9
  32. 2021FOCUS: Familiar Objects in Common and Uncommon Settingscited by 8
  33. 2023Take 5: Interpretable Image Classification with a Handful of Featurescited by 7
  34. 2023Text Descriptions are Compressive and Invariant Representations for Visual Learningcited by 7
  35. 2024Towards Compositionality in Concept Learningcited by 7
  36. 2025Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognitioncited by 7
  37. 2022Diagnostics for Deep Neural Networks with Automated Copy/Paste Attackscited by 6
  38. 2024Benchmarking Spurious Bias in Few-Shot Image Classifierscited by 6

50 most cited of 82 All 82 (JSON)