mlsanityData to 7 Oct 2026

Latent Space Explanation by Intervention

arXiv 2112.04895 ↗First posted 9 Dec 2021cs.LG cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

The success of deep neural nets heavily relies on their ability to encode complex relations between their input and their output. While this property serves to fit the training data well, it also obscures the mechanism that drives prediction. This study aims to reveal hidden concepts by employing an intervention mechanism that shifts the predicted class based on discrete variational autoencoders. An explanatory model then visualizes the encoded information from any hidden layer and its corresponding intervened representation. By the assessment of differences between the original representation and the intervened representation, one can determine the concepts that can alter the class, hence providing interpretability. We demonstrate the effectiveness of our approach on CelebA, where we show various visualizations for bias in the data and suggest different interventions to reveal and change bias.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 51 of its 66 references that are in the map. First posted on arXiv 9 Dec 2021.

Builds on

51 of 66 references.

References · 51

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2014Going Deeper with Convolutionscited by 4,231
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  9. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  10. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  11. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  12. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
38 more
  1. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  2. 2016End to End Learning for Self-Driving Carscited by 1,000
  3. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  4. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  5. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  6. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  7. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  8. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  9. 2015Recent Advances in Convolutional Neural Networkscited by 360
  10. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
  11. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  12. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  13. 2017Distilling a Neural Network Into a Soft Decision Treecited by 230
  14. 2019Task2Vec: Task Embedding for Meta-Learningcited by 197
  15. 2019Full-Gradient Representation for Neural Network Visualizationcited by 189
  16. 2019Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasetscited by 173
  17. 2019Explaining Classifiers with Causal Concept Effect (CaCE)cited by 135
  18. 2016Discrete Variational Autoencoderscited by 124
  19. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  20. 2015Fast Algorithms for Convolutional Neural Networkscited by 102
  21. 2015Visualizing Deep Convolutional Neural Networks Using Natural Pre-Imagescited by 93
  22. 2020Investigating Bias and Fairness in Facial Expression Recognitioncited by 87
  23. 2020A causal view of compositional zero-shot recognitioncited by 78
  24. 2020Secure and Robust Machine Learning for Healthcare: A Surveycited by 77
  25. 2020Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropiescited by 56
  26. 2019A Simple Baseline for Audio-Visual Scene-Aware Dialogcited by 52
  27. 2019Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networkscited by 49
  28. 2019Factor Graph Attentioncited by 41
  29. 2020TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aidscited by 41
  30. 2018Understanding Individual Decisions of CNNs via Contrastive Backpropagationcited by 40
  31. 2021Perceptual Score: What Data Modalities Does Your Model Perceive?cited by 35
  32. 2017High-Order Attention Models for Visual Question Answeringcited by 22
  33. 2019Machine Learning Techniques for Biomedical Image Segmentation: An Overview of Technical Aspects and Introduction to State-of-Art Applicationscited by 17
  34. 2020Similarity of Neural Networks with Gradientscited by 16
  35. 2021Are VQA Systems RAD? Measuring Robustness to Augmented Data with Focused Interventionscited by 15
  36. 2021Video and Text Matching with Conditioned Embeddingscited by 10
  37. 2020CLCNet: Deep learning-based Noise Reduction for Hearing Aids using Complex Linear Codingcited by 9
  38. 2016Learning a Metric Embedding for Face Recognition using the Multibatch Methodcited by 7

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 6; 2024: 1; 2025: 2; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptationcited by 68
  2. 2023Layer Collaboration in the Forward-Forward Algorithmcited by 19
  3. 2023Discriminative Class Tokens for Text-to-Image Diffusion Modelscited by 9
  4. 2022A Functional Information Perspective on Model Interpretationcited by 5
  5. 2023Interventional Probing in High Dimensions: An NLI Case Studycited by 5
  6. 2022Explaining Image Classifiers Using Contrastive Counterfactuals in Generative Latent Spacescited by 4
  7. 2025Knowing What You Cannot Explain: Learning to Reject Low-Quality Explanationscited by 2
  8. 2022Describing Sets of Images with Textual-PCAcited by 1
  9. 2023FICNN: A Framework for the Interpretation of Deep Convolutional Neural Networkscited by 0
  10. 2023Semiotics Networks Representing Perceptual Inferencecited by 0
  11. 2024Deep Model Interpretation with Limited Data : A Coreset-based Approachcited by 0
  12. 2025Generating Realistic, Diverse, and Fault-Revealing Inputs with Latent Space Interpolation for Testing Deep Neural Networkscited by 0
1 more
  1. 2026Latent Anchor-Driven Test Generation for Deep Neural Networkscited by 0