mlsanityData to 7 Oct 2026

Smooth InfoMax -- Towards Easier Post-Hoc Interpretability

arXiv 2408.12936 ↗First posted 23 Aug 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce Smooth InfoMax (SIM), a self-supervised representation learning method that incorporates interpretability constraints into the latent representations at different depths of the network. Based on β-VAEs, SIM's architecture consists of probabilistic modules optimized locally with the InfoNCE loss to produce Gaussian-distributed representations regularized toward the standard normal distribution. This creates smooth, well-defined, and better-disentangled latent spaces, enabling easier post-hoc analysis. Evaluated on speech data, SIM preserves the large-scale training benefits of Greedy InfoMax while improving the effectiveness of post-hoc interpretability methods across layers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 33 references that are in the map. First posted on arXiv 23 Aug 2024.

Builds on

19 of 33 references.

References · 19

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  3. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  4. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  5. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  6. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  7. 2018Disentangling by Factorisingcited by 614
  8. 2016Tutorial on Variational Autoencoderscited by 404
  9. 2018Understanding disentangling in β-VAEcited by 332
  10. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  11. 2015Visualizing Deep Convolutional Neural Networks Using Natural Pre-Imagescited by 93
  12. 2017Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Datacited by 92
7 more
  1. 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 85
  2. 2019Semi-Supervised Histology Classification using Deep Multiple Instance Learning and Contrastive Predictive Codingcited by 35
  3. 2021Segmental Contrastive Predictive Coding for Unsupervised Word Segmentationcited by 32
  4. 2019Optimizing for Interpretability in Deep Neural Networks with Tree Regularizationcited by 13
  5. 2019A Closer Look at Disentangling in β-VAEcited by 8
  6. 2022Improving Interpretability via Regularization of Neural Activation Sensitivitycited by 5
  7. 2021Encouraging Disentangled and Convex Representation with Controllable Interpolation Regularizationcited by 4

Built on by

Not cited yet.