# Identifying Interpretable Subspaces in Image Representations

arXiv: https://arxiv.org/abs/2307.10504 · HTML: https://mlsanity.com/paper/2307.10504/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 36 references that are in the map.
- First posted on arXiv 20 Jul 2023.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [Compositional Explanations of Neurons](https://mlsanity.com/paper/2006.14032/) (2020)
- [RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank](https://mlsanity.com/paper/2210.02885/) (2022)
- [CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks](https://mlsanity.com/paper/2204.10965/) (2022)
- [Natural Language Descriptions of Deep Visual Features](https://mlsanity.com/paper/2201.11114/) (2022)
- [Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks](https://mlsanity.com/paper/1801.03454/) (2018)
- [Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectors](https://mlsanity.com/paper/2006.15417/) (2020)
- [Solo-learn: A Library of Self-supervised Methods for Visual Representation Learning](https://mlsanity.com/paper/2108.01775/) (2021)
- [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021)
- [High Fidelity Visualization of What Your Self-Supervised Representation Knows About](https://mlsanity.com/paper/2112.09164/) (2021)
- [Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning](https://mlsanity.com/paper/2208.08831/) (2022)
- [Towards Unique and Informative Captioning of Images](https://mlsanity.com/paper/2009.03949/) (2020)
- [Measuring Self-Supervised Representation Quality for Downstream Classification using Discriminative Features](https://mlsanity.com/paper/2203.01881/) (2022)

## Built on by (most cited first, 30 of 32)

- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [Labeling Neural Representations with Inverse Recognition](https://mlsanity.com/paper/2311.13594/) (2023)
- [PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits](https://mlsanity.com/paper/2404.06453/) (2024)
- [Mechanistic understanding and validation of large AI models with SemanticLens](https://mlsanity.com/paper/2501.05398/) (2025)
- [Linear Explanations for Individual Neurons](https://mlsanity.com/paper/2405.06855/) (2024)
- [CoSy: Evaluating Textual Explanations of Neurons](https://mlsanity.com/paper/2405.20331/) (2024)
- [WWW: A Unified Framework for Explaining What, Where and Why of Neural Networks by Interpretation of Neuron Concepts](https://mlsanity.com/paper/2402.18956/) (2024)
- [AND: Audio Network Dissection for Interpreting Deep Acoustic Models](https://mlsanity.com/paper/2406.16990/) (2024)
- [LLM-assisted Concept Discovery: Automatically Identifying and Explaining Neuron Functions](https://mlsanity.com/paper/2406.08572/) (2024)
- [Enhancing Concept Localization in CLIP-based Concept Bottleneck Models](https://mlsanity.com/paper/2510.07115/) (2025)
- [Explaining the Implicit Neural Canvas: Connecting Pixels to Neurons by Tracing their Contributions](https://mlsanity.com/paper/2401.10217/) (2024)
- [Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations](https://mlsanity.com/paper/2508.01728/) (2025)
- [LaVCa: LLM-assisted Visual Cortex Captioning](https://mlsanity.com/paper/2502.13606/) (2025)
- [Manipulating Feature Visualizations with Gradient Slingshots](https://mlsanity.com/paper/2401.06122/) (2024)
- [Narrowing Information Bottleneck Theory for Multimodal Image-Text Representations Interpretability](https://mlsanity.com/paper/2502.14889/) (2025)
- [NeurFlow: Interpreting Neural Networks through Neuron Groups and Functional Interactions](https://mlsanity.com/paper/2502.16105/) (2025)
- [Discovering Hidden Visual Concepts Beyond Linguistic Input in Infant Learning](https://mlsanity.com/paper/2501.05205/) (2025)
- [QPM: Discrete Optimization for Globally Interpretable Image Classification](https://mlsanity.com/paper/2502.20130/) (2025)
- [Prospector Heads: Generalized Feature Attribution for Large Models & Data](https://mlsanity.com/paper/2402.11729/) (2024)
- [FairDropout: Using Example-Tied Dropout to Enhance Generalization of Minority Groups](https://mlsanity.com/paper/2502.06695/) (2025)
- [Embedding Shift Dissection on CLIP: Effects of Augmentations on VLM's Representation Learning](https://mlsanity.com/paper/2503.23495/) (2025)
- [CE-FAM: Concept-Based Explanation via Fusion of Activation Maps](https://mlsanity.com/paper/2509.23849/) (2025)
- [Mask-Free Neuron Concept Annotation for Interpreting Neural Networks in Medical Domain](https://mlsanity.com/paper/2407.11375/) (2024)
- [A Concept-Centric Approach to Multi-Modality Learning](https://mlsanity.com/paper/2412.13847/) (2024)
- [Beyond Top Activations: Efficient and Reliable Crowdsourced Evaluation of Automated Interpretability](https://mlsanity.com/paper/2506.07985/) (2025)
- [Guaranteed Optimal Compositional Explanations for Neurons](https://mlsanity.com/paper/2511.20934/) (2025)
- [Select, Hypothesize and Verify: Towards Verified Neuron Concept Interpretation](https://mlsanity.com/paper/2603.24953/) (2026)
- [LINE: LLM-based Iterative Neuron Explanations for Vision Models](https://mlsanity.com/paper/2604.08039/) (2026)
- [Contrastive Semantic Projection: Faithful Neuron Labeling with Contrastive Examples](https://mlsanity.com/paper/2604.22477/) (2026)
- [Multimodal Concept Bottleneck Models](https://mlsanity.com/paper/2606.19882/) (2026)

## Abstract (arXiv)

We propose Automatic Feature Explanation using Contrasting Concepts (FALCON), an interpretability framework to explain features of image representations. For a target feature, FALCON captions its highly activating cropped images using a large captioning dataset (like LAION-400m) and a pre-trained vision-language model like CLIP. Each word among the captions is scored and ranked leading to a small number of shared, human-understandable concepts that closely describe the target feature. FALCON also applies contrastive interpretation using lowly activating (counterfactual) images, to eliminate spurious concepts. Although many existing approaches interpret features independently, we observe in state-of-the-art self-supervised and supervised models, that less than 20% of the representation space can be explained by individual features. We show that features in larger spaces become more interpretable when studied in groups and can be explained with high-order scoring concepts through FALCON. We discuss how extracted concepts can be used to explain and debug failures in downstream tasks. Finally, we present a technique to transfer concepts from one (explainable) representation space to another unseen representation space by learning a simple linear transformation. Code available at https://github.com/NehaKalibhat/falcon-explain.