# AlignedCut: Visual Concepts Discovery on Brain-Guided Universal Feature Space

arXiv: https://arxiv.org/abs/2406.18344 · HTML: https://mlsanity.com/paper/2406.18344/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Massive Activations in Large Language Models](https://mlsanity.com/paper/2402.17762/) (2024)
- [Interpreting CLIP's Image Representation via Text-Based Decomposition](https://mlsanity.com/paper/2310.05916/) (2023)
- [Cut and Learn for Unsupervised Object Detection and Instance Segmentation](https://mlsanity.com/paper/2301.11320/) (2023)
- [Rosetta Neurons: Mining the Common Units in a Model Zoo](https://mlsanity.com/paper/2306.09346/) (2023)
- [Cognitive computational neuroscience](https://mlsanity.com/paper/1807.11819/) (2018)
- [The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenes](https://mlsanity.com/paper/2301.03198/) (2023)
- [BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity](https://mlsanity.com/paper/2310.04420/) (2023)
- [Brain Decodes Deep Nets](https://mlsanity.com/paper/2312.01280/) (2023)
- [Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations](https://mlsanity.com/paper/2312.06716/) (2023)

## Built on by (most cited first, 5 of 5)

- [Brain Mapping with Dense Features: Grounding Cortical Semantic Selectivity in Natural Images With Vision Transformers](https://mlsanity.com/paper/2410.05266/) (2024)
- [Meta-Learning an In-Context Transformer Model of Human Higher Visual Cortex](https://mlsanity.com/paper/2505.15813/) (2025)
- [Meta-learning In-Context Enables Training-Free Cross Subject Brain Decoding](https://mlsanity.com/paper/2604.08537/) (2026)
- [NeuroFlow: Toward Unified Visual Encoding and Decoding from Neural Activity](https://mlsanity.com/paper/2604.09817/) (2026)
- [FineHOI: Part-Aware Dense Representations for Zero-Shot Human-Object Interaction Detection](https://mlsanity.com/paper/2609.05959/) (2026)

## Abstract (arXiv)

We study the intriguing connection between visual data, deep networks, and the brain. Our method creates a universal channel alignment by using brain voxel fMRI response prediction as the training objective. We discover that deep networks, trained with different objectives, share common feature channels across various models. These channels can be clustered into recurring sets, corresponding to distinct brain regions, indicating the formation of visual concepts. Tracing the clusters of channel responses onto the images, we see semantically meaningful object segments emerge, even without any supervised decoder. Furthermore, the universal feature alignment and the clustering of channels produce a picture and quantification of how visual information is processed through the different network layers, which produces precise comparisons between the networks.