# Rosetta Neurons: Mining the Common Units in a Model Zoo

arXiv: https://arxiv.org/abs/2306.09346 · HTML: https://mlsanity.com/paper/2306.09346/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets](https://mlsanity.com/paper/2202.00273/) (2022)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [GANalyze: Toward Visual Definitions of Cognitive Image Properties](https://mlsanity.com/paper/1906.10112/) (2019)
- [Explaining in Style: Training a GAN to explain a classifier in StyleSpace](https://mlsanity.com/paper/2104.13369/) (2021)
- [There and Back Again: Revisiting Backpropagation Saliency Methods](https://mlsanity.com/paper/2004.02866/) (2020)
- [A Disentangling Invertible Interpretation Network for Explaining Latent Representations](https://mlsanity.com/paper/2004.13166/) (2020)
- [BlobGAN: Spatially Disentangled Scene Representations](https://mlsanity.com/paper/2205.02837/) (2022)
- [Semantic Pyramid for Image Generation](https://mlsanity.com/paper/2003.06221/) (2020)

## Built on by (most cited first, 30 of 46)

- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Interpreting CLIP's Image Representation via Text-Based Decomposition](https://mlsanity.com/paper/2310.05916/) (2023)
- [Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations](https://mlsanity.com/paper/2410.02762/) (2024)
- [Universal Neurons in GPT2 Language Models](https://mlsanity.com/paper/2401.12181/) (2024)
- [Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models](https://mlsanity.com/paper/2504.02821/) (2025)
- [Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment](https://mlsanity.com/paper/2502.03714/) (2025)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [Vision Transformers Don't Need Trained Registers](https://mlsanity.com/paper/2506.08010/) (2025)
- [Interpreting the Second-Order Effects of Neurons in CLIP](https://mlsanity.com/paper/2406.04341/) (2024)
- [Visual Large Language Models for Generalized and Specialized Applications](https://mlsanity.com/paper/2501.02765/) (2025)
- [Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMs](https://mlsanity.com/paper/2506.09047/) (2025)
- [There Will Be a Scientific Theory of Deep Learning](https://mlsanity.com/paper/2604.21691/) (2026)
- [Understanding Video Transformers via Universal Concept Discovery](https://mlsanity.com/paper/2401.10831/) (2024)
- [Data Attribution for Text-to-Image Models by Unlearning Synthesized Images](https://mlsanity.com/paper/2406.09408/) (2024)
- [Evolving Interpretable Visual Classifiers with Large Language Models](https://mlsanity.com/paper/2404.09941/) (2024)
- [Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale](https://mlsanity.com/paper/2604.18572/) (2026)
- [Defining and Extracting generalizable interaction primitives from DNNs](https://mlsanity.com/paper/2401.16318/) (2024)
- [Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformers](https://mlsanity.com/paper/2412.06639/) (2024)
- [Representation Potentials of Foundation Models for Multimodal Alignment: A Survey](https://mlsanity.com/paper/2510.05184/) (2025)
- [Chorus: Multi-Teacher Pretraining for Holistic 3D Gaussian Scene Encoding](https://mlsanity.com/paper/2512.17817/) (2025)
- [Cross-Modal Redundancy and the Geometry of Vision-Language Embeddings](https://mlsanity.com/paper/2602.06218/) (2026)
- [LG-CAV: Train Any Concept Activation Vector with Language Guidance](https://mlsanity.com/paper/2410.10308/) (2024)
- [Learning on Model Weights using Tree Experts](https://mlsanity.com/paper/2410.13569/) (2024)
- [AlignedCut: Visual Concepts Discovery on Brain-Guided Universal Feature Space](https://mlsanity.com/paper/2406.18344/) (2024)
- [Representational Similarity via Interpretable Visual Concepts](https://mlsanity.com/paper/2503.15699/) (2025)
- [Learning Model Representations Using Publicly Available Model Hubs](https://mlsanity.com/paper/2510.02096/) (2025)
- [Quantifying and Enabling the Interpretability of CLIP-like Models](https://mlsanity.com/paper/2409.06579/) (2024)
- [Causes and Consequences of Representational Similarity in Machine Learning Models](https://mlsanity.com/paper/2505.13899/) (2025)
- [Representational Alignment Across Model Layers and Brain Regions with Multi-Level Optimal Transport](https://mlsanity.com/paper/2510.01706/) (2025)
- [Attention Transfer Is Not Universally Effective for Vision Transformers](https://mlsanity.com/paper/2605.07191/) (2026)

## Abstract (arXiv)

Do different neural networks, trained for various vision tasks, share some common representations? In this paper, we demonstrate the existence of common features we call "Rosetta Neurons" across a range of models with different architectures, different tasks (generative and discriminative), and different types of supervision (class-supervised, text-supervised, self-supervised). We present an algorithm for mining a dictionary of Rosetta Neurons across several popular vision models: Class Supervised-ResNet50, DINO-ResNet50, DINO-ViT, MAE, CLIP-ResNet50, BigGAN, StyleGAN-2, StyleGAN-XL. Our findings suggest that certain visual concepts and structures are inherently embedded in the natural world and can be learned by different models regardless of the specific task or architecture, and without the use of semantic labels. We can visualize shared concepts directly due to generative models included in our analysis. The Rosetta Neurons facilitate model-to-model translation enabling various inversion-based manipulations, including cross-class alignments, shifting, zooming, and more, without the need for specialized training.