mlsanityData to 7 Oct 2026

Rosetta Neurons: Mining the Common Units in a Model Zoo

arXiv 2306.09346 ↗First posted 15 Jun 2023cs.CVcited by 46

Not on the map. Why?

Abstract (arXiv)

Do different neural networks, trained for various vision tasks, share some common representations? In this paper, we demonstrate the existence of common features we call "Rosetta Neurons" across a range of models with different architectures, different tasks (generative and discriminative), and different types of supervision (class-supervised, text-supervised, self-supervised). We present an algorithm for mining a dictionary of Rosetta Neurons across several popular vision models: Class Supervised-ResNet50, DINO-ResNet50, DINO-ViT, MAE, CLIP-ResNet50, BigGAN, StyleGAN-2, StyleGAN-XL. Our findings suggest that certain visual concepts and structures are inherently embedded in the natural world and can be learned by different models regardless of the specific task or architecture, and without the use of semantic labels. We can visualize shared concepts directly due to generative models included in our analysis. The Rosetta Neurons facilitate model-to-model translation enabling various inversion-based manipulations, including cross-class alignments, shifting, zooming, and more, without the need for specialized training.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 27 of its 39 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

27 of 39 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  12. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
15 more
  1. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  3. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  4. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  5. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  6. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  7. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  8. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  9. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  10. 2019GANalyze: Toward Visual Definitions of Cognitive Image Propertiescited by 193
  11. 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
  12. 2020There and Back Again: Revisiting Backpropagation Saliency Methodscited by 60
  13. 2020A Disentangling Invertible Interpretation Network for Explaining Latent Representationscited by 57
  14. 2022BlobGAN: Spatially Disentangled Scene Representationscited by 35
  15. 2020Semantic Pyramid for Image Generationcited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 15; 2025: 20; 2026: 10.

46 papers.

Sort:

By area:

  1. 2024The Platonic Representation Hypothesiscited by 362
  2. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
  3. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 103
  4. 2024Universal Neurons in GPT2 Language Modelscited by 100
  5. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 70
  6. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  7. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  8. 2025Vision Transformers Don't Need Trained Registerscited by 39
  9. 2024Interpreting the Second-Order Effects of Neurons in CLIPcited by 36
  10. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  11. 2025Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMscited by 35
  12. 2024Understanding Video Transformers via Universal Concept Discoverycited by 22
34 more
  1. 2024Data Attribution for Text-to-Image Models by Unlearning Synthesized Imagescited by 22
  2. 2026There Will Be a Scientific Theory of Deep Learningcited by 21
  3. 2024Evolving Interpretable Visual Classifiers with Large Language Modelscited by 18
  4. 2024Defining and Extracting generalizable interaction primitives from DNNscited by 14
  5. 2024Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformerscited by 11
  6. 2025Representation Potentials of Foundation Models for Multimodal Alignment: A Surveycited by 10
  7. 2026Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scalecited by 10
  8. 2024LG-CAV: Train Any Concept Activation Vector with Language Guidancecited by 9
  9. 2026Cross-Modal Redundancy and the Geometry of Vision-Language Embeddingscited by 9
  10. 2025Chorus: Multi-Teacher Pretraining for Holistic 3D Gaussian Scene Encodingcited by 8
  11. 2024Learning on Model Weights using Tree Expertscited by 7
  12. 2024AlignedCut: Visual Concepts Discovery on Brain-Guided Universal Feature Spacecited by 5
  13. 2025Representational Similarity via Interpretable Visual Conceptscited by 5
  14. 2025Learning Model Representations Using Publicly Available Model Hubscited by 4
  15. 2024Quantifying and Enabling the Interpretability of CLIP-like Modelscited by 3
  16. 2025Representational Alignment Across Model Layers and Brain Regions with Multi-Level Optimal Transportcited by 3
  17. 2026Attention Transfer Is Not Universally Effective for Vision Transformerscited by 3
  18. 2024Vision CNNs trained to estimate spatial latents learned similar ventral-stream-aligned representationscited by 2
  19. 2025Discovering Hidden Visual Concepts Beyond Linguistic Input in Infant Learningcited by 2
  20. 2025gen2seg: Generative Models Enable Generalizable Instance Segmentationcited by 2
  21. 2025Cross-Model Semantics in Representation Learningcited by 2
  22. 2025Pruning the Paradox: How CLIP's Most Informative Heads Enhance Performance While Amplifying Biascited by 1
  23. 2025Causes and Consequences of Representational Similarity in Machine Learning Modelscited by 1
  24. 2025Explaining How Visual, Textual and Multimodal Encoders Share Conceptscited by 1
  25. 2025Discovering Divergent Representations between Text-to-Image Modelscited by 1
  26. 2026Programs-of-Layers in LLMs through the Lens of Cortical Areascited by 1
  27. 2025Discovering Chunks in Neural Embeddings for Interpretabilitycited by 0
  28. 2025Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learningcited by 0
  29. 2025Concept-Guided Interpretability via Neural Chunkingcited by 0
  30. 2026Discovering Hidden Gems in Model Repositoriescited by 0
  31. 2026Neuron Populations Exhibit Divergent Selectivity with Scalecited by 0
  32. 2026EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignmentcited by 0
  33. 2026SharedSAE: One Feature Dictionary Across Language Modelscited by 0
  34. 2026From Interpretability Methods to Interpretable Modelscited by 0