mlsanityData to 14 Oct 2026

Similarity and Matching of Neural Network Representations

arXiv 2110.14633 ↗First posted 27 Oct 2021cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

We employ a toolset -- dubbed Dr. Frankenstein -- to analyse the similarity of representations in deep neural networks. With this toolset, we aim to match the activations on given layers of two trained neural networks by joining them with a stitching layer. We demonstrate that the inner representations emerging in deep convolutional neural networks with the same architecture but different initializations can be matched with a surprisingly high degree of accuracy even with a single, affine stitching layer. We choose the stitching layer from several possible classes of linear transformations and investigate their performance and properties. The task of matching representations is closely related to notions of similarity. Using this toolset, we also provide a novel viewpoint on the current line of research regarding similarity indices of neural network representations: the perspective of the performance on a task.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 27 Oct 2021.

Builds on

17 of 29 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2014Going Deeper with Convolutionscited by 4,241
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  6. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  8. 2014How transferable are features in deep neural networks?cited by 1,225
  9. 2020What is being transferred in transfer learning?cited by 368
  10. 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
  11. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 245
  12. 2014Understanding image representations by measuring their equivariance and equivalencecited by 243
5 more
  1. 2015Convergent Learning: Do different neural networks learn the same representations?cited by 220
  2. 2021Revisiting Model Stitching to Compare Neural Representationscited by 209
  3. 2020Linear Mode Connectivity in Multitask and Continual Learningcited by 145
  4. 2018Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representationcited by 62
  5. 2020Similarity Analysis of Contextual Word Representation Modelscited by 54

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 11; 2023: 22; 2024: 13; 2025: 15; 2026: 16.

78 papers.

Sort:

By area:

  1. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  2. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  3. 2022Relative representations enable zero-shot latent space communicationcited by 191
  4. 2022Analyzing Transformers in Embedding Spacecited by 132
  5. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 131
  6. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  7. 2022Deep Model Reassemblycited by 78
  8. 2022Identifiability of deep generative models without auxiliary informationcited by 70
  9. 2022On the Symmetries of Deep Learning Models and their Internal Representationscited by 64
  10. 2023Text-To-Concept (and Back) via Cross-Model Alignmentcited by 64
  11. 2024Demystifying Verbatim Memorization in Large Language Modelscited by 56
  12. 2023Latent Space Translation via Semantic Alignmentcited by 52
38 more
  1. 2024T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitchingcited by 33
  2. 2024Training-Free Pretrained Model Mergingcited by 32
  3. 2025Understanding the Emergence of Multimodal Representation Alignmentcited by 31
  4. 2022ModelDiff: A Framework for Comparing Learning Algorithmscited by 30
  5. 2023Stitchable Neural Networkscited by 26
  6. 2024Configurable Foundation Models: Building LLMs from a Modular Perspectivecited by 22
  7. 2024Objective drives the consistency of representational similarity across datasetscited by 18
  8. 2022Deep Incubation: Training Large Models by Divide-and-Conqueringcited by 13
  9. 2022Understanding Robust Learning through the Lens of Representation Similaritiescited by 12
  10. 2025Representation Potentials of Foundation Models for Multimodal Alignment: A Surveycited by 11
  11. 2025Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generatorcited by 11
  12. 2024Latent Space Translation via Inverse Relative Projectioncited by 10
  13. 2025Data Duplication: A Novel Multi-Purpose Attack Paradigm in Machine Unlearningcited by 9
  14. 2023Bootstrapping Parallel Anchors for Relative Representationscited by 8
  15. 2023Stitched ViTs are Flexible Vision Backbonescited by 8
  16. 2021Graph-Based Similarity of Neural Network Representationscited by 7
  17. 2023Measuring the Instability of Fine-Tuningcited by 7
  18. 2023Building Variable-sized Models via Learngene Poolcited by 7
  19. 2025Linear Representation Transferability Hypothesis: Leveraging Small Models to Steer Large Modelscited by 7
  20. 2023All Roads Lead to Rome? Exploring the Invariance of Transformers' Representationscited by 6
  21. 2023CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Modelcited by 5
  22. 2024How not to Stitch Representations to Measure Similarity: Task Loss Matching versus Direct Matchingcited by 5
  23. 2022Causes of Catastrophic Forgetting in Class-Incremental Semantic Segmentationcited by 4
  24. 2023Principles of Forgetting in Domain-Incremental Semantic Segmentation in Adverse Weather Conditionscited by 4
  25. 2023Efficient Stitchable Task Adaptationcited by 4
  26. 2024Relative Representations: Topological and Geometric Perspectivescited by 4
  27. 2025Connecting Neural Models Latent Geometries with Relative Geodesic Representationscited by 4
  28. 2023Effects of Architectures on Continual Semantic Segmentationcited by 3
  29. 2023How good are variational autoencoders at transfer learning?cited by 3
  30. 2024On Affine Homotopy between Language Encoderscited by 3
  31. 2024SimSMoE: Solving Representational Collapse via Similarity Measurecited by 3
  32. 2023StitchNet: Composing Neural Networks from Pre-Trained Fragmentscited by 2
  33. 2023FairBranch: Mitigating Bias Transfer in Fair Multi-task Learningcited by 2
  34. 2024R3L: Relative Representations for Reinforcement Learningcited by 2
  35. 2026Revisiting Model Stitching In the Foundation Model Eracited by 2
  36. 2026Stitched Value Model for Diffusion Alignmentcited by 2
  37. 2022What do CNNs Learn in the First Layer and Why? A Linear Systems Perspectivecited by 1
  38. 2023Mode Combinability: Exploring Convex Combinations of Permutation Aligned Modelscited by 1

50 most cited of 78 All 78 (JSON)