# Model Stitching: Looking For Functional Similarity Between Representations

arXiv: https://arxiv.org/abs/2303.11277 · HTML: https://mlsanity.com/paper/2303.11277/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 11 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 6 of 6)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Revisiting Model Stitching to Compare Neural Representations](https://mlsanity.com/paper/2106.07682/) (2021)
- [Grounding Representation Similarity with Statistical Testing](https://mlsanity.com/paper/2108.01661/) (2021)

## Built on by (most cited first, 10 of 10)

- [Objective drives the consistency of representational similarity across datasets](https://mlsanity.com/paper/2411.05561/) (2024)
- [How not to Stitch Representations to Measure Similarity: Task Loss Matching versus Direct Matching](https://mlsanity.com/paper/2412.11299/) (2024)
- [Efficient Stitchable Task Adaptation](https://mlsanity.com/paper/2311.17352/) (2023)
- [A Negative Result on Cross-Model Activation Transfer in a Pythia Multi-Hop Setting](https://mlsanity.com/paper/2606.03280/) (2026)
- [Model Alignment Search](https://mlsanity.com/paper/2501.06164/) (2025)
- [Bridging Functional and Representational Similarity via Usable Information](https://mlsanity.com/paper/2601.21568/) (2026)
- [Revisiting Model Stitching In the Foundation Model Era](https://mlsanity.com/paper/2603.12433/) (2026)
- [Keys in the Weights: Transformer Authentication Using Model-Bound Latent Representations](https://mlsanity.com/paper/2511.00973/) (2025)
- [ABRA: Teleporting Fine-Tuned Knowledge Across Domains for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2603.12409/) (2026)
- [Mixture of Training: Recombining Small-Scale Scaffolded Pretraining Runs into a Larger Language Model](https://mlsanity.com/paper/2608.13277/) (2026)

## Abstract (arXiv)

Model stitching (Lenc & Vedaldi 2015) is a compelling methodology to compare different neural network representations, because it allows us to measure to what degree they may be interchanged. We expand on a previous work from Bansal, Nakkiran & Barak which used model stitching to compare representations of the same shapes learned by differently seeded and/or trained neural networks of the same architecture. Our contribution enables us to compare the representations learned by layers with different shapes from neural networks with different architectures. We subsequently reveal unexpected behavior of model stitching. Namely, we find that stitching, based on convolutions, for small ResNets, can reach high accuracy if those layers come later in the first (sender) network than in the second (receiver), even if those layers are far apart.