# The effect of task and training on intermediate representations in convolutional neural networks revealed with modified RV similarity analysis

arXiv: https://arxiv.org/abs/1912.02260 · HTML: https://mlsanity.com/paper/1912.02260/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2019.

## Built on by (most cited first, 5 of 5)

- [Self-Supervised Pretraining Improves Self-Supervised Pretraining](https://mlsanity.com/paper/2103.12718/) (2021)
- [The Tunnel Effect: Building Data Representations in Deep Neural Networks](https://mlsanity.com/paper/2305.19753/) (2023)
- [Correcting Biased Centered Kernel Alignment Measures in Biological and Artificial Neural Networks](https://mlsanity.com/paper/2405.01012/) (2024)
- [On the Origins of the Block Structure Phenomenon in Neural Network Representations](https://mlsanity.com/paper/2202.07184/) (2022)
- [Successes and critical failures of neural networks in capturing human-like speech recognition](https://mlsanity.com/paper/2204.03740/) (2022)

## Abstract (arXiv)

Centered Kernel Alignment (CKA) was recently proposed as a similarity metric for comparing activation patterns in deep networks. Here we experiment with the modified RV-coefficient (RV2), which has very similar properties as CKA while being less sensitive to dataset size. We compare the representations of networks that received varying amounts of training on different layers: a standard trained network (all parameters updated at every step), a freeze trained network (layers gradually frozen during training), random networks (only some layers trained), and a completely untrained network. We found that RV2 was able to recover expected similarity patterns and provide interpretable similarity matrices that suggested hypotheses about how representations are affected by different training recipes. We propose that the superior performance achieved by freeze training can be attributed to representational differences in the penultimate layer. Our comparisons of random networks suggest that the inputs and targets serve as anchors on the representations in the lowest and highest layers.