# Understanding image representations by measuring their equivariance and equivalence

arXiv: https://arxiv.org/abs/1411.5908 · HTML: https://mlsanity.com/paper/1411.5908/

## Key facts

- Cited by 243 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2014.

## Built on by (most cited first, 30 of 243)

- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [The Platonic Representation Hypothesis](https://mlsanity.com/paper/2405.07987/) (2024)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [Revisiting Model Stitching to Compare Neural Representations](https://mlsanity.com/paper/2106.07682/) (2021)
- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [Linearly Mapping from Image to Text Space](https://mlsanity.com/paper/2209.15162/) (2022)
- [Analyzing Transformers in Embedding Space](https://mlsanity.com/paper/2209.02535/) (2022)
- [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023)
- [Approximately Equivariant Networks for Imperfectly Symmetric Dynamics](https://mlsanity.com/paper/2201.11969/) (2022)
- [Reliability of CKA as a Similarity Measure in Deep Learning](https://mlsanity.com/paper/2210.16156/) (2022)
- [Equivariant Imaging: Learning Beyond the Range Space](https://mlsanity.com/paper/2103.14756/) (2021)
- [Similarity and Matching of Neural Network Representations](https://mlsanity.com/paper/2110.14633/) (2021)
- [Deep Model Reassembly](https://mlsanity.com/paper/2210.17409/) (2022)
- [Identifiability of deep generative models without auxiliary information](https://mlsanity.com/paper/2206.10044/) (2022)
- [Equivariant Contrastive Learning](https://mlsanity.com/paper/2111.00899/) (2021)
- [On the Symmetries of Deep Learning Models and their Internal Representations](https://mlsanity.com/paper/2205.14258/) (2022)
- [Text-To-Concept (and Back) via Cross-Model Alignment](https://mlsanity.com/paper/2305.06386/) (2023)
- [Self-Supervised Multimodal Learning: A Survey](https://mlsanity.com/paper/2304.01008/) (2023)
- [Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs](https://mlsanity.com/paper/2110.02797/) (2021)
- [The Lie Derivative for Measuring Learned Equivariance](https://mlsanity.com/paper/2210.02984/) (2022)
- [Latent Space Translation via Semantic Alignment](https://mlsanity.com/paper/2311.00664/) (2023)
- [Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity](https://mlsanity.com/paper/2307.08286/) (2023)
- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)
- [Physics-Guided Deep Learning for Dynamical Systems: A Survey](https://mlsanity.com/paper/2107.01272/) (2021)
- [Time-Equivariant Contrastive Video Representation Learning](https://mlsanity.com/paper/2112.03624/) (2021)
- [Accurate 3-DoF Camera Geo-Localization via Ground-to-Satellite Image Matching](https://mlsanity.com/paper/2203.14148/) (2022)
- [Do Vision and Language Encoders Represent the World Similarly?](https://mlsanity.com/paper/2401.05224/) (2024)
- [3D molecule generation by denoising voxel grids](https://mlsanity.com/paper/2306.07473/) (2023)
- [Shared Global and Local Geometry of Language Model Embeddings](https://mlsanity.com/paper/2503.21073/) (2025)
- [Imaging with Equivariant Deep Learning](https://mlsanity.com/paper/2209.01725/) (2022)

## Abstract (arXiv)

Despite the importance of image representations such as histograms of oriented gradients and deep Convolutional Neural Networks (CNN), our theoretical understanding of them remains limited. Aiming at filling this gap, we investigate three key mathematical properties of representations: equivariance, invariance, and equivalence. Equivariance studies how transformations of the input image are encoded by the representation, invariance being a special case where a transformation has no effect. Equivalence studies whether two representations, for example two different parametrisations of a CNN, capture the same visual information or not. A number of methods to establish these properties empirically are proposed, including introducing transformation and stitching layers in CNNs. These methods are then applied to popular representations to reveal insightful aspects of their structure, including clarifying at which layers in a CNN certain geometric invariances are achieved. While the focus of the paper is theoretical, direct applications to structured-output regression are demonstrated too.