# From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication

arXiv: https://arxiv.org/abs/2310.01211 · HTML: https://mlsanity.com/paper/2310.01211/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Insights on representational similarity in neural networks with canonical correlation](https://mlsanity.com/paper/1806.05759/) (2018)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023)
- [Reliability of CKA as a Similarity Measure in Deep Learning](https://mlsanity.com/paper/2210.16156/) (2022)
- [Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification](https://mlsanity.com/paper/1608.04363/) (2016)
- [The Riemannian Geometry of Deep Generative Models](https://mlsanity.com/paper/1711.08014/) (2017)
- [On the Benefits of Invariance in Neural Networks](https://mlsanity.com/paper/2005.00178/) (2020)
- [Deep Set Prediction Networks](https://mlsanity.com/paper/1906.06565/) (2019)
- [Oriented Response Networks](https://mlsanity.com/paper/1701.01833/) (2017)
- [ASIF: Coupled Data Turns Unimodal Models to Multimodal Without Training](https://mlsanity.com/paper/2210.01738/) (2022)
- [Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations](https://mlsanity.com/paper/2202.10638/) (2022)
- [Rethinking Channel Dimensions for Efficient Model Design](https://mlsanity.com/paper/2007.00992/) (2020)
- [N24News: A New Dataset for Multimodal News Classification](https://mlsanity.com/paper/2108.13327/) (2021)
- [Symmetry-Based Representations for Artificial and Biological General Intelligence](https://mlsanity.com/paper/2203.09250/) (2022)
- [Learning disentangled representations via product manifold projection](https://mlsanity.com/paper/2103.01638/) (2021)
- [Learning Invariant Weights in Neural Networks](https://mlsanity.com/paper/2202.12439/) (2022)

## Built on by (most cited first, 21 of 21)

- [Latent Space Translation via Semantic Alignment](https://mlsanity.com/paper/2311.00664/) (2023)
- [Latent Space Translation via Inverse Relative Projection](https://mlsanity.com/paper/2406.15057/) (2024)
- [Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research](https://mlsanity.com/paper/2504.13101/) (2025)
- [Relative Representations of Latent Spaces enable Efficient Semantic Channel Equalization](https://mlsanity.com/paper/2411.19719/) (2024)
- [Escaping Plato's Cave: Towards the Alignment of 3D and Text Latent Spaces](https://mlsanity.com/paper/2503.05283/) (2025)
- [Multi-Way Representation Alignment](https://mlsanity.com/paper/2602.06205/) (2026)
- [Relative Representations: Topological and Geometric Perspectives](https://mlsanity.com/paper/2409.10967/) (2024)
- [Connecting Neural Models Latent Geometries with Relative Geodesic Representations](https://mlsanity.com/paper/2506.01599/) (2025)
- [When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective](https://mlsanity.com/paper/2506.03784/) (2025)
- [TOAST: Transformer Optimization using Adaptive and Simple Transformations](https://mlsanity.com/paper/2410.04941/) (2024)
- [Modality Alignment across Trees on Heterogeneous Hyperbolic Manifolds](https://mlsanity.com/paper/2510.27391/) (2025)
- [R3L: Relative Representations for Reinforcement Learning](https://mlsanity.com/paper/2404.12917/) (2024)
- [Cross-Model Semantics in Representation Learning](https://mlsanity.com/paper/2508.03649/) (2025)
- [Latent Functional Maps: a spectral framework for representation alignment](https://mlsanity.com/paper/2406.14183/) (2024)
- [Geometry matters: insights from Ollivier Ricci Curvature and Ricci Flow into representational alignment through Ollivier-Ricci Curvature and Ricci Flow](https://mlsanity.com/paper/2501.00919/) (2025)
- [Semantically Guided Action Anticipation](https://mlsanity.com/paper/2411.15557/) (2024)
- [Mapping representations in Reinforcement Learning via Semantic Alignment for Zero-Shot Stitching](https://mlsanity.com/paper/2503.01881/) (2025)
- [Relative Geometry of Neural Forecasters: Linking Accuracy and Alignment in Learned Latent Geometry](https://mlsanity.com/paper/2602.15676/) (2026)
- [A Stitch in Time Saves Nine: Preserving Policy Compatibility Under Perception Updates in End-to-End Autonomous Driving](https://mlsanity.com/paper/2606.21509/) (2026)
- [How Far Do Simple Transformations Translate Across Text Embedding Models?](https://mlsanity.com/paper/2608.05980/) (2026)
- [Train to Stitch: Topology-Preserving Regularization for Interoperable Representations](https://mlsanity.com/paper/2610.19401/) (2026)

## Abstract (arXiv)

It has been observed that representations learned by distinct neural networks conceal structural similarities when the models are trained under similar inductive biases. From a geometric perspective, identifying the classes of transformations and the related invariances that connect these representations is fundamental to unlocking applications, such as merging, stitching, and reusing different neural modules. However, estimating task-specific transformations a priori can be challenging and expensive due to several factors (e.g., weights initialization, training hyperparameters, or data modality). To this end, we introduce a versatile method to directly incorporate a set of invariances into the representations, constructing a product space of invariant components on top of the latent representations without requiring prior knowledge about the optimal invariance to infuse. We validate our solution on classification and reconstruction tasks, observing consistent latent similarity and downstream performance improvements in a zero-shot stitching setting. The experimental analysis comprises three modalities (vision, text, and graphs), twelve pretrained foundational models, nine benchmarks, and several architectures trained from scratch.