# Improving Relative Representations with Learned Anchors and Whitened Inner Products

arXiv: https://arxiv.org/abs/2605.30596 · HTML: https://mlsanity.com/paper/2605.30596/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 28 May 2026.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [The Multilingual Amazon Reviews Corpus](https://mlsanity.com/paper/2010.02573/) (2020)
- [Anisotropy Is Inherent to Self-Attention in Transformers](https://mlsanity.com/paper/2401.12143/) (2024)
- [Rethinking Channel Dimensions for Efficient Model Design](https://mlsanity.com/paper/2007.00992/) (2020)
- [Latent Space Translation via Inverse Relative Projection](https://mlsanity.com/paper/2406.15057/) (2024)
- [On the Importance of Embedding Norms in Self-Supervised Learning](https://mlsanity.com/paper/2502.09252/) (2025)
- [Relative Representations of Latent Spaces enable Efficient Semantic Channel Equalization](https://mlsanity.com/paper/2411.19719/) (2024)

## Built on by (most cited first, 1 of 1)

- [Sheaf-Based Federated Representation Learning](https://mlsanity.com/paper/2608.10016/) (2026)

## Abstract (arXiv)

Independently trained neural models typically converge to incompatible latent representations, creating a fundamental barrier to highly modular AI systems. While Relative Representations (RR) address this by mapping absolute coordinates to a shared space defined by similarities to common anchor points, traditional implementations rely on randomly sampled anchors and cosine similarity, which frequently fail to capture the anisotropic geometries of modern architectures like Transformers. In this work, we propose a robust framework for cross-model communication based on two improvements. We learn anchors as robust semantic prototypes and utilize a geometry-aware similarity metric which preserves discriminative magnitude information and is invariant to affine shifts. Our approach demonstrates significant gains in performance and consistency across vision and language tasks. Notably, it enables nearly lossless information transfer and stable zero-shot communication even between highly heterogeneous architectures, such as small language models of varying scales.