# Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actions

arXiv: https://arxiv.org/abs/2510.21706 · HTML: https://mlsanity.com/paper/2510.21706/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 58 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Builds on (most cited first, 30 of 42)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [What Should Not Be Contrastive in Contrastive Learning](https://mlsanity.com/paper/2008.05659/) (2020)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Unsupervised Learning of Disentangled Representations from Video](https://mlsanity.com/paper/1705.10915/) (2017)
- [Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding](https://mlsanity.com/paper/2007.10930/) (2020)
- [Learnable latent embeddings for joint behavioral and neural analysis](https://mlsanity.com/paper/2204.00673/) (2022)
- [On Linear Identifiability of Learned Representations](https://mlsanity.com/paper/2007.00810/) (2020)
- [Automatic Symmetry Discovery with Lie Algebra Convolutional Network](https://mlsanity.com/paper/2109.07103/) (2021)
- [Unsupervised Part-Based Disentangling of Object Shape and Appearance](https://mlsanity.com/paper/1903.06946/) (2019)
- [Learning Symmetric Embeddings for Equivariant World Models](https://mlsanity.com/paper/2204.11371/) (2022)
- [Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE](https://mlsanity.com/paper/2011.04798/) (2020)
- [Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning](https://mlsanity.com/paper/2303.16535/) (2023)
- [Self-supervised learning of Split Invariant Equivariant representations](https://mlsanity.com/paper/2302.10283/) (2023)
- [Generative Adversarial Symmetry Discovery](https://mlsanity.com/paper/2302.00236/) (2023)
- [Unsupervised Learning of Group Invariant and Equivariant Representations](https://mlsanity.com/paper/2202.07559/) (2022)
- [Large-scale neural recordings call for new insights to link brain and behavior](https://mlsanity.com/paper/2103.14662/) (2021)
- [Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning](https://mlsanity.com/paper/2306.13924/) (2023)
- [Learning Generalized Transformation Equivariant Representations via Autoencoding Transformations](https://mlsanity.com/paper/1906.08628/) (2019)

## Built on by (most cited first, 2 of 2)

- [Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation](https://mlsanity.com/paper/2606.18509/) (2026)
- [Discovering Physical Representation Languages](https://mlsanity.com/paper/2609.23381/) (2026)

## Abstract (arXiv)

We propose Equivariance by Contrast (EbC) to learn equivariant embeddings from observation pairs (y, g · y), where g is drawn from a finite group acting on the data. Our method jointly learns a latent space and a group representation in which group actions correspond to invertible linear maps -- without relying on group-specific inductive biases. We validate our approach on the infinite dSprites dataset with structured transformations defined by the finite group G:= (Rm × mathbbZn × mathbbZn), combining discrete rotations and periodic translations. The resulting embeddings exhibit high-fidelity equivariance, with group operations faithfully reproduced in latent space. On synthetic data, we further validate the approach on the non-abelian orthogonal group O(n) and the general linear group GL(n). We also provide a theoretical proof for identifiability. While broad evaluation across diverse group types on real-world data remains future work, our results constitute the first successful demonstration of general-purpose encoder-only equivariant learning from group action observations alone, including non-trivial non-abelian groups and a product group motivated by modeling affine equivariances in computer vision.