# Learning Group Actions In Disentangled Latent Image Representations

arXiv: https://arxiv.org/abs/2512.04015 · HTML: https://mlsanity.com/paper/2512.04015/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Builds on (most cited first, 30 of 31)

- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [Learning Steerable Filters for Rotation Equivariant CNNs](https://mlsanity.com/paper/1711.07289/) (2017)
- [Theory for Equivariant Quantum Neural Networks](https://mlsanity.com/paper/2210.08566/) (2022)
- [Clifford Group Equivariant Neural Networks](https://mlsanity.com/paper/2305.11141/) (2023)
- [CausalVAE: Structured Causal Disentanglement in Variational Autoencoder](https://mlsanity.com/paper/2004.08697/) (2020)
- [Equivariant Neural Rendering](https://mlsanity.com/paper/2006.07630/) (2020)
- [GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNs](https://mlsanity.com/paper/1911.05932/) (2019)
- [Unsupervised Learning of Group Invariant and Equivariant Representations](https://mlsanity.com/paper/2202.07559/) (2022)
- [Learning the Effect of Registration Hyperparameters with HyperMorph](https://mlsanity.com/paper/2203.16680/) (2022)
- [Latent Space Symmetry Discovery](https://mlsanity.com/paper/2310.00105/) (2023)
- [Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curve](https://mlsanity.com/paper/2212.03905/) (2022)
- [Brauer's Group Equivariant Neural Networks](https://mlsanity.com/paper/2212.08630/) (2022)
- [Color-based Segmentation of Sky/Cloud Images From Ground-based Cameras](https://mlsanity.com/paper/1606.03669/) (2016)
- [Invariant Shape Representation Learning For Image Classification](https://mlsanity.com/paper/2411.12201/) (2024)
- [Graph Automorphism Group Equivariant Neural Networks](https://mlsanity.com/paper/2307.07810/) (2023)
- [MGAug: Multimodal Geometric Augmentation in Latent Spaces of Image Deformations](https://mlsanity.com/paper/2312.13440/) (2023)
- [CMVAE: Causal Meta VAE for Unsupervised Meta-Learning](https://mlsanity.com/paper/2302.09731/) (2023)
- [TPIE: Topology-Preserved Image Editing With Text Instructions](https://mlsanity.com/paper/2411.16714/) (2024)
- [Denoising Multi-Beta VAE: Representation Learning for Disentanglement and Generation](https://mlsanity.com/paper/2507.06613/) (2025)
- [CoRLD: Contrastive Representation Learning Of Deformable Shapes In Images](https://mlsanity.com/paper/2503.17162/) (2025)
- [Variantional autoencoder with decremental information bottleneck for disentanglement](https://mlsanity.com/paper/2303.12959/) (2023)

## Abstract (arXiv)

Modeling group actions on latent representations enables controllable transformations of high-dimensional image data. Prior works applying group-theoretic priors or modeling transformations typically operate in the high-dimensional data space, where group actions apply uniformly across the entire input, making it difficult to disentangle the subspace that varies under transformations. While latent-space methods offer greater flexibility, they still require manual partitioning of latent variables into equivariant and invariant subspaces, limiting the ability to robustly learn and operate group actions within the representation space. To address this, we introduce a novel end-to-end framework that for the first time learns group actions on latent image manifolds, automatically discovering transformation-relevant structures without manual intervention. Our method uses learnable binary masks with straight-through estimation to dynamically partition latent representations into transformation-sensitive and invariant components. We formulate this within a unified optimization framework that jointly learns latent disentanglement and group transformation mappings. The framework can be seamlessly integrated with any standard encoder-decoder architecture. We validate our approach on five 2D/3D image datasets, demonstrating its ability to automatically learn disentangled latent factors for group actions in diverse data, while downstream classification tasks confirm the effectiveness of the learned representations. Our code is publicly available at https://github.com/farhanaswarnali/Learning-Group-Actions-In-Disentangled-Latent-Image-Representations .