mlsanityData to 7 Oct 2026

A Generative Model of Symmetry Transformations

arXiv 2403.01946 ↗First posted 4 Mar 2024cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Correctly capturing the symmetry transformations of data can lead to efficient models with strong generalization capabilities, though methods incorporating symmetries often require prior knowledge. While recent advancements have been made in learning those symmetries directly from the dataset, most of this work has focused on the discriminative setting. In this paper, we take inspiration from group theoretic ideas to construct a generative model that explicitly aims to capture the data's approximate symmetries. This results in a model that, given a prespecified but broad set of possible symmetries, learns to what extent, if at all, those symmetries are actually present. Our model can be seen as a generative process for data augmentation. We provide a simple algorithm for learning our generative model and empirically demonstrate its ability to capture symmetries under affine and color transformations, in an interpretable way. Combining our symmetry model with standard generative models results in higher marginal test-log-likelihoods and improved data efficiency.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 41 of its 57 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

41 of 57 references.

References · 41

  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  2. 2015Spatial Transformer Networkscited by 1,423
  3. 2016Group Equivariant Convolutional Networkscited by 1,154
  4. 2019Neural Spline Flowscited by 590
  5. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  6. 2018Towards a Definition of Disentangled Representationscited by 294
  7. 2015The Variational Fair Autoencodercited by 223
  8. 2019DIVA: Domain Invariant Variational Autoencoderscited by 114
  9. 2022Equivariance with Learned Canonicalization Functionscited by 106
  10. 2017Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observationscited by 100
  11. 2021Automatic Symmetry Discovery with Lie Algebra Convolutional Networkcited by 82
  12. 2022The Effects of Regularization and Data Augmentation are Class Dependentcited by 67
29 more
  1. 2018Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearancecited by 58
  2. 2018Importance Weighting and Variational Inferencecited by 55
  3. 2018Learning Invariances using the Marginal Likelihoodcited by 51
  4. 2021Lossy Compression for Lossless Predictioncited by 51
  5. 2021Learning Partial Equivariances from Datacited by 44
  6. 2022Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximationscited by 42
  7. 2021Galaxy Zoo DECaLS: Detailed Visual Morphology Measurements from Volunteers and Deep Learning for 314,000 Galaxiescited by 40
  8. 2023Generative Adversarial Symmetry Discoverycited by 40
  9. 2023Equivariant Adaptation of Large Pretrained Modelscited by 40
  10. 2022Unsupervised Learning of Group Invariant and Equivariant Representationscited by 38
  11. 2021Grounding inductive biases in natural images:invariance stems from variations in datacited by 25
  12. 2023Learning Probabilistic Symmetrization for Architecture Agnostic Equivariancecited by 25
  13. 2021Last Layer Marginal Likelihood for Invariance Learningcited by 21
  14. 2022Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitionscited by 18
  15. 2022Learning Invariant Weights in Neural Networkscited by 16
  16. 2023Self-Supervised Disentanglement by Leveraging Structure in Data Augmentationscited by 16
  17. 2021Group Equivariant Subsamplingcited by 15
  18. 2022Learning Instance-Specific Augmentations by Capturing Local Invariancescited by 15
  19. 2020Disentangling images with Lie group transformations and sparse codingcited by 14
  20. 2021Addressing the Topological Defects of Disentanglement via Distributed Operatorscited by 14
  21. 2019Disentangling and Learning Robust Representations with Natural Clusteringcited by 10
  22. 2023Stochastic Marginal Likelihood Gradients using Neural Tangent Kernelscited by 10
  23. 2022Equivariance-aware Architectural Optimization of Neural Networkscited by 8
  24. 2023Hyperparameter Optimization through Neural Network Partitioningcited by 8
  25. 2022Deep invariant networks with differentiable augmentation layerscited by 6
  26. 2017Unsupervised Transformation Learning via Convex Relaxationscited by 5
  27. 2017Learning Approximately Objective Priorscited by 3
  28. 2022Equivariant Priors for Compressed Sensing with Unknown Orientationcited by 3
  29. 2023Conditionally Invariant Representation Learning for Disentangling Cellular Heterogeneitycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 1.

10 papers.

  1. 2024Symmetry Discovery for Different Data Typescited by 7
  2. 2025Test-Time Canonicalization by Foundation Models for Robust Perceptioncited by 5
  3. 2025Symmetry-Aware Generative Modeling through Learned Canonicalizationcited by 4
  4. 2025Symmetry Understanding of 3D Shapes via Chirality Disentanglementcited by 3
  5. 2025Discovering Symmetry Groups with Flow Matchingcited by 3
  6. 2025CP2: Leveraging Geometry for Conformal Prediction via Canonicalizationcited by 2
  7. 2025Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actionscited by 2
  8. 2026Inverting Data Transformations via Diffusion Samplingcited by 2
  9. 2025RECON: Robust symmetry discovery via Explicit Canonical Orientation Normalizationcited by 1
  10. 2025Symmetry Matters: Auditing and Symmetrizing 3D Generative Modelscited by 0