mlsanityData to 7 Oct 2026

Learning Invariances using the Marginal Likelihood

arXiv 1808.05563 ↗First posted 16 Aug 2018cs.LG stat.MLcited by 51

Not on the map. Why?

Abstract (arXiv)

Generalising well in supervised learning tasks relies on correctly extrapolating the training data to a large region of the input space. One way to achieve this is to constrain the predictions to be invariant to transformations on the input that are known to be irrelevant (e.g. translation). Commonly, this is done through data augmentation, where the training set is enlarged by applying hand-crafted transformations to the inputs. We argue that invariances should instead be incorporated in the model structure, and learned using the marginal likelihood, which correctly rewards the reduced complexity of invariant models. We demonstrate this for Gaussian process models, due to the ease with which their marginal likelihood can be estimated. Our main contribution is a variational inference scheme for Gaussian processes containing invariances described by a sampling procedure. We learn the sampling procedure by back-propagating through it to maximise the marginal likelihood.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 11; 2023: 9; 2024: 5; 2025: 6; 2026: 1.

51 papers.

Sort:

  1. 2021Bayesian Neural Network Priors Revisitedcited by 121
  2. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  3. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  4. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  5. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 74
  6. 2022Bayesian Model Selection, the Marginal Likelihood, and Generalizationcited by 56
  7. 2023Time Series Contrastive Learning with Information-Aware Augmentationscited by 52
  8. 2022DEVIANT: Depth EquiVarIAnt NeTwork for Monocular 3D Object Detectioncited by 48
  9. 2021Learning Partial Equivariances from Datacited by 44
  10. 2022Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximationscited by 42
  11. 2022Relaxing Equivariance Constraints with Non-stationary Continuous Filterscited by 41
  12. 2021Data augmentation in Bayesian neural networks and the cold posterior effectcited by 36
38 more
  1. 2021Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fixcited by 27
  2. 2021Deep Neural Networks as Point Estimates for Deep Gaussian Processescited by 26
  3. 2021Grounding inductive biases in natural images:invariance stems from variations in datacited by 25
  4. 2023Learning Probabilistic Symmetrization for Architecture Agnostic Equivariancecited by 25
  5. 2023Learning Layer-wise Equivariances Automatically using Gradientscited by 23
  6. 2021Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effectcited by 22
  7. 2021Last Layer Marginal Likelihood for Invariance Learningcited by 21
  8. 2022Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learningcited by 20
  9. 2022Learning Invariant Weights in Neural Networkscited by 16
  10. 2023The Geometry of Neural Nets' Parameter Spaces Under Reparametrizationcited by 16
  11. 2021Learning Augmentation Distributions using Transformed Risk Minimizationcited by 13
  12. 2024A Complexity-Based Theory of Compositionalitycited by 13
  13. 2023Modulated Neural ODEscited by 10
  14. 2023Stochastic Marginal Likelihood Gradients using Neural Tangent Kernelscited by 10
  15. 2024A Generative Model of Symmetry Transformationscited by 10
  16. 2022Regularising for invariance to data augmentation improves supervised learningcited by 9
  17. 2023Incorporating Unlabelled Data into Bayesian Neural Networkscited by 9
  18. 2021Neural Networks for Learning Counterfactual G-Invariances from Single Environmentscited by 8
  19. 2021Scalable Variational Gaussian Processes via Harmonic Kernel Decompositioncited by 8
  20. 2022Adversarially Robust Models may not Transfer Better: Sufficient Conditions for Domain Transferability from the View of Regularizationcited by 8
  21. 2023Hyperparameter Optimization through Neural Network Partitioningcited by 8
  22. 2022Deep invariant networks with differentiable augmentation layerscited by 6
  23. 2025Koopman-Equivariant Gaussian Processescited by 6
  24. 2023Using and Abusing Equivariancecited by 4
  25. 2024Noether's razor: Learning Conserved Quantitiescited by 4
  26. 2024Sample-efficient Bayesian Optimisation Using Known Invariancescited by 4
  27. 2024A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learningcited by 3
  28. 2021Invariant Priors for Bayesian Quadraturecited by 2
  29. 2022A Simple Strategy to Provable Invariance via Orbit Mappingcited by 2
  30. 2025InvZW: Invariant Feature Learning via Noise-Adversarial Training for Robust Image Zero-Watermarkingcited by 2
  31. 2025Fine-Tuned In-Context Learners for Efficient Adaptationcited by 2
  32. 2021Rotating spiders and reflecting dogs: a class conditional approach to learning data augmentation distributionscited by 1
  33. 2021A Bayesian Approach to Invariant Deep Neural Networkscited by 1
  34. 2021Kernel Interpolation as a Bayes Point Machinecited by 1
  35. 2022A tradeoff between universality of equivariant models and learnability of symmetriescited by 1
  36. 2025Optimizing Data Augmentation through Bayesian Model Selectioncited by 1
  37. 2025On Equivariant Model Selection through the Lens of Uncertaintycited by 1
  38. 2025Reinforcement Learning Using known Invariancescited by 0

50 most cited of 51 All 51 (JSON)