mlsanityData to 14 Oct 2026

Residual Pathway Priors for Soft Equivariance Constraints

arXiv 2112.01388 ↗First posted 2 Dec 2021cs.LG stat.MLcited by 75

Not on the map. Why?

Abstract (arXiv)

There is often a trade-off between building deep learning systems that are expressive enough to capture the nuances of the reality, and having the right inductive biases for efficient learning. We introduce Residual Pathway Priors (RPPs) as a method for converting hard architectural constraints into soft priors, guiding models towards structured solutions, while retaining the ability to capture additional complexity. Using RPPs, we construct neural network priors with inductive biases for equivariances, but without limiting flexibility. We show that RPPs are resilient to approximate or misspecified symmetries, and are as effective as fully constrained models even when symmetries are exact. We showcase the broad applicability of RPPs with dynamical systems, tabular data, and reinforcement learning. In Mujoco locomotion tasks, where contact forces and directional rewards violate strict equivariance assumptions, the RPP outperforms baseline model-free RL agents, and also improves the learned transition models for model-based RL.

Key facts. Cited by 75 papers in the map (data through 14 Oct 2026). Builds on 49 of its 62 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

49 of 62 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2018Neural Ordinary Differential Equationscited by 3,779
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  7. 2016OpenAI Gymcited by 1,736
  8. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  9. 2017Deep Setscited by 1,421
  10. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  11. 2016Group Equivariant Convolutional Networkscited by 1,165
  12. 2021E(n) Equivariant Graph Neural Networkscited by 965
37 more
  1. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  2. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  3. 2019Hamiltonian Neural Networkscited by 712
  4. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  5. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  6. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  7. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  8. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  9. 2019General E(2)-Equivariant Steerable CNNscited by 446
  10. 2021Early Convolutions Help Transformers See Bettercited by 396
  11. 2016Learning Multiagent Communication with Backpropagationcited by 366
  12. 2018Invariant and Equivariant Graph Networkscited by 346
  13. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 342
  14. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
  15. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 295
  16. 2018Residual Reinforcement Learning for Robot Controlcited by 274
  17. 2019Cormorant: Covariant Molecular Neural Networkscited by 263
  18. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  19. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 187
  20. 2018Residual Policy Learningcited by 164
  21. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 162
  22. 2018Graph Convolutional Reinforcement Learningcited by 152
  23. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 142
  24. 2020Lorentz Group Equivariant Neural Network for Particle Physicscited by 121
  25. 2020On Learning Sets of Symmetric Elementscited by 98
  26. 2021Provably Strict Generalisation Benefit for Equivariant Modelscited by 91
  27. 2019Exploring Model-based Planning with Policy Networkscited by 90
  28. 2017Oriented Response Networkscited by 68
  29. 2020Plannable Approximations to MDP Homomorphisms: Equivariance under Actionscited by 61
  30. 2020Learning Invariances in Neural Networkscited by 52
  31. 2018Learning Invariances using the Marginal Likelihoodcited by 51
  32. 2019PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learningcited by 47
  33. 2020On the model-based stochastic value gradient for continuous reinforcement learningcited by 47
  34. 2020Towards Learning Convolutions from Scratchcited by 46
  35. 2020Equivariant Maps for Hierarchical Structurescited by 11
  36. 2021Physics-Augmented Learning: A New Paradigm Beyond Physics-Informed Learningcited by 11
  37. 2020Goal-conditioned Batch Reinforcement Learning for Rotation Invariant Locomotioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 9; 2023: 17; 2024: 14; 2025: 20; 2026: 15.

75 papers.

Sort:

By area:

  1. 2022Approximately Equivariant Networks for Imperfectly Symmetric Dynamicscited by 97
  2. 2023The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learningcited by 63
  3. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 62
  4. 2025Deep Learning is Not So Mysterious or Differentcited by 56
  5. 2023Approximation-Generalization Trade-offs under (Approximate) Group Equivariancecited by 46
  6. 2022Relaxing Equivariance Constraints with Non-stationary Continuous Filterscited by 41
  7. 2024The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials Across Chemical Domainscited by 40
  8. 2022LieGG: Studying Learned Lie Group Generatorscited by 31
  9. 2022VN-Transformer: Rotation-Equivariant Attention for Vector Neuronscited by 27
  10. 2022Symmetry Group Equivariant Architectures for Physicscited by 23
  11. 2023Learning Layer-wise Equivariances Automatically using Gradientscited by 23
  12. 2024Improving Equivariant Model Training via Constraint Relaxationcited by 23
38 more
  1. 2023Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolutioncited by 20
  2. 2022Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecastingcited by 17
  3. 2022Online Dynamics Learning for Predictive Control with an Application to Aerial Robotscited by 17
  4. 2023Regularizing Towards Soft Equivariance Under Mixed Symmetriescited by 16
  5. 2023Approximately Equivariant Graph Networkscited by 16
  6. 2024Morphological Symmetries in Roboticscited by 13
  7. 2024Relaxed Equivariance via Multitask Learningcited by 13
  8. 2022Knowledge-augmented Deep Learning and Its Applications: A Surveycited by 10
  9. 2023Almost Equivariance via Lie Algebra Convolutionscited by 10
  10. 2023SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRIcited by 10
  11. 2025SE(3)-Equivariant Robot Learning and Control: A Tutorial Surveycited by 10
  12. 2025Learning (Approximately) Equivariant Networks via Constrained Optimizationcited by 10
  13. 2024Approximate Equivariance in Reinforcement Learningcited by 9
  14. 2022Equivariance-aware Architectural Optimization of Neural Networkscited by 8
  15. 2023On discrete symmetries of robotics systems: A group-theoretic and data-driven analysiscited by 8
  16. 2024Approximately Equivariant Neural Processescited by 8
  17. 2023Understanding when Dynamics-Invariant Data Augmentations Benefit Model-Free Reinforcement Learning Updatescited by 7
  18. 2023E(2) Equivariant Neural Networks for Robust Galaxy Morphology Classificationcited by 7
  19. 2024Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networkscited by 7
  20. 2025Achieving Approximate Symmetry Is Exponentially Easier than Exact Symmetrycited by 7
  21. 2025Discovering Symbolic Differential Equations with Symmetry Invariantscited by 6
  22. 2023Equivariance Is Not All You Need: Characterizing the Utility of Equivariant Graph Neural Networks for Particle Physics Taskscited by 5
  23. 2024Approximately-symmetric neural networks for quantum spin liquidscited by 5
  24. 2024Learning Symmetries via Weight-Sharing with Doubly Stochastic Tensorscited by 5
  25. 2025AtlasD: Automatic Local Symmetry Discoverycited by 5
  26. 2025Visual Instruction Bottleneck Tuningcited by 5
  27. 2025A Regularization-Guided Equivariant Approach for Image Restorationcited by 5
  28. 2023Perspectives on the State and Future of Deep Learning - 2023cited by 4
  29. 2023Self-Supervised Detection of Perfect and Partial Input-Dependent Symmetriescited by 4
  30. 2024Equivariant Ensembles and Regularization for Reinforcement Learning in Map-based Path Planningcited by 4
  31. 2024Noether's razor: Learning Conserved Quantitiescited by 4
  32. 2026Data Augmentation: A Fourier Analysis Perspectivecited by 4
  33. 2025On Universality of Deep Equivariant Networkscited by 3
  34. 2025Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environmentscited by 3
  35. 2025Generalized Spherical Neural Operators: Green's Function Formulationcited by 3
  36. 2023Learning to Transform for Generalizable Instance-wise Invariancecited by 2
  37. 2024Current Symmetry Group Equivariant Convolution Frameworks for Representation Learningcited by 2
  38. 2025PEnGUiN: Partially Equivariant Graph NeUral Networks for Sample Efficient MARLcited by 2

50 most cited of 75 All 75 (JSON)