mlsanityData to 7 Oct 2026

LieTransformer: Equivariant self-attention for Lie Groups

arXiv 2012.10885 ↗First posted 20 Dec 2020cs.LG stat.MLcited by 100

Not on the map. Why?

Abstract (arXiv)

Group equivariant neural networks are used as building blocks of group invariant neural networks, which have been shown to improve generalisation performance and data efficiency through principled parameter sharing. Such works have mostly focused on group equivariant convolutions, building on the result that group equivariant linear maps are necessarily convolutions. In this work, we extend the scope of the literature to self-attention, that is emerging as a prominent building block of deep learning models. We propose the LieTransformer, an architecture composed of LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups. We demonstrate the generality of our approach by showing experimental results that are competitive to baseline methods on a wide range of tasks: shape counting on point clouds, molecular property regression and modelling particle trajectories under Hamiltonian dynamics.

Key facts. Cited by 100 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 24; 2023: 14; 2024: 18; 2025: 12; 2026: 14.

100 papers.

Sort:

By area:

  1. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  2. 2021Choose a Transformer: Fourier or Galerkincited by 333
  3. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 179
  4. 2021E(n) Equivariant Normalizing Flowscited by 165
  5. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  6. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 117
  7. 2021Geometric Deep Learning on Molecular Representationscited by 113
  8. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 112
  9. 2023EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoningcited by 97
  10. 2022Predicting Physics in Mesh-reduced Space with Temporal Attentioncited by 82
  11. 2021Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Dockingcited by 81
  12. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
38 more
  1. 2021SE(3) Equivariant Graph Neural Networks with Complete Local Framescited by 75
  2. 2022Equivariant Graph Mechanics Networks with Constraintscited by 67
  3. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 64
  4. 2022Geometrically Equivariant Graph Neural Networks: A Surveycited by 55
  5. 2022The Lie Derivative for Measuring Learned Equivariancecited by 51
  6. 2021Molformer: Motif-based Transformer on 3D Heterogeneous Molecular Graphscited by 44
  7. 2021Learning Partial Equivariances from Datacited by 44
  8. 2022Unsupervised Learning of Group Invariant and Equivariant Representationscited by 38
  9. 2022Energy-Motivated Equivariant Pretraining for 3D Molecular Graphscited by 37
  10. 2024Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Productscited by 35
  11. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  12. 2023E(2)-Equivariant Vision Transformercited by 27
  13. 2023SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biasescited by 27
  14. 2022Equivariant Graph Hierarchy-Based Neural Networkscited by 24
  15. 2022Symmetry Group Equivariant Architectures for Physicscited by 23
  16. 2022Transformer Meets Boundary Value Inverse Problemscited by 22
  17. 2023Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+cited by 22
  18. 2023Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materialscited by 22
  19. 2024E(n) Equivariant Topological Neural Networkscited by 22
  20. 2022Graph Neural Networks for Moleculescited by 21
  21. 2024Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structurescited by 20
  22. 2022E2PN: Efficient SE(3)-Equivariant Point Networkcited by 17
  23. 2021Equivariant Networks for Pixelized Spherescited by 16
  24. 2021Deep Learning for Bayesian Optimization of Scientific Problems with High-Dimensional Structurecited by 15
  25. 2021Group Equivariant Subsamplingcited by 15
  26. 2022An Analysis of Attention via the Lens of Exchangeability and Latent Variable Modelscited by 15
  27. 2021AIR-Nets: An Attention-Based Framework for Locally Conditioned Implicit Representationscited by 13
  28. 2022Equivariant Mesh Attention Networkscited by 12
  29. 2023Lie Group Decompositions for Equivariant Neural Networkscited by 12
  30. 2024RiEMann: Near Real-Time SE(3)-Equivariant Robot Manipulation without Point Cloud Segmentationcited by 12
  31. 2024Translation Equivariant Transformer Neural Processescited by 12
  32. 2021Iterative SE(3)-Transformerscited by 10
  33. 2022Continual Vision-based Reinforcement Learning with Group Symmetriescited by 10
  34. 2023Almost Equivariance via Lie Algebra Convolutionscited by 10
  35. 2024AI-driven inverse design of materials: Past, present and futurecited by 10
  36. 2022In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?cited by 9
  37. 2023Exploiting Code Symmetries for Learning Program Semanticscited by 9
  38. 2024Steerable Transformers for Volumetric Datacited by 8

50 most cited of 100 All 100 (JSON)