mlsanityData to 14 Oct 2026

SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks

arXiv 2006.10503 ↗First posted 18 Jun 2020cs.LG stat.MLcited by 607

Not on the map. Why?

Cited by 36 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce the SE(3)-Transformer, a variant of the self-attention module for 3D point clouds and graphs, which is equivariant under continuous 3D roto-translations. Equivariance is important to ensure stable and predictable performance in the presence of nuisance transformations of the data input. A positive corollary of equivariance is increased weight-tying within the model. The SE(3)-Transformer leverages the benefits of self-attention to operate on large point clouds and graphs with varying number of points, while guaranteeing SE(3)-equivariance for robustness. We evaluate our model on a toy N-body particle simulation dataset, showcasing the robustness of the predictions under rotations of the input. We further achieve competitive performance on two real-world datasets, ScanObjectNN and QM9. In all cases, our model outperforms a strong, non-equivariant attention baseline and an equivariant model without attention.

Key facts. Cited by 607 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 84; 2022: 103; 2023: 105; 2024: 113; 2025: 111; 2026: 91.

607 papers.

Sort:

By area:

  1. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  2. 2021E(n) Equivariant Graph Neural Networkscited by 965
  3. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 677
  4. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 516
  5. 2021Choose a Transformer: Fourier or Galerkincited by 343
  6. 2021GemNet: Universal Directional Graph Neural Networks for Moleculescited by 343
  7. 2021Vector Neurons: A General Framework for SO(3)-Equivariant Networkscited by 305
  8. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  9. 2021Crystal Diffusion Variational Autoencoder for Periodic Material Generationcited by 270
  10. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 264
  11. 2022Learning Local Equivariant Representations for Large-Scale Atomistic Dynamicscited by 256
  12. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
38 more
  1. 2021Geometric and Physical Quantities Improve E(3) Equivariant Message Passingcited by 228
  2. 2023EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representationscited by 204
  3. 2021A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscited by 187
  4. 2023Geometric Latent Diffusion Models for 3D Molecule Generationcited by 186
  5. 2022Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pocketscited by 183
  6. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 182
  7. 2022Graph neural networks for materials science and chemistrycited by 182
  8. 20233D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Predictioncited by 177
  9. 2022e3nn: Euclidean Neural Networkscited by 174
  10. 2021Spherical Message Passing for 3D Graph Networkscited by 168
  11. 2021Frame Averaging for Invariant and Equivariant Network Designcited by 168
  12. 2021E(n) Equivariant Normalizing Flowscited by 165
  13. 2021Learning Gradient Fields for Molecular Conformation Generationcited by 163
  14. 2023Crystal Structure Prediction by Joint Equivariant Diffusioncited by 148
  15. 2022EquiBind: Geometric Deep Learning for Drug Binding Structure Predictioncited by 144
  16. 2021Scalars are universal: Equivariant machine learning, structured like classical physicscited by 127
  17. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  18. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 120
  19. 2022Sym-NCO: Leveraging Symmetricity for Neural Combinatorial Optimizationcited by 119
  20. 2023Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNscited by 117
  21. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 113
  22. 2021Geometric Deep Learning on Molecular Representationscited by 113
  23. 2022Equivariance with Learned Canonicalization Functionscited by 109
  24. 2022Transformer for Graphs: An Overview from Architecture Perspectivecited by 107
  25. 2023EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoningcited by 97
  26. 2023On the Expressive Power of Geometric Graph Neural Networkscited by 90
  27. 2024DecompDiff: Diffusion Models with Decomposed Priors for Structure-Based Drug Designcited by 88
  28. 2022Periodic Graph Transformers for Crystal Material Property Predictioncited by 87
  29. 2023Everything is Connected: Graph Neural Networkscited by 85
  30. 2023Attending to Graph Transformerscited by 83
  31. 2021Independent SE(3)-Equivariant Models for End-to-End Rigid Protein Dockingcited by 82
  32. 2021SE(3) Equivariant Graph Neural Networks with Complete Local Framescited by 76
  33. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 75
  34. 2022Equivariant Energy-Guided SDE for Inverse Molecular Designcited by 73
  35. 2023FAENet: Frame Averaging Equivariant GNN for Materials Modelingcited by 73
  36. 2023A Comprehensive Survey on Deep Graph Representation Learningcited by 72
  37. 2023RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmarkcited by 71
  38. 2022ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphscited by 69

50 most cited of 607 All 607 (JSON)