mlsanityData to 7 Oct 2026

Iterative SE(3)-Transformers

arXiv 2102.13419 ↗First posted 26 Feb 2021cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

When manipulating three-dimensional data, it is possible to ensure that rotational and translational symmetries are respected by applying so-called SE(3)-equivariant models. Protein structure prediction is a prominent example of a task which displays these symmetries. Recent work in this area has successfully made use of an SE(3)-equivariant model, applying an iterative SE(3)-equivariant attention mechanism. Motivated by this application, we implement an iterative version of the SE(3)-Transformer, an SE(3)-equivariant attention-based model for graph data. We address the additional complications which arise when applying the SE(3)-Transformer in an iterative fashion, compare the iterative and single-pass versions on a toy problem, and consider why an iterative model may be beneficial in some problem settings. We make the code for our implementation available to the community.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 26 Feb 2021.

Builds on

20 of 36 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Graph Attention Networkscited by 8,854
  3. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  4. 2017Non-local Neural Networkscited by 2,290
  5. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  6. 2016Group Equivariant Convolutional Networkscited by 1,154
  7. 2018Self-Attention with Relative Position Representationscited by 1,112
  8. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  9. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  10. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  11. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  12. 2018Neural Relational Inference for Interacting Systemscited by 339
8 more
  1. 2017A Structured Self-attentive Sentence Embeddingcited by 318
  2. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
  3. 2019Modeling Point Clouds with Self-Attention and Gumbel Subset Samplingcited by 151
  4. 2018Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networkscited by 129
  5. 2018Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactionscited by 100
  6. 2020LieTransformer: Equivariant self-attention for Lie Groupscited by 100
  7. 2017VAIN: Attentional Multi-agent Predictive Modelingcited by 51
  8. 2018N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentialscited by 42

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 2; 2023: 3; 2024: 2.

10 papers.

  1. 2023Attending to Graph Transformerscited by 81
  2. 2021SE(3) Equivariant Graph Neural Networks with Complete Local Framescited by 75
  3. 2023E(n) Equivariant Message Passing Simplicial Networkscited by 23
  4. 2022Geometry-Complete Perceptron Networks for 3D Molecular Graphscited by 15
  5. 2022Improving the Sample-Complexity of Deep Classification Networks with Invariant Integrationcited by 7
  6. 2021EBM-Fold: Fully-Differentiable Protein Folding Powered by Energy-based Modelscited by 3
  7. 2023Deep Neural Networks with Efficient Guaranteed Invariancescited by 2
  8. 2024Current Symmetry Group Equivariant Convolution Frameworks for Representation Learningcited by 2
  9. 2024Equi-GSPR: Equivariant SE(3) Graph Network Model for Sparse Point Cloud Registrationcited by 2
  10. 2021ChebLieNet: Invariant Spectral Graph NNs Turned Equivariant by Riemannian Geometry on Lie Groupscited by 1