# Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning

arXiv: https://arxiv.org/abs/2306.01474 · HTML: https://mlsanity.com/paper/2306.01474/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 103 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 30 of 55)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [Directional Message Passing for Molecular Graphs](https://mlsanity.com/paper/2003.03123/) (2020)
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](https://mlsanity.com/paper/2203.02923/) (2022)
- [MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields](https://mlsanity.com/paper/2206.07697/) (2022)
- [Equivariant message passing for the prediction of tensorial properties and molecular spectra](https://mlsanity.com/paper/2102.03150/) (2021)
- [GemNet: Universal Directional Graph Neural Networks for Molecules](https://mlsanity.com/paper/2106.08903/) (2021)
- [Learning from Protein Structure with Geometric Vector Perceptrons](https://mlsanity.com/paper/2009.01411/) (2020)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [Learning Local Equivariant Representations for Large-Scale Atomistic Dynamics](https://mlsanity.com/paper/2204.05249/) (2022)
- [Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules](https://mlsanity.com/paper/2011.14115/) (2020)
- [The Open Catalyst 2020 (OC20) Dataset and Community Challenges](https://mlsanity.com/paper/2010.09990/) (2020)
- [Evaluating Protein Transfer Learning with TAPE](https://mlsanity.com/paper/1906.08230/) (2019)
- [Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets](https://mlsanity.com/paper/2205.07249/) (2022)
- [TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials](https://mlsanity.com/paper/2202.02541/) (2022)
- [DeepDTA: Deep Drug-Target Binding Affinity Prediction](https://mlsanity.com/paper/1801.10193/) (2018)
- [E(n) Equivariant Normalizing Flows](https://mlsanity.com/paper/2105.09016/) (2021)
- [Protein Representation Learning by Geometric Structure Pretraining](https://mlsanity.com/paper/2203.06125/) (2022)
- [A 3D Generative Model for Structure-Based Drug Design](https://mlsanity.com/paper/2203.10446/) (2022)
- [3D Infomax improves GNNs for Molecular Property Prediction](https://mlsanity.com/paper/2110.04126/) (2021)
- [Pre-training via Denoising for Molecular Property Prediction](https://mlsanity.com/paper/2206.00133/) (2022)

## Built on by (most cited first, 19 of 19)

- [A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications](https://mlsanity.com/paper/2403.00485/) (2024)
- [Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks?](https://mlsanity.com/paper/2410.11443/) (2024)
- [ProFSA: Self-supervised Pocket Pretraining via Protein Fragment-Surroundings Alignment](https://mlsanity.com/paper/2310.07229/) (2023)
- [UniMoMo: Unified Generative Modeling of 3D Molecules for De Novo Binder Design](https://mlsanity.com/paper/2503.19300/) (2025)
- [UniIF: Unified Molecule Inverse Folding](https://mlsanity.com/paper/2405.18968/) (2024)
- [ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling](https://mlsanity.com/paper/2403.12995/) (2024)
- [An Equivariant Pretrained Transformer for Unified 3D Molecular Representation Learning](https://mlsanity.com/paper/2402.12714/) (2024)
- [Tokenizing Electron Cloud in Protein-Ligand Interaction Learning](https://mlsanity.com/paper/2505.19014/) (2025)
- [Universally Invariant Learning in Equivariant GNNs](https://mlsanity.com/paper/2510.13169/) (2025)
- [Unified all-atom molecule generation with neural fields](https://mlsanity.com/paper/2511.15906/) (2025)
- [Geometric Deep Learning for Structure-Based Drug Design: A Survey](https://mlsanity.com/paper/2306.11768/) (2023)
- [HeMeNet: Heterogeneous Multichannel Equivariant Network for Protein Multitask Learning](https://mlsanity.com/paper/2404.01693/) (2024)
- [Learning Protein-Ligand Binding in Hyperbolic Space](https://mlsanity.com/paper/2508.15480/) (2025)
- [Unified Biomolecular Trajectory Generation via Pretrained Variational Bridge](https://mlsanity.com/paper/2602.07588/) (2026)
- [From Macro to Micro: Benchmarking Microscopic Spatial Intelligence on Molecules via Vision-Language Models](https://mlsanity.com/paper/2512.10867/) (2025)
- [h-MINT: Modeling Pocket-Ligand Binding with Hierarchical Molecular Interaction Network](https://mlsanity.com/paper/2604.23134/) (2026)
- [CARD: Coarse-to-fine Autoregressive Modeling with Radix-based Decomposition for Transferable Free Energy Estimation](https://mlsanity.com/paper/2605.02657/) (2026)
- [Curvature-Informed Potential Energy Surface for Protein-Ligand Binding Affinity Prediction](https://mlsanity.com/paper/2606.14217/) (2026)
- [Blending experimental electron density information to protein-ligand interaction](https://mlsanity.com/paper/2610.14862/) (2026)

## Abstract (arXiv)

Many processes in biology and drug discovery involve various 3D interactions between molecules, such as protein and protein, protein and small molecule, etc. Given that different molecules are usually represented in different granularity, existing methods usually encode each type of molecules independently with different models, leaving it defective to learn the various underlying interaction physics. In this paper, we first propose to universally represent an arbitrary 3D complex as a geometric graph of sets, shedding light on encoding all types of molecules with one model. We then propose a Generalist Equivariant Transformer (GET) to effectively capture both domain-specific hierarchies and domain-agnostic interaction physics. To be specific, GET consists of a bilevel attention module, a feed-forward module and a layer normalization module, where each module is E(3) equivariant and specialized for handling sets of variable sizes. Notably, in contrast to conventional pooling-based hierarchical models, our GET is able to retain fine-grained information of all levels. Extensive experiments on the interactions between proteins, small molecules and RNA/DNAs verify the effectiveness and generalization capability of our proposed method across different domains.