# Large Language-Geometry Model: When LLM meets Equivariance

arXiv: https://arxiv.org/abs/2502.11149 · HTML: https://mlsanity.com/paper/2502.11149/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 16 Feb 2025.

## Builds on (most cited first, 24 of 24)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [E(n) Equivariant Graph Neural Networks](https://mlsanity.com/paper/2102.09844/) (2021)
- [Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds](https://mlsanity.com/paper/1802.08219/) (2018)
- [E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials](https://mlsanity.com/paper/2101.03164/) (2021)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks](https://mlsanity.com/paper/2006.10503/) (2020)
- [SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models](https://mlsanity.com/paper/2307.10635/) (2023)
- [What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks](https://mlsanity.com/paper/2305.18365/) (2023)
- [Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing](https://mlsanity.com/paper/2212.10789/) (2022)
- [MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter](https://mlsanity.com/paper/2310.12798/) (2023)
- [Fine-Tuned Language Models Generate Stable Inorganic Materials as Text](https://mlsanity.com/paper/2402.04379/) (2024)
- [UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction](https://mlsanity.com/paper/2402.11838/) (2024)
- [Iterative Refinement Graph Neural Network for Antibody Sequence-Structure Co-design](https://mlsanity.com/paper/2110.04624/) (2021)
- [Conditional Antibody Design as 3D Equivariant Graph Translation](https://mlsanity.com/paper/2208.06073/) (2022)
- [Equivariant Graph Mechanics Networks with Constraints](https://mlsanity.com/paper/2203.06442/) (2022)
- [Equivariant Graph Neural Operator for Modeling 3D Dynamics](https://mlsanity.com/paper/2401.11037/) (2024)
- [Prot2Text: Multimodal Protein's Function Generation with GNNs and Transformers](https://mlsanity.com/paper/2307.14367/) (2023)
- [SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases](https://mlsanity.com/paper/2308.13212/) (2023)
- [Integrating Chemistry Knowledge in Large Language Models via Prompt Engineering](https://mlsanity.com/paper/2404.14467/) (2024)

## Built on by (most cited first, 10 of 10)

- [ReQFlow: Rectified Quaternion Flow for Efficient and High-Quality Protein Backbone Generation](https://mlsanity.com/paper/2502.14637/) (2025)
- [Advances in Protein Representation Learning: Methods, Applications, and Future Directions](https://mlsanity.com/paper/2503.16659/) (2025)
- [Universally Invariant Learning in Equivariant GNNs](https://mlsanity.com/paper/2510.13169/) (2025)
- [IBCircuit: Towards Holistic Circuit Discovery with Information Bottleneck](https://mlsanity.com/paper/2602.22581/) (2026)
- [STORM-BORN: A Challenging Mathematical Derivations Dataset Curated via a Human-in-the-Loop Multi-Agent Framework](https://mlsanity.com/paper/2506.01531/) (2025)
- [Geometric Mixture Models for Electrolyte Conductivity Prediction](https://mlsanity.com/paper/2510.15403/) (2025)
- [EquiVLA: A General Framework for Rotationally Equivariant Vision-Language-Action Models](https://mlsanity.com/paper/2606.19784/) (2026)
- [Twin Worlds: Equivariance-Based Abstention for Evidence-Grounded Reasoning](https://mlsanity.com/paper/2608.28018/) (2026)
- [Entropy and Attention Dynamics in Small Language Models: A Trace-Level Structural Analysis on the TruthfulQA Benchmark](https://mlsanity.com/paper/2604.03589/) (2026)
- [Beyond Semantic Search: Towards Referential Anchoring in Composed Image Retrieval](https://mlsanity.com/paper/2604.05393/) (2026)

## Abstract (arXiv)

Accurately predicting 3D structures and dynamics of physical systems is crucial in scientific applications. Existing approaches that rely on geometric Graph Neural Networks (GNNs) effectively enforce E(3)-equivariance, but they often fall in leveraging extensive broader information. While direct application of Large Language Models (LLMs) can incorporate external knowledge, they lack the capability for spatial reasoning with guaranteed equivariance. In this paper, we propose EquiLLM, a novel framework for representing 3D physical systems that seamlessly integrates E(3)-equivariance with LLM capabilities. Specifically, EquiLLM comprises four key components: geometry-aware prompting, an equivariant encoder, an LLM, and an equivariant adaptor. Essentially, the LLM guided by the instructive prompt serves as a sophisticated invariant feature processor, while 3D directional information is exclusively handled by the equivariant encoder and adaptor modules. Experimental results demonstrate that EquiLLM delivers significant improvements over previous methods across molecular dynamics simulation, human motion simulation, and antibody design, highlighting its promising generalizability.