mlsanityData to 14 Oct 2026

Geometry Informed Tokenization of Molecules for Language Model Generation

arXiv 2408.10120 ↗First posted 19 Aug 2024cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

We consider molecule generation in 3D space using language models (LMs), which requires discrete tokenization of 3D molecular geometries. Although tokenization of molecular graphs exists, that for 3D geometries is largely unexplored. Here, we attempt to bridge this gap by proposing the Geo2Seq, which converts molecular geometries into SE(3)-invariant 1D discrete sequences. Geo2Seq consists of canonical labeling and invariant spherical representation steps, which together maintain geometric and atomic fidelity in a format conducive to LMs. Our experiments show that, when coupled with Geo2Seq, various LMs excel in molecular geometry generation, especially in controlled generation tasks.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 66 of its 95 references that are in the map. First posted on arXiv 19 Aug 2024.

Builds on

66 of 95 references.

References · 66

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  8. 2023A Survey of Large Language Modelscited by 2,576
  9. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  10. 2018Hierarchical Neural Story Generationcited by 1,193
  11. 2021E(n) Equivariant Graph Neural Networkscited by 965
  12. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 677
38 more
  1. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 607
  2. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  3. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
  4. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 516
  5. 2022Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphscited by 264
  6. 2019Cormorant: Covariant Molecular Neural Networkscited by 263
  7. 2022Torsional Diffusion for Molecular Conformer Generationcited by 263
  8. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 243
  9. 2023Geometric Latent Diffusion Models for 3D Molecule Generationcited by 186
  10. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 182
  11. 2022Translation between Molecules and Natural Languagecited by 177
  12. 2019Symmetry-adapted generation of 3d point sets for the targeted discovery of moleculescited by 172
  13. 2021Spherical Message Passing for 3D Graph Networkscited by 168
  14. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 166
  15. 2021E(n) Equivariant Normalizing Flowscited by 165
  16. 2021Learning Gradient Fields for Molecular Conformation Generationcited by 163
  17. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 149
  18. 2023Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systemscited by 121
  19. 2021GeoMol: Torsional Geometric Generation of Molecular 3D Conformer Ensemblescited by 107
  20. 2022Diffusion-based Molecule Generation with Informative Prior Bridgescited by 106
  21. 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associationscited by 100
  22. 2023MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generationcited by 98
  23. 2023Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspectivecited by 92
  24. 2019Molecular geometry prediction using a deep generative graph neural networkcited by 91
  25. 2021Learning Neural Generative Dynamics for Molecular Conformation Generationcited by 88
  26. 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
  27. 2023MolXPT: Wrapping Molecules with Text for Generative Pre-trainingcited by 81
  28. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  29. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
  30. 2019A Generative Model for Molecular Distance Geometrycited by 73
  31. 2022Equivariant Energy-Guided SDE for Inverse Molecular Designcited by 73
  32. 2023GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learningcited by 70
  33. 2022ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphscited by 69
  34. 2022MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Designcited by 66
  35. 2022Benchmarking Graphormer on Large-Scale Molecular Modeling Datasetscited by 64
  36. 2021An End-to-End Framework for Molecular Conformation Generation via Bilevel Programmingcited by 59
  37. 2023Language models can generate molecules, materials, and protein binding sites directly in three dimensions as XYZ, CIF, and PDB filescited by 59
  38. 2023DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphscited by 40

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 1.

10 papers.

  1. 2025Dynamic Search for Inference-Time Alignment in Diffusion Modelscited by 37
  2. 2024Learning Graph Quantized Tokenizerscited by 19
  3. 2025Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physicscited by 11
  4. 2024Bio2Token: All-atom tokenization of any biomolecular structure with Mambacited by 6
  5. 2025Tokenizing Electron Cloud in Protein-Ligand Interaction Learningcited by 3
  6. 2025InertialAR: Autoregressive 3D Molecule Generation with Inertial Framescited by 3
  7. 2025NEAT: Neighborhood-Guided, Efficient, Autoregressive Set Transformer for 3D Molecular Generationcited by 3
  8. 2026Towards Agentic Intelligence for Materials Sciencecited by 3
  9. 2024Tokenizing 3D Molecule Structure with Quantized Spherical Coordinatescited by 1
  10. 2024Exploiting Latent Linearity in LLMs Improves Explainable Molecular Representation Learningcited by 0