mlsanityData to 14 Oct 2026

Language models in molecular discovery

arXiv 2309.16235 ↗First posted 28 Sep 2023physics.chem-ph cs.AI cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

The success of language models, especially transformer-based architectures, has trickled into other domains giving rise to "scientific language models" that operate on small molecules, proteins or polymers. In chemistry, language models contribute to accelerating the molecule discovery cycle as evidenced by promising recent findings in early-stage drug discovery. Here, we review the role of language models in molecular discovery, underlining their strength in de novo drug design, property prediction and reaction chemistry. We highlight valuable open-source software assets thus lowering the entry barrier to the field of scientific language modeling. Last, we sketch a vision for future molecular design that combines a chatbot interface with access to computational chemistry tools. Our contribution serves as a valuable resource for researchers, chemists, and AI enthusiasts interested in understanding how language models can and will be used to accelerate chemical discovery.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 37 of its 107 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

37 of 107 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  11. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  12. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
25 more
  1. 2022Galactica: A Large Language Model for Sciencecited by 744
  2. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 337
  3. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  4. 2018Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Modelscited by 275
  5. 2021Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Developmentcited by 264
  6. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 229
  7. 2021Large-Scale Chemical Language Representations Capture Molecular Structure and Propertiescited by 206
  8. 2019Gradio: Hassle-Free Sharing and Testing of ML Models in the Wildcited by 178
  9. 2022Translation between Molecules and Natural Languagecited by 177
  10. 2020ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 162
  11. 2017Deep Reinforcement Learning for De-Novo Drug Designcited by 161
  12. 2021Towards artificial general intelligence via a multimodal foundation modelcited by 112
  13. 2020Molecular representation learning with language models and domain-relevant auxiliary taskscited by 94
  14. 2023Unifying Molecular and Textual Representations via Multi-task Language Modellingcited by 93
  15. 2015Inferring Algorithmic Patterns with Stack-Augmented Recurrent Netscited by 76
  16. 2018Molecular generative model based on conditional variational autoencoder for de novo molecular designcited by 73
  17. 2021Learning to Extend Molecular Scaffolds with Structural Motifscited by 71
  18. 2020Mapping the Space of Chemical Reactions Using Attention-Based Neural Networkscited by 70
  19. 2021Keeping it Simple: Language Models can learn Complex Molecular Distributionscited by 68
  20. 2017SMILES Enumeration as Data Augmentation for Neural Network Modeling of Moleculescited by 59
  21. 2017"Found in Translation": Predicting Outcomes of Complex Organic Chemistry Reactions using Neural Sequence-to-Sequence Modelscited by 55
  22. 2022Regression Transformer: Concurrent sequence regression and generation for molecular language modelingcited by 50
  23. 2022TorchDrug: A Powerful and Flexible Machine Learning Platform for Drug Discoverycited by 45
  24. 2022Accelerating Material Design with the Generative Toolkit for Scientific Discoverycited by 16
  25. 2019GEN: Highly Efficient SMILES Explorer Using Autodidactic Generative Examination Networkscited by 4

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 2.

6 papers.

  1. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 99
  2. 2025SciToolAgent: A Knowledge Graph-Driven Scientific Agent for Multi-Tool Integrationcited by 43
  3. 2024A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applicationscited by 28
  4. 2025A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimizationcited by 13
  5. 2024Geometry Informed Tokenization of Molecules for Language Model Generationcited by 10
  6. 2024MolFusion: Multimodal Fusion Learning for Molecular Representations via Multi-granularity Viewscited by 0