mlsanityData to 7 Oct 2026

MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery

arXiv 2603.03517 ↗First posted 3 Mar 2026cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks. Simply increasing model size or introducing reasoning tokens does not yield significant performance gains. To address this gap, we introduce the MMAI Gym for Science, a one-stop shop molecular data formats and modalities as well as task-specific reasoning, training, and benchmarking recipes designed to teach foundation models the 'language of molecules' in order to solve practical drug discovery problems. We use MMAI Gym to train an efficient Liquid Foundation Model (LFM) for these applications, demonstrating that smaller, purpose-trained foundation models can outperform substantially larger general-purpose or specialist models on molecular benchmarks. Across essential drug discovery tasks - including molecular optimization, ADMET property prediction, retrosynthesis, drug-target activity prediction, and functional group reasoning - the resulting model achieves near specialist-level performance and, in the majority of settings, surpasses larger models, while remaining more efficient and broadly applicable in the domain.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 61 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

35 of 61 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  5. 2020GLU Variants Improve Transformercited by 1,724
  6. 2019Root Mean Square Layer Normalizationcited by 1,425
  7. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  8. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  9. 2022Galactica: A Large Language Model for Sciencecited by 736
  10. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  11. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  12. 2018Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Modelscited by 270
23 more
  1. 2021Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Developmentcited by 258
  2. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 238
  3. 2022Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pocketscited by 182
  4. 2022Translation between Molecules and Natural Languagecited by 173
  5. 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Datasetcited by 124
  6. 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associationscited by 100
  7. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 88
  8. 2017Retrosynthetic reaction prediction using neural sequence-to-sequence modelscited by 82
  9. 2023MolDiff: Addressing the Atom-Bond Inconsistency Problem in 3D Molecule Diffusion Generationcited by 65
  10. 2025Training a Scientific Reasoning Model for Chemistrycited by 54
  11. 2025TxGemma: Efficient and Agentic LLMs for Therapeuticscited by 40
  12. 2025Nature Language Model: Deciphering the Language of Nature for Scientific Discoverycited by 35
  13. 2023Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformerscited by 30
  14. 2023nach0: Multimodal Natural and Chemical Languages Foundation Modelcited by 26
  15. 2024L+M-24: Building a Dataset for Language + Molecules @ ACL 2024cited by 21
  16. 2024A Fine-tuning Dataset and Benchmark for Large Language Models for Protein Understandingcited by 19
  17. 2025GeLLMO: Generalizing Large Language Models for Multi-property Molecule Optimizationcited by 16
  18. 2024BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learningcited by 12
  19. 2025FGBench: A Dataset and Benchmark for Molecular Property Reasoning at Functional Group-Level in Large Language Modelscited by 8
  20. 2025MolTextNet: A Two-Million Molecule-Text Dataset for Multimodal Molecular Learningcited by 6
  21. 2023CREMP: Conformer-rotamer ensembles of macrocyclic peptides for machine learningcited by 5
  22. 2026When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMscited by 4
  23. 2024nach0-pc: Multi-task Language Model with Molecular Point Cloud Encodercited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026DrugPlayGround: Benchmarking Large Language Models and Embeddings for Drug Discovery
  2. 2025DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discovery
  3. 2025Large Language Model Agent for Modular Task Execution in Drug Discovery
  4. 2024MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction
  5. 2024SciDFM: A Large Language Model with Mixture-of-Experts for Science
  6. 2025BioMedGPT-Mol: Multi-task Learning for Molecular Understanding and Generation
  7. 2026Deciphering Scientific Reasoning Steps from Outcome Data for Molecule Optimization
  8. 2025Reasoning-Enhanced Large Language Models for Molecular Property Prediction