mlsanityData to 14 Oct 2026

textttMiniMol: A Parameter-Efficient Foundation Model for Molecular Learning

arXiv 2404.14986 ↗First posted 23 Apr 2024cs.LG cs.AIcited by 20

Not on the map. Why?

Abstract (arXiv)

In biological tasks, data is rarely plentiful as it is generated from hard-to-gather measurements. Therefore, pre-training foundation models on large quantities of available data and then transfer to low-data downstream tasks is a promising direction. However, how to design effective foundation models for molecular learning remains an open question, with existing approaches typically focusing on models with large parameter capacities. In this work, we propose textttMiniMol, a foundational model for molecular learning with 10 million parameters. textttMiniMol is pre-trained on a mix of roughly 3300 sparsely defined graph- and node-level tasks of both quantum and biological nature. The pre-training dataset includes approximately 6 million molecules and 500 million labels. To demonstrate the generalizability of textttMiniMol across tasks, we evaluate it on downstream tasks from the Therapeutic Data Commons (TDC) ADMET group showing significant improvements over the prior state-of-the-art foundation model across 17 tasks. textttMiniMol will be a public and open-sourced model for future research.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 27 of its 50 references that are in the map. First posted on arXiv 23 Apr 2024.

Builds on

27 of 50 references.

References · 27

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  5. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  6. 2022Galactica: A Large Language Model for Sciencecited by 744
  7. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  8. 2020Directional Message Passing for Molecular Graphscited by 561
  9. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  10. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 329
  11. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 320
  12. 2021Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Developmentcited by 264
15 more
  1. 2023Fine-tuning Language Models for Factualitycited by 221
  2. 2021Large-Scale Chemical Language Representations Capture Molecular Structure and Propertiescited by 206
  3. 2022Graph neural networks for materials science and chemistrycited by 182
  4. 2022ChemBERTa-2: Towards Chemical Foundation Modelscited by 127
  5. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 103
  6. 2019SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discoverycited by 90
  7. 2023MolFM: A Multimodal Molecular Foundation Modelcited by 50
  8. 2020An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storagecited by 45
  9. 2017Learning Graph-Level Representation for Drug Discoverycited by 43
  10. 2023Distributed Inference and Fine-tuning of Large Language Models Over The Internetcited by 42
  11. 2023Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasetscited by 39
  12. 2023From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Predictioncited by 39
  13. 2015AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discoverycited by 38
  14. 2023Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+cited by 22
  15. 2022MolE: a molecular foundation model for drug discoverycited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 10.

20 papers.

Sort:

By area:

  1. 2025Position: Graph Learning Will Lose Relevance Due To Poor Benchmarkscited by 53
  2. 2024Fully-inductive Node Classification on Arbitrary Graphscited by 39
  3. 2025Accelerating Scientific Discovery with Autonomous Goal-evolving Agentscited by 19
  4. 2024Beyond Efficiency: Molecular Data Pruning for Enhanced Generalizationcited by 9
  5. 2025Towards Graph Foundation Models: A Transferability Perspectivecited by 7
  6. 2026Tabular foundation models for in-context prediction of molecular propertiescited by 6
  7. 2025Molecular Machine Learning in Chemical Process Designcited by 5
  8. 2024Pretraining Graph Transformers with Atom-in-a-Molecule Quantum Properties for Improved ADMET Modelingcited by 3
  9. 2024MiNT: Multi-Network Training for Transfer Learning on Temporal Graphscited by 2
  10. 2025Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?cited by 2
  11. 2025Can Molecular Foundation Models Know What They Don't Know? A Simple Remedy with Preference Optimizationcited by 2
  12. 2026CAGenMol: Condition-Aware Diffusion Language Model for Goal-Directed Molecular Generationcited by 2
8 more
  1. 2026Riemannian Geometry Speaks Louder Than Words: From Graph Foundation Model to Next-Generation Graph Intelligencecited by 1
  2. 2026Monroe: A Molecular Foundation Model for In-Context Probabilistic Inferencecited by 1
  3. 2026Amortized Molecular Optimization via Group Relative Policy Optimizationcited by 0
  4. 2026A Systematic Evaluation of Co-folding Model Representations for Small-Molecule Learningcited by 0
  5. 2026Self-Driving Datasets: From 20 Million Papers to Nuanced Biomedical Knowledge at Scalecited by 0
  6. 2026GLACIER: A Multimodal Student-Teacher Foundation Model for Molecular Property Predictioncited by 0
  7. 2026Molecular Property Prediction under Structural Shift with Tabular Foundation Modelscited by 0
  8. 2026Does Pretraining Teach Chemical Language Models Chemistry?cited by 0