# ChemFM as a Scaling Law Guided Foundation Model Pre-trained on Informative Chemicals

arXiv: https://arxiv.org/abs/2410.21422 · HTML: https://mlsanity.com/paper/2410.21422/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2024.

## Built on by (most cited first, 10 of 10)

- [Chem42: a Family of chemical Language Models for Target-aware Ligand Generation](https://mlsanity.com/paper/2503.16563/) (2025)
- [MolPILE -- large-scale, diverse dataset for molecular representation learning](https://mlsanity.com/paper/2509.18353/) (2025)
- [Unveiling Scaling Behaviors in Molecular Language Models: Effects of Model Size, Data, and Representation](https://mlsanity.com/paper/2601.22757/) (2026)
- [How Well Do Large-Scale Chemical Language Models Transfer to Downstream Tasks?](https://mlsanity.com/paper/2602.11618/) (2026)
- [Cross-Modality Controlled Molecule Generation with Diffusion Language Model](https://mlsanity.com/paper/2508.14748/) (2025)
- [Multi-scale Graph Autoregressive Modeling: Molecular Property Prediction via Next Token Prediction](https://mlsanity.com/paper/2601.02530/) (2026)
- [MolLangData: A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Method](https://mlsanity.com/paper/2602.02320/) (2026)
- [FORGE: Fragment-Oriented Ranking and Generation for Context-Aware Molecular Optimization](https://mlsanity.com/paper/2605.10230/) (2026)
- [SC3: The Multi-Solvent Solubility Challenge and Benchmark](https://mlsanity.com/paper/2606.07656/) (2026)
- [GLACIER: A Multimodal Student-Teacher Foundation Model for Molecular Property Prediction](https://mlsanity.com/paper/2606.11382/) (2026)

## Abstract (arXiv)

Traditional AI methods often rely on task-specific model designs and training, which constrain both the scalability of model size and generalization across different tasks. Here, we introduce ChemFM, a large foundation model specifically developed for chemicals. By conducting a series of scaling experiments, we identify UniChem as the informative molecular database for pre-training the foundation model. ChemFM comprises 3 billion parameters and is pre-trained on 178 million molecules using self-supervised causal language modeling to extract generalizable molecular representations. This model can be adapted to diverse downstream chemical applications using either full-parameter or parameter-efficient fine-tuning methods. ChemFM consistently outperforms state-of-the-art task-specific AI models across all tested tasks. Notably, it achieves up to 67.48% performance improvement across 34 property prediction benchmarks, up to 33.80% reduction in mean average deviation between conditioned and actual properties of generated molecules in conditional molecular generation tasks, and up to 3.7% top-1 accuracy improvement across 4 reaction prediction datasets. Moreover, ChemFM demonstrates its superior performance in predicting antibiotic activity and cytotoxicity, highlighting its potential to advance the discovery of novel antibiotics. Furthermore, we demonstrate that, as a foundation model, ChemFM exhibits strong data efficiency, requiring significantly fewer labeled training samples to achieve state-of-the-art performance. We anticipate that ChemFM will significantly advance chemistry research by providing a foundation model capable of effectively generalizing across a broad range of tasks with minimal additional training.