# A Foundation Chemical Language Model for Comprehensive Fragment-Based Drug Discovery

arXiv: https://arxiv.org/abs/2509.19586 · HTML: https://mlsanity.com/paper/2509.19586/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Builds on (most cited first, 5 of 5)

- [ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction](https://mlsanity.com/paper/2010.09885/) (2020)
- [Large-Scale Chemical Language Representations Capture Molecular Structure and Properties](https://mlsanity.com/paper/2106.09553/) (2021)
- [A Deep Generative Model for Fragment-Based Molecule Generation](https://mlsanity.com/paper/2002.12826/) (2020)
- [Molecule Generation with Fragment Retrieval Augmentation](https://mlsanity.com/paper/2411.12078/) (2024)
- [Fragment-based Sequential Translation for Molecular Optimization](https://mlsanity.com/paper/2111.01009/) (2021)

## Abstract (arXiv)

We introduce FragAtlas-62M, a specialized foundation model trained on the largest fragment dataset to date. Built on the complete ZINC-22 fragment subset comprising over 62 million molecules, it achieves unprecedented coverage of fragment chemical space. Our GPT-2 based model (42.7M parameters) generates 99.90% chemically valid fragments. Validation across 12 descriptors and three fingerprint methods shows generated fragments closely match the training distribution (all effect sizes < 0.4). The model retains 53.6% of known ZINC fragments while producing 22% novel structures with practical relevance. We release FragAtlas-62M with training code, preprocessed data, documentation, and model weights to accelerate adoption.