# Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences

arXiv: https://arxiv.org/abs/2606.16905 · HTML: https://mlsanity.com/paper/2606.16905/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 103 references that are in the map.
- First posted on arXiv 15 Jun 2026.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Scaling Laws for Transfer](https://mlsanity.com/paper/2102.01293/) (2021)
- [Structure-based Drug Design with Equivariant Diffusion Models](https://mlsanity.com/paper/2210.13695/) (2022)
- [Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets](https://mlsanity.com/paper/2205.07249/) (2022)
- [3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction](https://mlsanity.com/paper/2303.03543/) (2023)
- [Biological Sequence Design with GFlowNets](https://mlsanity.com/paper/2203.04115/) (2022)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)
- [A 3D Generative Model for Structure-Based Drug Design](https://mlsanity.com/paper/2203.10446/) (2022)
- [Conditioning by adaptive sampling for robust design](https://mlsanity.com/paper/1901.10060/) (2019)
- [Generating 3D Molecules for Target Protein Binding](https://mlsanity.com/paper/2204.09410/) (2022)
- [Conservative Objective Models for Effective Offline Model-Based Optimization](https://mlsanity.com/paper/2107.06882/) (2021)
- [DecompDiff: Diffusion Models with Decomposed Priors for Structure-Based Drug Design](https://mlsanity.com/paper/2403.07902/) (2024)
- [AdaLead: A simple and robust adaptive greedy search algorithm for sequence design](https://mlsanity.com/paper/2010.02141/) (2020)
- [The reparameterization trick for acquisition functions](https://mlsanity.com/paper/1712.00424/) (2017)
- [DiffBP: Generative Diffusion of 3D Molecules for Target Protein Binding](https://mlsanity.com/paper/2211.11214/) (2022)
- [Deciphering antibody affinity maturation with language models and weakly supervised learning](https://mlsanity.com/paper/2112.07782/) (2021)
- [Root-aligned SMILES: A Tight Representation for Chemical Reaction Prediction](https://mlsanity.com/paper/2203.11444/) (2022)
- [Structure-based drug design with geometric deep learning](https://mlsanity.com/paper/2210.11250/) (2022)
- [Nature Language Model: Deciphering the Language of Nature for Scientific Discovery](https://mlsanity.com/paper/2502.07527/) (2025)
- [Functional-Group-Based Diffusion for Pocket-Specific Molecule Generation and Elaboration](https://mlsanity.com/paper/2306.13769/) (2023)
- [MOFDiff: Coarse-grained Diffusion for Metal-Organic Framework Design](https://mlsanity.com/paper/2310.10732/) (2023)
- [GENERator: A Long-Context Generative Genomic Foundation Model](https://mlsanity.com/paper/2502.07272/) (2025)
- [Improving Protein Optimization with Smoothed Fitness Landscapes](https://mlsanity.com/paper/2307.00494/) (2023)
- [CBGBench: Fill in the Blank of Protein-Molecule Complex Binding Graph](https://mlsanity.com/paper/2406.10840/) (2024)
- [Inverse folding for antibody sequence design using deep learning](https://mlsanity.com/paper/2310.19513/) (2023)
- [PubChemQC B3LYP/6-31G*//PM6 dataset: the Electronic Structures of 86 Million Molecules using B3LYP/6-31G* calculations](https://mlsanity.com/paper/2305.18454/) (2023)
- [Flexible MOF Generation with Torsion-Aware Flow Matching](https://mlsanity.com/paper/2505.17914/) (2025)

## Built on by (most cited first, 3 of 3)

- [BioMatrix: Towards a Comprehensive Biological Foundation Model Spanning the Modality Matrix of Sequences, Structures, and Language](https://mlsanity.com/paper/2606.22138/) (2026)
- [TheBioCollection: Unified Pre-Training Scale LLM Corpus for Biology](https://mlsanity.com/paper/2607.08803/) (2026)
- [S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation](https://mlsanity.com/paper/2607.15686/) (2026)

## Abstract (arXiv)

In this report, we present LOGOS (Language Of Generative Objects in Science), a scientific generative language model that unifies heterogeneous tasks across the natural sciences within a single autoregressive framework based on a shared scientific grammar. It encodes diverse scientific objects and their spatial interactions as token sequences over a common vocabulary. By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner, without relying on explicit coordinates or geometric neural networks. This unified representation enables a wide range of downstream tasks to be formulated consistently as next-token prediction in the same grammar space, creating strong alignment between continued multi-domain pre-training and downstream objectives. Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines, providing preliminary evidence for the feasibility of "one model fits all" in the natural sciences. We train LOGOS models at different scales (1B, 3B, and 8B parameters) and find a consistent positive correlation between model size and performance. This suggests that the future of AI for Science (AI4S) may not lie in building an independent technical stack that is separated from large language models (LLMs). Instead, it may depend on deeply aligning scientific foundation models with LLMs through shared architectures, shared training paradigms, and shared inference infrastructure, so that LLMs can truly become a new entry point for AI4S. We release the model weights and associated resources to facilitate further research.