# SMILES-Mamba: Chemical Mamba Foundation Models for Drug ADMET Prediction

arXiv: https://arxiv.org/abs/2408.05696 · HTML: https://mlsanity.com/paper/2408.05696/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 12 of its 51 references that are in the map.
- First posted on arXiv 11 Aug 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [An Empirical Study of Mamba-based Language Models](https://mlsanity.com/paper/2406.07887/) (2024)
- [MambaOut: Do We Really Need Mamba for Vision?](https://mlsanity.com/paper/2405.07992/) (2024)
- [HINT: Hierarchical Interaction Network for Trial Outcome Prediction Leveraging Web Data](https://mlsanity.com/paper/2102.04252/) (2021)
- [TWIN-GPT: Digital Twins for Clinical Trials via Large Language Model](https://mlsanity.com/paper/2404.01273/) (2024)
- [Uncertainty Quantification on Clinical Trial Outcome Prediction](https://mlsanity.com/paper/2401.03482/) (2024)
- [TrialDura: Hierarchical Attention Transformer for Interpretable Clinical Trial Duration Prediction](https://mlsanity.com/paper/2404.13235/) (2024)
- [MambaCapsule: Towards Transparent Cardiac Disease Diagnosis with Electrocardiography Using Mamba Capsule Network](https://mlsanity.com/paper/2407.20893/) (2024)
- [TrialEnroll: Predicting Clinical Trial Enrollment Success with Deep & Cross Network and Large Language Models](https://mlsanity.com/paper/2407.13115/) (2024)

## Built on by (most cited first, 13 of 13)

- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills](https://mlsanity.com/paper/2512.16301/) (2025)
- [Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs](https://mlsanity.com/paper/2506.12182/) (2025)
- [Protein-Mamba: Biological Mamba Models for Protein Function Prediction](https://mlsanity.com/paper/2409.14617/) (2024)
- [Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models](https://mlsanity.com/paper/2503.11224/) (2025)
- [Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validation](https://mlsanity.com/paper/2603.12430/) (2026)
- [DeepProtein: Deep Learning Library and Benchmark for Protein Sequence Learning](https://mlsanity.com/paper/2410.02023/) (2024)
- [Language Models for Controllable DNA Sequence Design](https://mlsanity.com/paper/2507.19523/) (2025)
- [Accelerated Markov Chain Monte Carlo Using Adaptive Weighting Scheme](https://mlsanity.com/paper/2408.12888/) (2024)
- [3D Interaction Geometric Pre-training for Molecular Relational Learning](https://mlsanity.com/paper/2412.02957/) (2024)
- [Bladder Cancer Diagnosis with Deep Learning: A Multi-Task Framework and Online Platform](https://mlsanity.com/paper/2508.15379/) (2025)
- [Kinetic-Mamba: Mamba-Assisted Predictions of Stiff Chemical Kinetics](https://mlsanity.com/paper/2512.14471/) (2025)
- [Chemical Language Models for Natural Products: A State-Space Model Approach](https://mlsanity.com/paper/2602.13958/) (2026)

## Abstract (arXiv)

In drug discovery, predicting the absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties of small-molecule drugs is critical for ensuring safety and efficacy. However, the process of accurately predicting these properties is often resource-intensive and requires extensive experimental data. To address this challenge, we propose SMILES-Mamba, a two-stage model that leverages both unlabeled and labeled data through a combination of self-supervised pretraining and fine-tuning strategies. The model first pre-trains on a large corpus of unlabeled SMILES strings to capture the underlying chemical structure and relationships, before being fine-tuned on smaller, labeled datasets specific to ADMET tasks. Our results demonstrate that SMILES-Mamba exhibits competitive performance across 22 ADMET datasets, achieving the highest score in 14 tasks, highlighting the potential of self-supervised learning in improving molecular property prediction. This approach not only enhances prediction accuracy but also reduces the dependence on large, labeled datasets, offering a promising direction for future research in drug discovery.