# Chem3DLLM: 3D Multimodal Large Language Models for Chemistry

arXiv: https://arxiv.org/abs/2508.10696 · HTML: https://mlsanity.com/paper/2508.10696/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2025.

## Built on by (most cited first, 7 of 7)

- [SciReasoner: Laying the Scientific Reasoning Ground Across Disciplines](https://mlsanity.com/paper/2509.21320/) (2025)
- [LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning](https://mlsanity.com/paper/2602.07075/) (2026)
- [SpecMol: A Spectroscopy-Grounded Foundation Model for Multi-Task Molecular Learning](https://mlsanity.com/paper/2509.21861/) (2025)
- [Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models](https://mlsanity.com/paper/2607.12771/) (2026)
- [Scaling-Aware Adapter for Structure-Grounded LLM Reasoning](https://mlsanity.com/paper/2602.02780/) (2026)
- [Language-Informed Flow Matching for Trend-Guided Structure-Based 3D Molecular Generation](https://mlsanity.com/paper/2608.31009/) (2026)
- [TMCS: Tool-Grounded Multi-Agent Reasoning for Compositional Chemical Problem Solving](https://mlsanity.com/paper/2609.35336/) (2026)

## Abstract (arXiv)

In the real world, a molecule is a 3D geometric structure. Compared to 1D SMILES sequences and 2D molecular graphs, 3D molecules represent the most informative molecular modality. Despite the rapid progress of autoregressive-based language models, they cannot handle the generation of 3D molecular conformation due to several challenges: 1) 3D molecular structures are incompatible with LLMs' discrete token space, 2) integrating heterogeneous inputs like proteins, ligands, and text remains difficult within a unified model, and 3) LLMs lack essential scientific priors, hindering the enforcement of physical and chemical constraints during generation. To tackle these issues, we present Chem3DLLM, a unified protein-conditioned multimodal large language model. Our approach designs a novel reversible text encoding for 3D molecular structures using run-length compression, achieving 3x size reduction while preserving complete structural information. This enables seamless integration of molecular geometry with protein pocket features in a single LLM architecture. We employ reinforcement learning with stability-based rewards to optimize chemical validity and incorporate a lightweight protein embedding projector for end-to-end training. Experimental results on structure-based drug design demonstrate state-of-the-art performance with a Vina score of -7.21, validating our unified multimodal approach for practical drug discovery applications.