# Large Language Models are In-Context Molecule Learners

arXiv: https://arxiv.org/abs/2403.04197 · HTML: https://mlsanity.com/paper/2403.04197/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 7 Mar 2024.

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [Translation between Molecules and Natural Language](https://mlsanity.com/paper/2204.11817/) (2022)
- [A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language](https://mlsanity.com/paper/2209.05481/) (2022)
- [MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter](https://mlsanity.com/paper/2310.12798/) (2023)
- [BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations](https://mlsanity.com/paper/2310.07276/) (2023)
- [Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective](https://mlsanity.com/paper/2306.06615/) (2023)
- [MolXPT: Wrapping Molecules with Text for Generative Pre-training](https://mlsanity.com/paper/2305.10688/) (2023)
- [A Systematic Survey of Chemical Pre-trained Models](https://mlsanity.com/paper/2210.16484/) (2022)
- [Conflicts, Villains, Resolutions: Towards models of Narrative Media Framing](https://mlsanity.com/paper/2306.02052/) (2023)
- [SynerGPT: In-Context Learning for Personalized Drug Synergy Prediction and Drug Design](https://mlsanity.com/paper/2307.11694/) (2023)
- [HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendation](https://mlsanity.com/paper/2301.05430/) (2023)

## Built on by (most cited first, 18 of 18)

- [Speak-to-Structure: Evaluating LLMs in Open-domain Natural Language-Driven Molecule Generation](https://mlsanity.com/paper/2412.14642/) (2024)
- [A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimization](https://mlsanity.com/paper/2505.16094/) (2025)
- [MolReFlect: Towards In-Context Fine-grained Alignments between Molecules and Texts](https://mlsanity.com/paper/2411.14721/) (2024)
- [From Generalist to Specialist: A Survey of Large Language Models for Chemistry](https://mlsanity.com/paper/2412.19994/) (2024)
- [Instruction-Based Molecular Graph Generation with Unified Text-Graph Diffusion Model](https://mlsanity.com/paper/2408.09896/) (2024)
- [Atomas: Hierarchical Alignment on Molecule-Text for Unified Molecule Understanding and Generation](https://mlsanity.com/paper/2404.16880/) (2024)
- [Bridging the Gap Between Molecule and Textual Descriptions via Substructure-aware Alignment](https://mlsanity.com/paper/2510.26157/) (2025)
- [In-Context Examples Matter: Improving Emotion Recognition in Conversation with Instruction Tuning](https://mlsanity.com/paper/2508.11889/) (2025)
- [MoRA: On-the-fly Molecule-aware Low-Rank Adaptation Framework for LLM-based Multi-Modal Molecular Assistant](https://mlsanity.com/paper/2510.12245/) (2025)
- [RTMol: Rethinking Molecule-text Alignment in a Round-trip View](https://mlsanity.com/paper/2511.12135/) (2025)
- [Agents on a Tree: Pathwise Coordination for Multi-Objective Molecular Optimization](https://mlsanity.com/paper/2606.00008/) (2026)
- [Instruction Learning Paradigms: A Dual Perspective on White-box and Black-box LLMs](https://mlsanity.com/paper/2506.21573/) (2025)
- [MolErr2Fix: Benchmarking LLM Trustworthiness in Chemistry via Modular Error Detection, Localization, Explanation, and Revision](https://mlsanity.com/paper/2509.00063/) (2025)
- [From Exposure to Internalization: Dual-Stream Calibration for In-context Clinical Reasoning](https://mlsanity.com/paper/2604.06262/) (2026)
- [Agentic Molecular Recovery via Molecule-Aware Exploration](https://mlsanity.com/paper/2606.05847/) (2026)
- [Retrieval-Augmented Multimodal Learning for Enzyme-Substrate Interaction Prediction Under Low-Homology Shift](https://mlsanity.com/paper/2606.22823/) (2026)
- [MolGVR: A Chemistry-Grounded Framework for Text-to-Molecule Generation](https://mlsanity.com/paper/2607.29479/) (2026)
- [Molecular LLM Agents: From Architectural Design to Scientific Autonomy](https://mlsanity.com/paper/2608.23104/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated exceptional performance in biochemical tasks, especially the molecule caption translation task, which aims to bridge the gap between molecules and natural language texts. However, previous methods in adapting LLMs to the molecule-caption translation task required extra domain-specific pre-training stages, suffered weak alignment between molecular and textual spaces, or imposed stringent demands on the scale of LLMs. To resolve the challenges, we propose In-Context Molecule Adaptation (ICMA), as a new paradigm allowing LLMs to learn the molecule-text alignment from context examples via In-Context Molecule Tuning. Specifically, ICMA incorporates the following three stages: Hybrid Context Retrieval, Post-retrieval Re-ranking, and In-context Molecule Tuning. Initially, Hybrid Context Retrieval utilizes BM25 Caption Retrieval and Molecule Graph Retrieval to retrieve similar informative context examples. Additionally, Post-retrieval Re-ranking is composed of Sequence Reversal and Random Walk selection to further improve the quality of retrieval results. Finally, In-Context Molecule Tuning unlocks the in-context learning and reasoning capability of LLMs with the retrieved examples and adapts the parameters of LLMs for better alignment between molecules and texts. Experimental results demonstrate that ICMA can empower LLMs to achieve state-of-the-art or comparable performance without extra training corpora and intricate structures, showing that LLMs are inherently in-context molecule learners.