mlsanityData to 14 Oct 2026

SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration

arXiv 2409.02231 ↗First posted 3 Sep 2024physics.chem-ph cs.LGcited by 14

On the map: LLM alignment / RLHF / preference optimisation → Preference optimization for molecular design

Abstract (arXiv)

We show that large language model (LLMs) can be transformed via supervised fine-tuning (SFT) of engineered prompts into SmileyLlama for exploring the chemical space of drug molecules. We benchmark SmileyLlama against pre-trained LLMs and chemical language models (CLM) trained from scratch for generating valid and novel drug-like molecules, and use direct preference optimization (DPO) to both improve SmileyLlama's adherence to a prompt and as part of the iMiner reinforcement learning framework to predict molecules with optimized 3D conformations and high binding affinity to drug targets. By training an LLM to speak directly as a CLM, while retaining most of its natural language capabilities, we show that we can reliably generate molecules with user-specified properties rather than acting only as a chatbot with knowledge of chemistry or as a virtual assistant. While SmileyLlama is geared toward drug discovery, the SFT/DPO/LLM framework can be extended to other chemical, biological, and materials applications.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Preference optimization for molecular design, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 68 of its 140 references that are in the map. First posted on arXiv 3 Sep 2024.

Builds on

68 of 140 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Preference Optimization for Molecular Language Models18 Oct

Other references · 67

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2024The Llama 3 Herd of Modelscited by 14,715
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2017Deep reinforcement learning from human preferencescited by 3,912
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  11. 2016Pointer Sentinel Mixture Modelscited by 3,096
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
38 more
  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  2. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  7. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  8. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  9. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 765
  10. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  11. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  12. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  13. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  14. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 404
  15. 2018MolGAN: An implicit generative model for small molecular graphscited by 393
  16. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 341
  17. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  18. 2017Molecular De Novo Design through Deep Reinforcement Learningcited by 267
  19. 2021Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Developmentcited by 264
  20. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 256
  21. 2017Regularizing and Optimizing LSTM Language Modelscited by 205
  22. 2018Optimization of Molecules via Deep Reinforcement Learningcited by 200
  23. 2018Fr'echet ChemNet Distance: A metric for generative models for molecules in drug discoverycited by 194
  24. 2024Are We Done with MMLU?cited by 194
  25. 2022Translation between Molecules and Natural Languagecited by 177
  26. 2017Deep Reinforcement Learning for De-Novo Drug Designcited by 161
  27. 2022Sample Efficiency Matters: A Benchmark for Practical Molecular Optimizationcited by 155
  28. 2024LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Datasetcited by 128
  29. 2023Mechanistically analyzing the effects of fine-tuning on procedurally defined taskscited by 115
  30. 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 108
  31. 2023BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associationscited by 100
  32. 2020Accelerating Training of Transformer-Based Language Models with Progressive Layer Droppingcited by 93
  33. 2023Scaling Vision-Language Models with Sparse Mixture of Expertscited by 88
  34. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 79
  35. 20211-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speedcited by 75
  36. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  37. 2021Keeping it Simple: Language Models can learn Complex Molecular Distributionscited by 68
  38. 2024Efficient Evolutionary Search Over Chemical Space with Large Language Modelscited by 61

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 7.

14 papers.

Sort:

By area:

  1. 2025Hypothesis Generation for Materials Discovery and Design Using Goal-Driven and Constraint-Guided LLM Agentscited by 30
  2. 2024Speak-to-Structure: Evaluating LLMs in Open-domain Natural Language-Driven Molecule Generationcited by 25
  3. 2025MatterChat: A Multi-Modal LLM for Material Sciencecited by 18
  4. 2025A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimizationcited by 13
  5. 2025SynLlama: Generating Synthesizable Molecules and Their Analogs with Large Language Modelscited by 9
  6. 2025Atom-anchored LLMs speak Chemistry: A Retrosynthesis Demonstrationcited by 4
  7. 2025Mol-MoE: Training Preference-Guided Routers for Molecule Generationcited by 3
  8. 2026M^4olGen: Multi-Agent, Multi-Stage Molecular Generation under Precise Multi-Property Constraintscited by 3
  9. 2026Towards Agentic Intelligence for Materials Sciencecited by 3
  10. 2026Towards Diverse Scientific Hypothesis Search with Large Language Modelscited by 3
  11. 2026COGTRL: Training LLMs for Scientific Discovery Assistance using Cognitive Traces via Reinforcement Learningcited by 1
  12. 2026DrugPlayGround: Benchmarking Large Language Models and Embeddings for Drug Discoverycited by 0
2 more
  1. 2026EvoMD-LLM: Learning the Language of Species Evolution in Reactive Molecular Dynamicscited by 0
  2. 2026ElemeNet: Multiscale Molecular Machine Learning with Uncertainty Quantification Across the Periodic Tablecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ChatMol: A Versatile Molecule Designer Based on the Numerically Enhanced Large Language Model
  2. 2026Evaluating the Progression of Large Language Model Capabilities for Small-Molecule Drug Design
  3. 2025Test-Time Training Scaling Laws for Chemical Exploration in Drug Design
  4. 2026Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesis
  5. 2026Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling
  6. 2024Crossing New Frontiers: Knowledge-Augmented Large Language Model Prompting for Zero-Shot Text-Based De Novo Molecule Design
  7. 2025Reasoning-Enhanced Large Language Models for Molecular Property Prediction
  8. 2025Large Language Model Agent for Modular Task Execution in Drug Discovery