Preference Optimization for Molecular Language Models
On the map: LLM alignment / RLHF / preference optimisation → Preference optimization for molecular design
Abstract (arXiv)
Molecular language modeling is an effective approach to generating novel chemical structures. However, these models do not a priori encode certain preferences a chemist may desire. We investigate the use of fine-tuning using Direct Preference Optimization to better align generated molecules with chemist preferences. Our findings suggest that this approach is simple, efficient, and highly effective.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Preference optimization for molecular design, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 4 of its 11 references that are in the map. First posted on arXiv 18 Oct 2023.
Builds on
4 of 11 references.
References · 4
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2018Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Modelscited by 275
- 2017ChemTS: An Efficient Python Library for de novo Molecular Generationcited by 32
- 2017Generating Focussed Molecule Libraries for Drug Discovery with Recurrent Neural Networkscited by 12
Built on by
Citing papers per year: 2024: 5; 2025: 1; 2026: 1.
7 papers.
- 2025Multi-domain Distribution Learning for De Novo Drug Designcited by 15
- 2024SmileyLlama: Modifying Large Language Models for Directed Chemical Space Explorationcited by 14
- 2024Aligning Transformers with Continuous Feedback via Energy Rank Alignmentcited by 7
- 2024Boltzmann-Aligned Inverse Folding Model as a Predictor of Mutational Effects on Protein-Protein Interactionscited by 5
- 2024Improving Inverse Folding for Peptide Design with Diversity-regularized Direct Preference Optimizationcited by 5
- 2024Preference optimization of protein language models as a multi-objective binder design paradigmcited by 4
- 2026LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generatorscited by 0
- 2026LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators12 Oct 2026
- 2025Multi-domain Distribution Learning for De Novo Drug Design25 Aug 2025
- 2024Improving Inverse Folding for Peptide Design with Diversity-regularized Direct Preference Optimization25 Oct 2024
- 2024Boltzmann-Aligned Inverse Folding Model as a Predictor of Mutational Effects on Protein-Protein Interactions12 Oct 2024
- 2024SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration3 Sep 2024
- 2024Aligning Transformers with Continuous Feedback via Energy Rank Alignment21 May 2024
- 2024Preference optimization of protein language models as a multi-objective binder design paradigm7 Mar 2024
Similar, not cited
Close in wording, no citation either way
- 2023Domain-Agnostic Molecular Generation with Chemical Feedback
- 2024ALMol: Aligned Language-Molecule Translation LLMs through Offline Preference Contrastive Optimisation
- 2026Scaffold-Conditioned Preference Triplets for Controllable Molecular Optimization with Large Language Models
- 2024A Review of Large Language Models and Autonomous Agents in Chemistry
- 2025MT-Mol:Multi Agent System with Tool-based Reasoning for Molecular Optimization
- 2025NovoMolGen: Rethinking Molecular Language Model Pretraining
- 2026Training Large Language Models for Small-Molecule Design with Synthetic Task Scaling
- 2025Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilization