mlsanityData to 7 Oct 2026

DrugImproverGPT: A Large Language Model for Drug Optimization with Fine-Tuning via Structured Policy Optimization

arXiv 2502.07237 ↗First posted 11 Feb 2025cs.LG cs.CL q-bio.BMcited by 4

Not on the map. Why?

Abstract (arXiv)

Finetuning a Large Language Model (LLM) is crucial for generating results towards specific objectives. This research delves into the realm of drug optimization and introduce a novel reinforcement learning algorithm to finetune a drug optimization LLM-based generative model, enhancing the original drug across target objectives, while retains the beneficial chemical properties of the original drug. This work is comprised of two primary components: (1) DrugImprover: A framework tailored for improving robustness and efficiency in drug optimization. It includes a LLM designed for drug optimization and a novel Structured Policy Optimization (SPO) algorithm, which is theoretically grounded. This algorithm offers a unique perspective for fine-tuning the LLM-based generative model by aligning the improvement of the generated molecule with the input molecule under desired objectives. (2) A dataset of 1 million compounds, each with OEDOCK docking scores on 5 human proteins associated with cancer cells and 24 binding sites from SARS-CoV-2 virus. We conduct a comprehensive evaluation of SPO and demonstrate its effectiveness in improving the original drug across target properties. Our code and dataset will be publicly available at: https://github.com/xuefeng-cs/DrugImproverGPT.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 41 of its 86 references that are in the map. First posted on arXiv 11 Feb 2025.

Builds on

41 of 86 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2020Learning to summarize from human feedbackcited by 2,562
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  12. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
29 more
  1. 2018DeepMind Control Suitecited by 935
  2. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  3. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  4. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  5. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  6. 2018Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generationcited by 363
  7. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  8. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  9. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  10. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  11. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  12. 2023Preference Ranking Optimization for Human Alignmentcited by 283
  13. 2017Molecular De Novo Design through Deep Reinforcement Learningcited by 264
  14. 2018Optimization of Molecules via Deep Reinforcement Learningcited by 199
  15. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  16. 2017Deep Reinforcement Learning for De-Novo Drug Designcited by 161
  17. 2021Offline RL Without Off-Policy Evaluationcited by 150
  18. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 116
  19. 2017Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Modelscited by 116
  20. 2019Learning from Dialogue after Deployment: Feed Yourself, Chatbot!cited by 102
  21. 2019Better Rewards Yield Better Summaries: Learning to Summarise Without Referencescited by 76
  22. 2020Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learningcited by 57
  23. 2020MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimizationcited by 48
  24. 2019Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluatorscited by 34
  25. 2023Blending Imitation and Reinforcement Learning for Robust Policy Improvementcited by 16
  26. 2021C5T5: Controllable Generation of Organic Molecules with Transformerscited by 15
  27. 2023Transformers and Large Language Models for Chemistry and Drug Discoverycited by 13
  28. 2015Giraffe: Using Deep Reinforcement Learning to Play Chesscited by 11
  29. 2023Active Policy Improvement from Multiple Black-box Oraclescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimizationcited by 12
  2. 2025Monte Carlo Tree Diffusion with Multiple Experts for Protein Designcited by 2
  3. 2025FragmentGPT: A Unified GPT Model for Fragment Growing, Linking, and Merging in Molecular Designcited by 1
  4. 2026Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processescited by 0