mlsanityData to 7 Oct 2026

g-DPO: Scalable Preference Optimization for Protein Language Models

arXiv 2510.19474 ↗First posted 22 Oct 2025cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference optimization for molecular design

Abstract (arXiv)

Direct Preference Optimization (DPO) is an effective approach for aligning protein language models with experimental design goals. However, DPO faces a scalability bottleneck: the number of possible training pairs grows quadratically with the number of labeled sequences, leading to prohibitive training times even for modestly sized datasets. We introduce g-DPO, a framework that (i) uses sequence space clustering to prune redundant pairs while preserving training signal, and (ii) amortizes likelihood computations with group-based approximations. Across three protein engineering tasks, g-DPO maintains in silico and in vitro performance that is statistically indistinguishable from standard DPO, while converging 1.7x to 5.4x times faster, with speedups that scale with dataset size and the structure of the underlying mutational landscape.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Preference optimization for molecular design, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 23 of its 37 references that are in the map. First posted on arXiv 22 Oct 2025.

Builds on

23 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025De Novo Molecular Design Enabled by Direct Preference Optimization and Curriculum Learning2 Apr
  2. 2024Preference optimization of protein language models as a multi-objective binder design paradigm7 Mar

Other references · 21

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  7. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  8. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  9. 2023Preference Ranking Optimization for Human Alignmentcited by 283
  10. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  11. 2024Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferencescited by 39
  12. 2024Curriculum Direct Preference Optimization for Diffusion and Consistency Modelscited by 36
9 more
  1. 2025Protein Large Language Models: A Comprehensive Surveycited by 18
  2. 2023DiscoBAX: Discovery of Optimal Intervention Sets in Genomic Experiment Designcited by 12
  3. 2025ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignmentcited by 9
  4. 2025Steering Generative Models with Experimental Data for Protein Fitness Optimizationcited by 7
  5. 2025Improving Protein Sequence Design through Designability Preference Optimizationcited by 6
  6. 2025K-order Ranking Preference Optimization for Large Language Modelscited by 6
  7. 2025Protein Inverse Folding From Structure Feedbackcited by 5
  8. 2025Controllable Protein Sequence Generation with LLM Preference Optimizationcited by 4
  9. 2025EnerBridge-DPO: Energy-Guided Protein Inverse Folding with Markov Bridges and Direct Preference Optimizationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Physio-DPO: Aligning Large Language Models with the Protein Energy Landscape to Eliminate Structural Hallucinations
  2. 2025AnnoDPO: Protein Functional Annotation Learning with Direct Preference Optimization
  3. 2026Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization
  4. 2026ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design
  5. 2025From Supervision to Exploration: What Does Protein Language Model Learn During Reinforcement Learning?
  6. 2026Repurposing Protein Language Models for Latent Flow-Based Fitness Optimization
  7. 2024Improving Inverse Folding for Peptide Design with Diversity-regularized Direct Preference Optimization
  8. 2024Reinforcement Learning for Sequence Design Leveraging Protein Language Models