mlsanityData to 7 Oct 2026

Physio-DPO: Aligning Large Language Models with the Protein Energy Landscape to Eliminate Structural Hallucinations

arXiv 2601.00647 ↗First posted 2 Jan 2026cs.CL cs.CE q-bio.QMcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference optimization for molecular design

Abstract (arXiv)

Large Protein Language Models have shown strong potential for generative protein design, yet they frequently produce structural hallucinations, generating sequences with high linguistic likelihood that fold into thermodynamically unstable conformations. Existing alignment approaches such as Direct Preference Optimization are limited in this setting, as they model preferences as binary labels and ignore the continuous structure of the physical energy landscape. We propose Physio-DPO, a physics informed alignment framework that grounds protein language models in thermodynamic stability. Physio-DPO introduces a magnitude aware objective that scales optimization updates according to the energy gap between native structures and physics perturbed hard negatives. Experiments show that Physio-DPO consistently outperforms strong baselines including SFT, PPO, and standard DPO, reducing self consistency RMSD to 1.28 AA and increasing foldability to 92.8%. Qualitative analysis further demonstrates that Physio-DPO effectively mitigates structural hallucinations by recovering biophysical interactions such as hydrophobic core packing and hydrogen bond networks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Preference optimization for molecular design, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 9 of its 27 references that are in the map. First posted on arXiv 2 Jan 2026.

Builds on

9 of 27 references.

References · 9

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  5. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  6. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  7. 2022Tranception: protein fitness prediction with autoregressive transformers and inference-time retrievalcited by 78
  8. 2024ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocationcited by 34
  9. 2024Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedbackcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025g-DPO: Scalable Preference Optimization for Protein Language Models
  2. 2025EnerBridge-DPO: Energy-Guided Protein Inverse Folding with Markov Bridges and Direct Preference Optimization
  3. 2025Aligning Protein Conformation Ensemble Generation with Physical Feedback
  4. 2026ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design
  5. 2025Protein Inverse Folding From Structure Feedback
  6. 2024Aligning Transformers with Continuous Feedback via Energy Rank Alignment
  7. 2026Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization
  8. 2026Property-driven Protein Inverse Folding With Multi-Objective Preference Alignment