AnnoDPO: Protein Functional Annotation Learning with Direct Preference Optimization
On the map: LLM alignment / RLHF / preference optimisation → Preference optimization for molecular design
Abstract (arXiv)
Deciphering protein function remains a fundamental challenge in protein representation learning. The task presents significant difficulties for protein language models (PLMs) due to the sheer volume of functional annotation categories and the highly imbalanced distribution of annotated instances across biological ontologies. Inspired by the remarkable success of reinforcement learning from human feedback (RLHF) in large language model (LLM) alignment, we propose AnnoDPO, a novel multi-modal framework for protein function prediction that leverages Direct Preference Optimization (DPO) to enhance annotation learning. Our methodology addresses the dual challenges of annotation scarcity and category imbalance through preference-aligned training objectives, establishing a new paradigm for biological knowledge integration in protein representation learning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Preference optimization for molecular design, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 20 of its 45 references that are in the map. First posted on arXiv 8 Jun 2025.
Builds on
20 of 45 references.
References · 20
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2020Learning to summarize from human feedbackcited by 2,562
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
- 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
- 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
- 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
- 2023Preference Ranking Optimization for Human Alignmentcited by 283
- 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 267
8 more
- 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
- 2020ProGen: Language Modeling for Protein Generationcited by 127
- 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 101
- 2023ProtST: Multi-Modality Learning of Protein Sequences and Biomedical Textscited by 78
- 2022OntoProtein: Protein Pretraining With Gene Ontology Embeddingcited by 70
- 2023Policy Optimization in RLHF: The Impact of Out-of-preference Datacited by 35
- 2017DeepGO: Predicting protein functions from sequence and interactions using a deep ontology-aware classifiercited by 20
- 2025Controllable Protein Sequence Generation with LLM Preference Optimizationcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025g-DPO: Scalable Preference Optimization for Protein Language Models
- 2026Interleaved Tool-Call Reasoning for Protein Function Understanding
- 2023InstructProtein: Aligning Human and Protein Language via Knowledge Instruction
- 2025Understanding protein function with a multimodal retrieval-augmented foundation model
- 2026Protein Language Models Diverge from Natural Language: Comparative Analysis and Improved Inference
- 2026ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design
- 2026AMix-2: Establishing Protein as a Native Modality in Large Language Models
- 2026Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization