mlsanityData to 7 Oct 2026

Prototypical Reward Network for Data-Efficient RLHF

arXiv 2406.06606 ↗First posted 6 Jun 2024cs.CL cs.AIcited by 22

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

The reward model for Reinforcement Learning from Human Feedback (RLHF) has proven effective in fine-tuning Large Language Models (LLMs). Notably, collecting human feedback for RLHF can be resource-intensive and lead to scalability issues for LLMs and complex tasks. Our proposed framework Proto-RM leverages prototypical networks to enhance reward models under limited human feedback. By enabling stable and reliable structural learning from fewer samples, Proto-RM significantly enhances LLMs' adaptability and accuracy in interpreting human preferences. Extensive experiments on various datasets demonstrate that Proto-RM significantly improves the performance of reward models and LLMs in human feedback tasks, achieving comparable and usually better results than traditional methods, while requiring significantly less data. in data-limited scenarios. This research offers a promising direction for enhancing the efficiency of reward models and optimizing the fine-tuning of language models under restricted feedback conditions.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 35 of its 60 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

35 of 60 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Secrets of RLHF in Large Language Models Part II: Reward Modeling11 Jan
  2. 2023The History and Risks of Reinforcement Learning and Human Feedback20 Oct
  3. 2023Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of RLHF16 Sep
  4. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback27 Jul
  5. 2023Secrets of RLHF in Large Language Models Part I: PPO11 Jul
  6. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tears11 Apr
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback12 Apr
  8. 2021WebGPT: Browser-assisted question-answering with human feedback17 Dec
  9. 2019Fine-Tuning Language Models from Human Preferences18 Sep
  10. 2017Deep reinforcement learning from human preferences12 Jun

Other references · 25

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2020Learning to summarize from human feedbackcited by 2,562
  7. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  8. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  9. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 364
  10. 2017Deep Reinforcement Learning: An Overviewcited by 327
  11. 2023Preference Ranking Optimization for Human Alignmentcited by 283
  12. 2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
13 more
  1. 2020Graph Prototypical Networks for Few-shot Learning on Attributed Networkscited by 90
  2. 2023Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queriescited by 82
  3. 2015The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classificationcited by 76
  4. 2019Interpretable and Steerable Sequence Learning via Prototypescited by 72
  5. 2018Generative Adversarial User Model for Reinforcement Learning Based Recommendation Systemcited by 64
  6. 2022Understanding AdamW through Proximal Methods and Scale-Freenesscited by 46
  7. 2023From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Modelscited by 45
  8. 2017Gaussian Prototypical Networks for Few-Shot Learning on Omniglotcited by 20
  9. 2022Few-shot Named Entity Recognition with Entity-level Prototypical Network Enhanced by Dispersedly Distributed Prototypescited by 17
  10. 2023Semi-Offline Reinforcement Learning for Optimized Text Generationcited by 12
  11. 2024TFWT: Tabular Feature Weighting with Transformercited by 12
  12. 2022Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizescited by 10
  13. 2021The Online Pivot: Lessons Learned from Teaching a Text and Data Mining Course in Lockdown, Enhancing online Teaching with Pair Programming and Digital Badgescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 8; 2026: 4.

22 papers.

Sort:

By area:

  1. 2024AgentReview: Exploring Peer Review Dynamics with LLM Agentscited by 92
  2. 2025Entropy-based Exploration Conduction for Multi-step Reasoningcited by 28
  3. 2024Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generationcited by 17
  4. 2024AR Overlay: Training Image Pose Estimation on Curved Surface in a Synthetic Waycited by 15
  5. 2024Topology-aware Reinforcement Feature Space Reconstruction for Graph Datacited by 11
  6. 2024Blind Spot Navigation in Large Language Model Reasoning with Thought Space Explorercited by 7
  7. 2025Journey into Automation: Image-Derived Pavement Texture Extraction and Evaluationcited by 7
  8. 2025Diversity-oriented Data Augmentation with Large Language Modelscited by 7
  9. 2024Real-time system optimal traffic routing under uncertainties -- Can physics models boost reinforcement learning?cited by 6
  10. 2026Conformal Feedback Alignment: Quantifying Answer-Level Reliability for Robust LLM Alignmentcited by 4
  11. 2024Retrieval-Augmented Feature Generation for Domain-Specific Classificationcited by 3
  12. 2024Enhancing Exchange Rate Forecasting with Explainable Deep Learning Modelscited by 2
10 more
  1. 2024Enhancing Risk Assessment in Transformers with Loss-at-Risk Functionscited by 2
  2. 2025Broaden your SCOPE! Efficient Multi-turn Conversation Planning for LLMs with Semantic Spacecited by 2
  3. 2025Applications of Large Language Model Reasoning in Feature Generationcited by 2
  4. 2025ProtoTS: Learning Hierarchical Prototypes for Explainable Time Series Forecastingcited by 2
  5. 2026Long-form RewardBench: Evaluating Reward Models for Long-form Generationcited by 2
  6. 2024Mitigating Knowledge Conflicts in Language Model-Driven Question Answeringcited by 1
  7. 2025LEKA:LLM-Enhanced Knowledge Augmentationcited by 1
  8. 2026CroSearch-R1: Better Leveraging Cross-lingual Knowledge for Retrieval-Augmented Generationcited by 1
  9. 2025GFRIEND: Generative Few-shot Reward Inference through EfficieNt DPOcited by 0
  10. 2026BrailleBench: Investigating Multi-Criteria Braille Comprehension in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback
  2. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
  3. 2024How to Evaluate Reward Models for RLHF
  4. 2025A Unified Pairwise Framework for RLHF: Bridging Generative Reward Modeling and Policy Optimization
  5. 2026RMB: Reward Model Boosting Mitigates Reward Hacking
  6. 2024Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
  7. 2026Reinforcement Learning from Human Feedback: A Statistical Perspective
  8. 2025Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs