# Reward modeling and RLHF Area: [LLM alignment / RLHF / preference optimisation](https://mlsanity.com/alignment/index.md). HTML: https://mlsanity.com/alignment/reward-modeling-and-rlhf/ ## Key facts - Reward modeling and RLHF: 1,065 papers on arXiv since 2017 (data through 14 Oct 2026). - 505 of its 1,065 mapped papers (47%) cite “Deep reinforcement learning from human preferences” (2017, arXiv 1706.03741). - 437 of its 1,065 mapped papers (41%) cite “Direct Preference Optimization: Your Language Model is Secretly a Reward Model” (2023, arXiv 2305.18290). - 31 new papers 17 Sep–14 Oct. ## Most cited here - [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017), cited by 505 papers here - [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023), cited by 437 papers here - [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022), cited by 335 papers here - [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019), cited by 258 papers here - [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022), cited by 226 papers here - [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023), cited by 136 papers here - [RewardBench: Evaluating Reward Models for Language Modeling](https://mlsanity.com/paper/2403.13787/) (2024), cited by 136 papers here - [PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training](https://mlsanity.com/paper/2106.05091/) (2021), cited by 117 papers here - [Reward learning from human preferences and demonstrations in Atari](https://mlsanity.com/paper/1811.06521/) (2018), cited by 115 papers here - [Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisons](https://mlsanity.com/paper/2301.11270/) (2023), cited by 109 papers here ## New (17 Sep–14 Oct) - [Wrong Votes, Right Gradients: Why Majority-Vote Rewards Track Verifiable RL](https://mlsanity.com/paper/2610.20898/) (2026-10-14) - [RM-OPD: On-Policy Distillation for Generative Reward Models](https://mlsanity.com/paper/2610.20174/) (2026-10-14) - [Efficient Resource-Shiftable RL Post-training with Generative Rewards](https://mlsanity.com/paper/2610.16866/) (2026-10-12) - [Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback](https://mlsanity.com/paper/2610.15141/) (2026-10-10) - [Disentangling Task-Relevant Latent Dynamics for Reward Modeling](https://mlsanity.com/paper/2610.14159/) (2026-10-09) - [Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning](https://mlsanity.com/paper/2610.13504/) (2026-10-09) - [EmoGavel: Grounding-Aware Reinforcement Learning with Verifiable Rewards for Multimodal Emotion Preference Learning](https://mlsanity.com/paper/2610.13353/) (2026-10-09) - [A Scoping Review and Experimental Study on Reinforcement Learning from Human Feedback for Human-Robot Collaboration](https://mlsanity.com/paper/2610.09891/) (2026-10-07) - [Learning a Ranking from Human Feedback in Log-Concave Random Utility Models](https://mlsanity.com/paper/2610.07973/) (2026-10-06) - [Groupwise Distortion Guarantees for Preference-Based Alignment](https://mlsanity.com/paper/2610.05450/) (2026-10-04) - [OuroReward: Sequential Reward Scheduling for Reinforcement Learning in Text-to-3D Generation](https://mlsanity.com/paper/2610.03423/) (2026-10-02) - [Bandits via Additive Quantized Representations](https://mlsanity.com/paper/2610.02440/) (2026-10-01) - [Robust Nash Alignment under Preference Uncertainty](https://mlsanity.com/paper/2610.00715/) (2026-09-30) - [Robust Risk-Sensitive Reinforcement Learning from Corrupted Human Feedback](https://mlsanity.com/paper/2609.38938/) (2026-09-30) - [Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving](https://mlsanity.com/paper/2609.38862/) (2026-09-30) - [ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Context](https://mlsanity.com/paper/2609.36684/) (2026-09-29) - [RankBuffer: Efficient Ranking-Based Rewards for Open-Ended Generation](https://mlsanity.com/paper/2609.36652/) (2026-09-29) - [Using Context Is Not Enough: Test-Time Training for Personalized Reward Modeling](https://mlsanity.com/paper/2609.35109/) (2026-09-28) - [Quality Determines Direction, Length Shapes Magnitude: Length Control for Open-Ended Reinforcement Learning](https://mlsanity.com/paper/2609.34718/) (2026-09-28) - [ARS: Agentic Reward System for Robot Learning](https://mlsanity.com/paper/2609.34484/) (2026-09-28) - [RewardExplainer: Learning Reward Model Explanations from Counterfactual Preference Feedback](https://mlsanity.com/paper/2609.33989/) (2026-09-27) - [Achieve What You Imagined: Learning to Align Actions with Visual Plans](https://mlsanity.com/paper/2609.33832/) (2026-09-27) - [Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policies](https://mlsanity.com/paper/2609.33653/) (2026-09-27) - [Direct Hidden-State Alignment: Mapping and Controlling Preference Expression in LLMs](https://mlsanity.com/paper/2609.33298/) (2026-09-27) - [RMB: Reward Model Boosting Mitigates Reward Hacking](https://mlsanity.com/paper/2609.33221/) (2026-09-27) - [Axiom Satisfiability of Linear Rewards in Alignment](https://mlsanity.com/paper/2610.06892/) (2026-09-26) - [Saturation-Insensitive Dueling Bandits with General Function Approximation](https://mlsanity.com/paper/2609.33011/) (2026-09-26) - [Improving LLM Collaboration via Multi-Agent Preference Learning](https://mlsanity.com/paper/2609.32827/) (2026-09-26) - [BiasReducer: Adaptive Bias Mitigation for Reward Models](https://mlsanity.com/paper/2609.32720/) (2026-09-26) - [PHIRL: Aligning Learned Rewards with Task Progress for Inverse Reinforcement Learning](https://mlsanity.com/paper/2609.31855/) (2026-09-25)