Data to 14 Oct 2026

Batch Active Preference-Based Learning of Reward Functions

arXiv 1810.04303 ↗First posted 10 Oct 2018cs.LG cs.AI cs.ROcited by 87

Not on the map. Why?

Cited by 41 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Data generation and labeling are usually an expensive part of learning for robotics. While active learning methods are commonly used to tackle the former problem, preference-based learning is a concept that attempts to solve the latter by querying users with preference questions. In this paper, we will develop a new algorithm, batch active preference-based learning, that enables efficient learning of reward functions using as few data samples as possible while still having short query generation times. We introduce several approximations to the batch active learning problem, and provide theoretical guarantees for the convergence of our algorithms. Finally, we present our experimental results for a variety of robotics tasks in simulation. Our results suggest that our batch active learning algorithm requires only a few queries that are computed in a short amount of time. We then showcase our algorithm in a study to learn human users' preferences.

Key facts. Cited by 87 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 11; 2022: 5; 2023: 20; 2024: 19; 2025: 15; 2026: 16.

87 papers.

Sort:

By area:

2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 155
2024Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewardscited by 148
2021B-Pref: Benchmarking Preference-Based Reinforcement Learningcited by 106
2022SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learningcited by 97
2020Active Preference-Based Gaussian Process Regression for Reward Learningcited by 71
2022Reward Uncertainty for Exploration in Preference-based Reinforcement Learningcited by 61
2023"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomycited by 60
2023Benchmarks and Algorithms for Offline Preference-Based Reward Learningcited by 56
2022Causal Confusion and Reward Misidentification in Preference-Based Reward Learningcited by 53
2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53

38 more

2021Learning Multimodal Rewards from Rankingscited by 41
2023LLF-Bench: Benchmark for Interactive Learning from Language Feedbackcited by 33
2025Generating Robot Constitutions & Benchmarks for Semantic Safetycited by 33
2023Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferencescited by 26
2023Reinforcement Learning from Diverse Human Preferencescited by 24
2023A density estimation perspective on learning from pairwise human preferencescited by 23
2025Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Modelscited by 23
2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 21
2021Offline Preference-Based Apprenticeship Learningcited by 17
2024Natural Language Reinforcement Learningcited by 17
2023Query-Policy Misalignment in Preference-Based Reinforcement Learningcited by 15
2025FDPP: Fine-tune Diffusion Policy with Human Preferencecited by 15
2021APReL: A Library for Active Preference-based Reward Learning Algorithmscited by 14
2023SIRL: Similarity-based Implicit Representation Learningcited by 14
2023What Matters to You? Towards Visual Representation Alignment for Robot Learningcited by 14
2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
2023Active Reward Learning from Online Preferencescited by 13
2023Active Reward Learning from Multiple Teacherscited by 13
2023Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedbackcited by 13
2023Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedbackcited by 13
2021The Reasonable Crowd: Towards evidence-based and interpretable models of driving behaviorcited by 12
2021Uncertain Decisions Facilitate Better Preference Learningcited by 10
2024Hummer: Towards Limited Competitive Preference Datasetcited by 10
2025ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignmentcited by 10
2021Preference-based Learning of Reward Function Featurescited by 9
2024Sample-Efficient Preference-based Reinforcement Learning with Dynamics Aware Rewardscited by 9
2023Embodied Active Learning of Relational State Abstractions for Bilevel Planningcited by 8
2024Batch Active Learning of Reward Functions from Human Preferencescited by 8
2024Pareto-Optimal Learning from Preferences with Hidden Contextcited by 8
2023Eliciting User Preferences for Personalized Multi-Objective Decision Making through Comparative Feedbackcited by 7
2025TREND: Tri-teaching for Robust Preference-based Reinforcement Learning with Demonstrationscited by 7
2021Learning Human Rewards by Inferring Their Latent Intelligence Levels in Multi-Agent Games: A Theory-of-Mind Approach with Application to Driving Datacited by 6
2024Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learningcited by 6
2024MoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimizationcited by 6
2024Improving Instruction Following in Language Models through Proxy-Based Uncertainty Estimationcited by 6
2025Inference-Time Personalized Alignment with a Few User Preference Queriescited by 6
2021Assisted Robust Reward Designcited by 5
2024Natural Language Reinforcement Learningcited by 5

50 most cited of 87 All 87 (JSON)