mlsanityData to 14 Oct 2026

Fairness in Preference-based Reinforcement Learning

arXiv 2306.09995 ↗First posted 16 Jun 2023cs.LG cs.AI cs.CYcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we address the issue of fairness in preference-based reinforcement learning (PbRL) in the presence of multiple objectives. The main objective is to design control policies that can optimize multiple objectives while treating each objective fairly. Toward this objective, we design a new fairness-induced preference-based reinforcement learning or FPbRL. The main idea of FPbRL is to learn vector reward functions associated with multiple objectives via new welfare-based preferences rather than reward-based preference in PbRL, coupled with policy learning via maximizing a generalized Gini welfare function. Finally, we provide experiment studies on three different environments to show that the proposed FPbRL approach can achieve both efficiency and equity for learning effective and fair policies.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 16 Jun 2023.

Builds on

17 of 31 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2020Learning to summarize from human feedbackcited by 2,588
  4. 2018A Reductions Approach to Fair Classificationcited by 587
  5. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
  6. 2021Recursively Summarizing Books with Human Feedbackcited by 290
  7. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 173
  8. 2018A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indicescited by 97
  9. 2016Fairness in Reinforcement Learningcited by 95
  10. 2020Learning Fair Policies in Multiobjective (Deep) Reinforcement Learning with Average and Discounted Rewardscited by 60
  11. 2019Algorithms for Fairness in Sequential Decision Makingcited by 56
  12. 2019Learning Fairness in Multi-Agent Systemscited by 44
5 more
  1. 2020Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learningcited by 40
  2. 2017Multi-objective Bandits: Optimizing the Generalized Gini Indexcited by 28
  3. 2022Welfare and Fairness in Multi-objective Reinforcement Learningcited by 20
  4. 2023Achieving Fairness in Multi-Agent Markov Decision Processes Using Reinforcement Learningcited by 8
  5. 2018An Efficient Primal-Dual Algorithm for Fair Combinatorial Optimization Problemscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 4.

9 papers.

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  2. 2024The Max-Min Formulation of Multi-Objective Reinforcement Learning: From Theory to a Model-Free Algorithmcited by 15
  3. 2025Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approachcited by 5
  4. 2025FairDICE: Fairness-Driven Offline Multi-Objective Reinforcement Learningcited by 2
  5. 2026Learning Fair Pareto-Optimal Policies in Multi-Objective Reinforcement Learningcited by 2
  6. 2026Offline Constrained RLHF with Multiple Preference Oraclescited by 1
  7. 2024Fair Resource Allocation in Weakly Coupled Markov Decision Processescited by 0
  8. 2026AETDICE: Unified Framework and Offline Optimization for Nonlinear Multi-Objective RLcited by 0
  9. 2026Inference-Time Policy Alignment for Fair Reinforcement Learningcited by 0