mlsanityData to 14 Oct 2026

[Re] FairDICE: A Fair Tradeoff in Multi-objective Offline RL

arXiv 2603.03454 ↗First posted 3 Mar 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Offline Reinforcement Learning (RL) is an emerging field of RL in which policies are learned solely from demonstrations. Within offline RL, some environments involve balancing multiple objectives, but existing multi-objective offline RL algorithms do not provide an efficient way to find a fair compromise. FairDICE (see arXiv:2506.08062v2) seeks to fill this gap by adapting OptiDICE (an offline RL algorithm) to automatically learn weights for multiple objectives to e.g. incentivise fairness among objectives. As this would be a valuable contribution, this replication study examines the replicability of claims made regarding FairDICE. We find that many theoretical claims hold, but an error in the code reduces FairDICE to standard behaviour cloning in continuous environments, and many important hyperparameters were originally underspecified. After rectifying this, we show in experiments extending the original paper that FairDICE can scale to complex environments and high-dimensional rewards, though it can be reliant on (online) hyperparameter tuning. We conclude that FairDICE is a theoretically interesting method, but the experimental justification requires significant revision.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 32 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

18 of 32 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update1 Feb
  2. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  3. 2021OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation21 Jun
  4. 2020Hyperparameter Selection for Offline Reinforcement Learning17 Jul
  5. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 12

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  2. 2017Improved Training of Wasserstein GANscited by 2,235
  3. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 763
  4. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  5. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  6. 2017A Rewriting System for Convex Optimization Problemscited by 179
  7. 2018Dynamic Weights in Multi-Objective Deep Reinforcement Learningcited by 100
  8. 2020Reinforcement Learning via Fenchel-Rockafellar Dualitycited by 100
  9. 2023Scaling Pareto-Efficient Decision Making Via Offline Multi-Objective RLcited by 36
  10. 2022Welfare and Fairness in Multi-objective Reinforcement Learningcited by 20
  11. 2024The Max-Min Formulation of Multi-Objective Reinforcement Learning: From Theory to a Model-Free Algorithmcited by 15
  12. 2025FairDICE: Fairness-Driven Offline Multi-Objective Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PyCFRL: A Python library for counterfactually fair offline reinforcement learning via sequential data preprocessing
  2. 2024Fairness in Reinforcement Learning: A Survey
  3. 2025A Clean Slate for Offline Reinforcement Learning
  4. 2024Policy-regularized Offline Multi-objective Reinforcement Learning
  5. 2021A Minimalist Approach to Offline Reinforcement Learning
  6. 2023Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets
  7. 2024An Offline Adaptation Framework for Constrained Multi-Objective Reinforcement Learning
  8. 2020Offline Reinforcement Learning Hands-On