mlsanityData to 14 Oct 2026

A Distribution Mapping Approach to Counterfactually Fair Reinforcement Learning

arXiv 2608.08743 ↗First posted 9 Aug 2026stat.ML cs.CY cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Reinforcement learning (RL) seeks to optimize sequential decisions to maximize population-level benefits over time. However, when deployed in high-stakes settings such as healthcare, RL decisions might systematically restrict some subpopulation's access to valuable services in a manner contrary to the values and goals of stakeholders. Counterfactual fairness (CF) offers a promising framework to address this problem based on causal reasoning. This paper develops a data preprocessing algorithm that, when used in tandem with policy learning, enables CF in RL. Our algorithm relies on a novel quantile distribution mapping method for sequentially estimating the counterfactual states and rewards in the data preprocessing step, subsuming common additivity assumptions used for counterfactual prediction as a special case. We theoretically prove that the per-step level of counterfactual unfairness and infinite-horizon suboptimality gap can be bounded under mild regularity conditions. We also empirically test our algorithm in numerical experiments as well as in application to a real-world interventional digital health dataset.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 32 references that are in the map. First posted on arXiv 9 Aug 2026.

Builds on

16 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Counterfactually Fair Reinforcement Learning via Sequential Data Preprocessing10 Jan

Other references · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Counterfactual Fairnesscited by 804
  3. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 287
  4. 2019Batch Policy Learning under Constraintscited by 244
  5. 2015Contextual Markov Decision Processescited by 220
  6. 2018Delayed Impact of Fair Machine Learningcited by 184
  7. 2019Causal Modeling for Fairness in Dynamical Systemscited by 47
  8. 2021Fast Rates for the Regret of Offline Reinforcement Learningcited by 26
  9. 2020Quantile regression with deep ReLU Networks: Estimators and minimax ratescited by 24
  10. 2022Counterfactual Fairness with Partially Known Causal Graphcited by 20
  11. 2021Achieving Counterfactual Fairness for Causal Banditcited by 15
  12. 2020Counterfactual fairness: removing direct effects through regularizationcited by 14
3 more
  1. 2022A Complete Characterization of Linear Estimators for Offline Policy Evaluationcited by 7
  2. 2023Evaluating Dynamic Conditional Quantile Treatment Effects with Applications in Ridesharingcited by 7
  3. 2026Double Fairness Policy Learning: Integrating Action Fairness and Outcome Fairness in Decision-makingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PyCFRL: A Python library for counterfactually fair offline reinforcement learning via sequential data preprocessing
  2. 2025Feasibility-Guided Fair Adaptive Offline Reinforcement Learning for Medicaid Care Management
  3. 2025Counterfactual Realizability
  4. 2025A Causal Lens for Learning Long-term Fair Policies
  5. 2024Out-of-Distribution Adaptation in Offline RL: Counterfactual Reasoning via Causal Normalizing Flows
  6. 2025Hybrid Adaptive Conformal Offline Reinforcement Learning for Fair Population Health Management
  7. 2025An Identifiable Cost-Aware Causal Decision-Making Framework Using Counterfactual Reasoning
  8. 2026Fairness under uncertainty in sequential decisions