mlsanityData to 14 Oct 2026

Representation Balancing MDPs for Off-Policy Policy Evaluation

arXiv 1805.09044 ↗First posted 23 May 2018cs.LG cs.AI stat.MLcited by 20

Not on the map. Why?

Abstract (arXiv)

We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value accurately. We draw inspiration from recent work in causal reasoning, and propose a new finite sample generalization error bound for value estimates from MDP models. Using this upper bound as an objective, we develop a learning algorithm of an MDP model with a balanced representation, and show that our approach can yield substantially lower MSE in common synthetic benchmarks and a HIV treatment simulation domain.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 4; 2023: 4; 2024: 4.

20 papers.

Sort:

By area:

  1. 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 110
  2. 2021Average-Reward Off-Policy Policy Evaluation with Function Approximationcited by 31
  3. 2022Data-Driven Offline Decision-Making via Invariant Representation Learningcited by 31
  4. 2021Finite-Sample Analysis of Off-Policy Natural Actor-Critic Algorithmcited by 26
  5. 2021Finite-Sample Analysis of Off-Policy Natural Actor-Critic with Linear Function Approximationcited by 22
  6. 2022A Unified Framework for Alternating Offline Model Training and Policy Learningcited by 14
  7. 2021Non-asymptotic Confidence Intervals of Off-policy Evaluation: Primal and Dual Boundscited by 11
  8. 2022Scaling Marginalized Importance Sampling to High-Dimensional State-Spaces via State Abstractioncited by 7
  9. 2023Off-policy Evaluation in Doubly Inhomogeneous Environmentscited by 7
  10. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 6
  11. 2021Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluationcited by 5
  12. 2023Model-based Offline Reinforcement Learning with Local Misspecificationcited by 5
8 more
  1. 2023Leveraging Factored Action Spaces for Off-Policy Evaluationcited by 3
  2. 2024OPERA: Automatic Offline Policy Evaluation with Re-weighted Aggregates of Multiple Estimatorscited by 3
  3. 2024Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluationcited by 3
  4. 2021Model Selection for Production System via Automated Online Experimentscited by 2
  5. 2021Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learningcited by 2
  6. 2023K-Nearest-Neighbor Resampling for Off-Policy Evaluation in Stochastic Controlcited by 2
  7. 2024Concept-driven Off Policy Evaluationcited by 1
  8. 2024Short-Long Policy Evaluation with Novel Actionscited by 0