mlsanityData to 7 Oct 2026

Combining Parametric and Nonparametric Models for Off-Policy Evaluation

arXiv 1905.05787 ↗First posted 14 May 2019cs.LG stat.MLcited by 17

Not on the map. Why?

Abstract (arXiv)

We consider a model-based approach to perform batch off-policy evaluation in reinforcement learning. Our method takes a mixture-of-experts approach to combine parametric and non-parametric models of the environment such that the final value estimate has the least expected error. We do so by first estimating the local accuracy of each model and then using a planner to select which model to use at every time step as to minimize the return error estimate along entire trajectories. Across a variety of domains, our mixture-based approach outperforms the individual models alone as well as state-of-the-art importance sampling-based estimators.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 2; 2023: 4; 2024: 6; 2025: 2.

17 papers.

Sort:

By area:

  1. 2021Average-Reward Off-Policy Policy Evaluation with Function Approximationcited by 30
  2. 2022Model-Free and Model-Based Policy Evaluation when Causality is Uncertaincited by 12
  3. 2021Learning-to-defer for sequential medical decision-making under uncertaintycited by 10
  4. 2023When is Agnostic Reinforcement Learning Statistically Tractable?cited by 8
  5. 2023Off-policy Evaluation in Doubly Inhomogeneous Environmentscited by 7
  6. 2024An Optimal Tightness Bound for the Simulation Lemmacited by 7
  7. 2024Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experimentscited by 6
  8. 2022Generalizing Off-Policy Evaluation From a Causal Perspective For Sequential Decision-Makingcited by 4
  9. 2024OPERA: Automatic Offline Policy Evaluation with Re-weighted Aggregates of Multiple Estimatorscited by 3
  10. 2021Coarse-Grained Smoothness for RL in Metric Spacescited by 2
  11. 2023K-Nearest-Neighbor Resampling for Off-Policy Evaluation in Stochastic Controlcited by 2
  12. 2024Causal Deepsets for Off-policy Evaluation under Spatial or Spatio-temporal Interferencescited by 2
5 more
  1. 2024Decision-Point Guided Safe Policy Improvementcited by 2
  2. 2024Limit Order Book Simulation and Trade Evaluation with K-Nearest-Neighbor Resamplingcited by 1
  3. 2025Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluationcited by 1
  4. 2023Statistically Efficient Variance Reduction with Double Policy Estimation for Off-Policy Evaluation in Sequence-Modeled Reinforcement Learningcited by 0
  5. 2025When Machine Learning Meets Importance Sampling: A More Efficient Rare Event Estimation Approachcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol
  2. 2021COMBO: Conservative Offline Model-Based Policy Optimization
  3. 2020Reliable Off-policy Evaluation for Reinforcement Learning
  4. 2023On the Sample Complexity of Vanilla Model-Based Offline Reinforcement Learning with Dependent Samples
  5. 2020Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies
  6. 2023Model-based Offline Reinforcement Learning with Local Misspecification
  7. 2021Offline RL Without Off-Policy Evaluation
  8. 2020MOPO: Model-based Offline Policy Optimization