mlsanityData to 7 Oct 2026

Off-Policy Evaluation via the Regularized Lagrangian

arXiv 2007.03438 ↗First posted 7 Jul 2020cs.LG math.OC stat.MLcited by 86

Not on the map. Why?

Cited by 26 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The recently proposed distribution correction estimation (DICE) family of estimators has advanced the state of the art in off-policy evaluation from behavior-agnostic data. While these estimators all perform some form of stationary distribution correction, they arise from different derivations and objective functions. In this paper, we unify these estimators as regularized Lagrangians of the same linear program. The unification allows us to expand the space of DICE estimators to new alternatives that demonstrate improved performance. More importantly, by analyzing the expanded space of estimators both mathematically and empirically we find that dual solutions offer greater flexibility in navigating the tradeoff between optimization stability and estimation bias, and generally provide superior estimates in practice.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 27; 2022: 20; 2023: 18; 2024: 9; 2025: 9; 2026: 1.

86 papers.

Sort:

By area:

  1. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  2. 2021OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimationcited by 121
  3. 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 109
  4. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 92
  5. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  6. 2021Reward is enough for convex MDPscited by 82
  7. 2022COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimationcited by 69
  8. 2025WorldGym: World Model as An Environment for Policy Evaluationcited by 61
  9. 2021Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiencycited by 54
  10. 2021Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approachcited by 43
  11. 2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learningcited by 40
  12. 2021Instabilities of Offline RL with Pre-Trained Neural Representationcited by 38
38 more
  1. 2021Nearly Horizon-Free Offline Reinforcement Learningcited by 36
  2. 2021A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processescited by 34
  3. 2022Offline Reinforcement Learning Under Value and Density-Ratio Realizability: The Power of Gapscited by 34
  4. 2021Autoregressive Dynamics Models for Offline Policy Evaluation and Optimizationcited by 32
  5. 2021Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimizationcited by 31
  6. 2022Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangiancited by 29
  7. 2021Active Offline Policy Selectioncited by 26
  8. 2020Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficientcited by 24
  9. 2022Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimalitycited by 24
  10. 2023Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matchingcited by 24
  11. 2021On Instrumental Variable Regression for Deep Offline Policy Evaluationcited by 22
  12. 2022LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimationcited by 22
  13. 2022Markovian Interference in Experimentscited by 22
  14. 2024ODICE: Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Updatecited by 21
  15. 2021Doubly Robust Off-Policy Actor-Critic: Convergence and Optimalitycited by 20
  16. 2021A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representationcited by 19
  17. 2023Distributional Offline Policy Evaluation with Predictive Error Guaranteescited by 19
  18. 2022Learning Bellman Complete Representations for Offline Policy Evaluationcited by 17
  19. 2022When is Realizability Sufficient for Off-Policy Reinforcement Learning?cited by 17
  20. 2021Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approachcited by 16
  21. 2023High-probability sample complexities for policy evaluation with linear function approximationcited by 16
  22. 2022A Unified Framework for Alternating Offline Model Training and Policy Learningcited by 14
  23. 2023Hallucinated Adversarial Control for Conservative Offline Policy Evaluationcited by 13
  24. 2023Towards Assessing and Benchmarking Risk-Return Tradeoff of Off-Policy Evaluationcited by 13
  25. 2024Offline Multi-task Transfer RL with Representational Penalizationcited by 13
  26. 2022Inference on Strongly Identified Functionals of Weakly Identified Functionscited by 12
  27. 2023Efficient Policy Evaluation with Offline Data Informed Behavior Policy Designcited by 12
  28. 2021Non-asymptotic Confidence Intervals of Off-policy Evaluation: Primal and Dual Boundscited by 11
  29. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximationcited by 11
  30. 2021Infinite-Horizon Offline Reinforcement Learning with Linear Function Approximation: Curse of Dimensionality and Algorithmcited by 10
  31. 2022Bellman Residual Orthogonalization for Offline Reinforcement Learningcited by 10
  32. 2024Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximationcited by 10
  33. 2021False Correlation Reduction for Offline Reinforcement Learningcited by 9
  34. 2023Offline Minimax Soft-Q-learning Under Realizability and Partial Coveragecited by 9
  35. 2023Conservative State Value Estimation for Offline Reinforcement Learningcited by 9
  36. 2022Beyond the Return: Off-policy Function Estimation under User-specified Error-measuring Distributionscited by 8
  37. 2023Offline Learning of Closed-Loop Deep Brain Stimulation Controllers for Parkinson Disease Treatmentcited by 8
  38. 2024Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Datacited by 8

50 most cited of 86 All 86 (JSON)