mlsanityData to 7 Oct 2026

Conditional Importance Sampling for Off-Policy Learning

arXiv 1910.07479 ↗First posted 16 Oct 2019cs.LG stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

The principal contribution of this paper is a conceptual framework for off-policy reinforcement learning, based on conditional expectations of importance sampling ratios. This framework yields new perspectives and understanding of existing off-policy algorithms, and reveals a broad space of unexplored algorithms. We theoretically analyse this space, and concretely investigate several algorithms that arise from this framework.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 2; 2023: 2; 2024: 0; 2025: 1; 2026: 3.

12 papers.

  1. 2026Good SFT Optimizes for SFT, Better SFT Prepares for Reinforcement Learningcited by 10
  2. 2023Marginal Density Ratio for Off-Policy Evaluation in Contextual Banditscited by 9
  3. 2021High-Confidence Off-Policy (or Counterfactual) Variance Estimationcited by 8
  4. 2021SOPE: Spectrum of Off-Policy Estimatorscited by 5
  5. 2022Low Variance Off-policy Evaluation with State-based Importance Samplingcited by 5
  6. 2021State Relevance for Off-Policy Evaluationcited by 3
  7. 2023A new Gradient TD Algorithm with only One Step-size: Convergence Rate Analysis using L-λ Smoothnesscited by 2
  8. 2025Exploiting Similarities in A/B Testing with Off-Policy Estimationcited by 1
  9. 2021UVIP: Model-Free Approach to Evaluate Reinforcement Learning Algorithmscited by 0
  10. 2022Marginalized Operators for Off-policy Reinforcement Learningcited by 0
  11. 2026Quotient DAGs for Off-Policy Evaluation:Forward-Flow Importance Sampling and Exact Slate Propensitiescited by 0
  12. 2026Free Everywhere, Exact on Trees: PPO's Dropped Correction Buys Sample Efficiency Under Aggressive Reusecited by 0