mlsanityData to 14 Oct 2026

POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learning

arXiv 2001.04032 ↗First posted 13 Jan 2020stat.ML cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

Many medical decision-making tasks can be framed as partially observed Markov decision processes (POMDPs). However, prevailing two-stage approaches that first learn a POMDP and then solve it often fail because the model that best fits the data may not be well suited for planning. We introduce a new optimization objective that (a) produces both high-performing policies and high-quality generative models, even when some observations are irrelevant for planning, and (b) does so in batch off-policy settings that are typical in healthcare, when only retrospective data is available. We demonstrate our approach on synthetic examples and a challenging medical decision-making problem.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). First posted on arXiv 13 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 8; 2022: 10; 2023: 6; 2024: 2; 2025: 5; 2026: 4.

36 papers.

Sort:

By area:

  1. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 114
  2. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 95
  3. 2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settingscited by 68
  4. 2021A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processescited by 36
  5. 2022Planning in Observable POMDPs in Quasipolynomial Timecited by 25
  6. 2022Future-Dependent Value-Based Off-Policy Evaluation in POMDPscited by 21
  7. 2022Scalable Decision-Focused Learning in Restless Multi-Armed Bandits with Application to Maternal and Child Healthcited by 12
  8. 2022Decision-Focused Learning without Differentiable Optimization: Learning Locally Optimized Decision Lossescited by 12
  9. 2022Data-Efficient Pipeline for Offline Reinforcement Learning with Limited Datacited by 12
  10. 2021The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulationcited by 11
  11. 2023TaskMet: Task-Driven Metric Learning for Model Learningcited by 11
  12. 2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
24 more
  1. 2025Solver-Free Decision-Focused Learning for Linear Optimization Problemscited by 9
  2. 2023Benchmarking PtO and PnO Methods in the Predictive Combinatorial Optimization Regimecited by 8
  3. 2022Offline Policy Optimization with Eligible Actionscited by 7
  4. 2021Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatmentcited by 4
  5. 2021Learning Under Adversarial and Interventional Shiftscited by 3
  6. 2021Learning MDPs from Features: Predict-Then-Optimize for Sequential Decision Problems by Reinforcement Learningcited by 3
  7. 2021On Learning Prediction-Focused Mixturescited by 3
  8. 2025Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AIcited by 3
  9. 2022Relative Sparsity for Medical Decision Problemscited by 2
  10. 2022Benchmarking Offline Reinforcement Learning Algorithms for E-Commerce Order Fraud Evaluationcited by 1
  11. 2023Decision-Focused Evaluation: Analyzing Performance of Deployed Restless Multi-Arm Banditscited by 1
  12. 2023Decision-Focused Model-based Reinforcement Learning for Reward Transfercited by 1
  13. 2024Efficient Off-Policy Learning for High-Dimensional Action Spacescited by 1
  14. 2026Action Shapley: A Training Data Selection Metric for World Model in Reinforcement Learningcited by 1
  15. 2021Extracting Expert's Goals by What-if Interpretable Modelingcited by 0
  16. 2022Towards Robust Off-Policy Evaluation via Human Inputscited by 0
  17. 2023Inference for relative sparsitycited by 0
  18. 2024Efficient Public Health Intervention Planning Using Decomposition-Based Decision-Focused Learningcited by 0
  19. 2025The Sample Complexity of Online Strategic Decision Making with Information Asymmetry and Knowledge Transportabilitycited by 0
  20. 2025Treatment, evidence, imitation, and chatcited by 0
  21. 2025Neural Index Policies for Restless Multi-Action Bandits with Heterogeneous Budgetscited by 0
  22. 2026Missingness-MDPs: Bridging the Theory of Missing Data and POMDPscited by 0
  23. 2026VentAgent: When LLMs Learn to Breathe -- Multi-Objective Arbitration for ARDS Ventilationcited by 0
  24. 2026DT2: Decision-Targeted Digital Twinscited by 0