mlsanityData to 14 Oct 2026

Short-Long Policy Evaluation with Novel Actions

arXiv 2407.03674 ↗First posted 4 Jul 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

From incorporating LLMs in education, to identifying new drugs and improving ways to charge batteries, innovators constantly try new strategies in search of better long-term outcomes for students, patients and consumers. One major bottleneck in this innovation cycle is the amount of time it takes to observe the downstream effects of a decision policy that incorporates new interventions. The key question is whether we can quickly evaluate long-term outcomes of a new decision policy without making long-term observations. Organizations often have access to prior data about past decision policies and their outcomes, evaluated over the full horizon of interest. Motivated by this, we introduce a new setting for short-long policy evaluation for sequential decision making tasks. Our proposed methods significantly outperform prior results on simulators of HIV treatment, kidney dialysis and battery charging. We also demonstrate that our methods can be useful for applications in AI safety by quickly identifying when a new decision policy is likely to have substantially lower performance than past policies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

25 of 44 references.

References · 25

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  3. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  4. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
  5. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 286
  6. 2019Batch Policy Learning under Constraintscited by 244
  7. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  8. 2023Composing Parameter-Efficient Modules with Arithmetic Operationscited by 156
  9. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guaranteescited by 140
  10. 2022AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuningcited by 137
  11. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 95
  12. 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 73
13 more
  1. 2019Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activitycited by 70
  2. 2020Meta-learning framework with applications to zero-shot time-series forecastingcited by 70
  3. 2019Off-Policy Estimation of Long-Term Average Outcomes with Applications to Mobile Healthcited by 42
  4. 2018Best arm identification in multi-armed bandits with delayed feedbackcited by 29
  5. 2020Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Frameworkcited by 29
  6. 2020A One-step Approach to Covariate Shift Adaptationcited by 21
  7. 2018Representation Balancing MDPs for Off-Policy Policy Evaluationcited by 20
  8. 2023Impatient Bandits: Optimizing Recommendations for the Long-Term Without Delaycited by 17
  9. 2023Inferring the Long-Term Causal Effects of Long-Term Treatments from Short-Term Experimentscited by 9
  10. 2015Optimization of anemia treatment in hemodialysis patients via reinforcement learningcited by 6
  11. 2023Evaluating the Surrogate Index as a Decision-Making Tool Using 200 A/B Tests at Netflixcited by 6
  12. 2022A Reinforcement Learning Approach to Estimating Long-term Treatment Effectscited by 5
  13. 2020Learning to Transfer Dynamic Models of Underactuated Soft Robotic Handscited by 1

Built on by

Not cited yet.