mlsanityData to 7 Oct 2026

Robust Offline Policy Learning with Observational Data from Multiple Sources

arXiv 2410.08537 ↗First posted 11 Oct 2024cs.LG stat.MLcited by 3

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

We consider the problem of using observational bandit feedback data from multiple heterogeneous data sources to learn a personalized decision policy that robustly generalizes across diverse target settings. To achieve this, we propose a minimax regret optimization objective to ensure uniformly low regret under general mixtures of the source distributions. We develop a policy learning algorithm tailored to this objective, combining doubly robust offline policy evaluation techniques and no-regret learning algorithms for minimax optimization. Our regret analysis shows that this approach achieves the minimal worst-case mixture regret up to a moderated vanishing rate of the total data across all sources. Our analysis, extensions, and experimental results demonstrate the benefits of this approach for learning robust decision policies from multiple data sources.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 15 of its 32 references that are in the map. First posted on arXiv 11 Oct 2024.

Builds on

15 of 32 references.

References · 15

  1. 2019Agnostic Federated Learningcited by 451
  2. 2016Generalized Random Forestscited by 309
  3. 2020Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizabilitycited by 114
  4. 2018A Contextual Bandit Bake-offcited by 64
  5. 2018Offline Multi-Action Policy Learning: Generalization and Optimizationcited by 64
  6. 2017Balanced Policy Evaluation and Learningcited by 51
  7. 2017Active Learning for Cost-Sensitive Classificationcited by 40
  8. 2022Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequalitycited by 32
  9. 2022Minimax Regret Optimization for Robust Machine Learning under Distribution Shiftcited by 31
  10. 2020Optimal Off-Policy Evaluation from Multiple Logging Policiescited by 30
  11. 2021Policy Learning with Adaptively Collected Datacited by 21
  12. 2017Effective Evaluation using Logged Bandit Feedback from Multiple Loggerscited by 20
3 more
  1. 2020A Theory of Multiple-Source Adaptation with Limited Target Labeled Datacited by 18
  2. 2022Multi-Task Off-Policy Learning from Bandit Feedbackcited by 9
  3. 2019Off-policy Learning for Multiple Loggerscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Minimax Regret Estimation for Generalizing Heterogeneous Treatment Effects with Multisite Datacited by 5
  2. 2025Doubly Robust Fusion of Many Treatments for Policy Learningcited by 1
  3. 2026Should I stay or should I show? Learning to selectively disclose informationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Federated Offline Policy Learning
  2. 2024Is Offline Decision Making Possible with Only Few Samples? Reliable Decisions in Data-Starved Bandits via Trust Region Enhancement
  3. 2026Offline-to-Online Learning in Linear Bandits
  4. 2023Oracle-Efficient Pessimism: Offline Policy Optimization in Contextual Bandits
  5. 2021Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization
  6. 2026Dynamic Minimax Regret Optimization for Robust LLM Post-Training
  7. 2026Offline Constrained RLHF with Multiple Preference Oracles
  8. 2025Reinforcement Learning for Individual Optimal Policy from Heterogeneous Data