# Robust Offline Policy Learning with Observational Data from Multiple Sources

arXiv: https://arxiv.org/abs/2410.08537 · HTML: https://mlsanity.com/paper/2410.08537/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Placed in

- Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/)

## Builds on (most cited first, 15 of 15)

- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizability](https://mlsanity.com/paper/2003.12699/) (2020)
- [Offline Multi-Action Policy Learning: Generalization and Optimization](https://mlsanity.com/paper/1810.04778/) (2018)
- [A Contextual Bandit Bake-off](https://mlsanity.com/paper/1802.04064/) (2018)
- [Balanced Policy Evaluation and Learning](https://mlsanity.com/paper/1705.07384/) (2017)
- [Active Learning for Cost-Sensitive Classification](https://mlsanity.com/paper/1703.01014/) (2017)
- [Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequality](https://mlsanity.com/paper/2212.09900/) (2022)
- [Minimax Regret Optimization for Robust Machine Learning under Distribution Shift](https://mlsanity.com/paper/2202.05436/) (2022)
- [Optimal Off-Policy Evaluation from Multiple Logging Policies](https://mlsanity.com/paper/2010.11002/) (2020)
- [Policy Learning with Adaptively Collected Data](https://mlsanity.com/paper/2105.02344/) (2021)
- [Effective Evaluation using Logged Bandit Feedback from Multiple Loggers](https://mlsanity.com/paper/1703.06180/) (2017)
- [A Theory of Multiple-Source Adaptation with Limited Target Labeled Data](https://mlsanity.com/paper/2007.09762/) (2020)
- [Multi-Task Off-Policy Learning from Bandit Feedback](https://mlsanity.com/paper/2212.04720/) (2022)
- [Off-policy Learning for Multiple Loggers](https://mlsanity.com/paper/1907.09652/) (2019)

## Built on by (most cited first, 3 of 3)

- [Minimax Regret Estimation for Generalizing Heterogeneous Treatment Effects with Multisite Data](https://mlsanity.com/paper/2412.11136/) (2024)
- [Doubly Robust Fusion of Many Treatments for Policy Learning](https://mlsanity.com/paper/2505.08092/) (2025)
- [Should I stay or should I show? Learning to selectively disclose information](https://mlsanity.com/paper/2609.39818/) (2026)

## Abstract (arXiv)

We consider the problem of using observational bandit feedback data from multiple heterogeneous data sources to learn a personalized decision policy that robustly generalizes across diverse target settings. To achieve this, we propose a minimax regret optimization objective to ensure uniformly low regret under general mixtures of the source distributions. We develop a policy learning algorithm tailored to this objective, combining doubly robust offline policy evaluation techniques and no-regret learning algorithms for minimax optimization. Our regret analysis shows that this approach achieves the minimal worst-case mixture regret up to a moderated vanishing rate of the total data across all sources. Our analysis, extensions, and experimental results demonstrate the benefits of this approach for learning robust decision policies from multiple data sources.