mlsanityData to 7 Oct 2026

Robust Generalization despite Distribution Shift via Minimum Discriminating Information

arXiv 2106.04443 ↗First posted 8 Jun 2021cs.LG cs.IT math.ITcited by 6

Not on the map. Why?

Abstract (arXiv)

Training models that perform well under distribution shifts is a central challenge in machine learning. In this paper, we introduce a modeling framework where, in addition to training data, we have partial structural knowledge of the shifted test distribution. We employ the principle of minimum discriminating information to embed the available prior knowledge, and use distributionally robust optimization to account for uncertainty due to the limited samples. By leveraging large deviation results, we obtain explicit generalization bounds with respect to the unknown shifted distribution. Lastly, we demonstrate the versatility of our framework by demonstrating it on two rather distinct applications: (1) training classifiers on systematically biased data and (2) off-policy evaluation in Markov Decision Processes.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 25 of its 88 references that are in the map. First posted on arXiv 8 Jun 2021.

Builds on

25 of 88 references.

References · 25

  1. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
  2. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  3. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
  4. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  5. 2019A Theory of Regularized Markov Decision Processescited by 279
  6. 2017A unified view of entropy-regularized Markov decision processescited by 174
  7. 2019Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learningcited by 169
  8. 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
  9. 2014From Predictive to Prescriptive Analyticscited by 142
  10. 2015Distributionally Robust Logistic Regressioncited by 124
  11. 2019Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processescited by 118
  12. 2018Anchor regression: heterogeneous data meets causalitycited by 116
13 more
  1. 2013Data-Driven Robust Optimizationcited by 84
  2. 2017Regularization via Mass Transportationcited by 79
  3. 2019Distributionally Robust Optimization and Generalization in Kernel Methodscited by 73
  4. 2020Coping with Label Shift via Distributionally Robust Optimisationcited by 69
  5. 2020Distributionally Robust Bayesian Optimizationcited by 60
  6. 2018Stable Prediction across Unknown Environmentscited by 55
  7. 2019Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimationcited by 48
  8. 2017From Data to Decisions: Distributionally Robust Optimization is Optimalcited by 36
  9. 2015Counterfactual Risk Minimization: Learning from Logged Bandit Feedbackcited by 33
  10. 2019Entropic Regularization of Markov Decision Processescited by 15
  11. 2017Bootstrap Robust Prescriptive Analyticscited by 12
  12. 2020A Pareto Dominance Principle for Data-Driven Optimizationcited by 6
  13. 2017Generalized maximum entropy estimationcited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 1; 2024: 0; 2025: 3.

6 papers.

  1. 2022RankFeat: Rank-1 Feature Removal for Out-of-distribution Detectioncited by 52
  2. 2021A Survey of Learning Criteria Going Beyond the Usual Riskcited by 4
  3. 2025Towards Optimal Offline Reinforcement Learningcited by 2
  4. 2025Distributional Adversarial Attacks and Training in Deep Hedgingcited by 2
  5. 2025Why Cannot Neural Networks Master Extrapolation? Insights from Physical Lawscited by 2
  6. 2023RankFeat&RankWeight: Rank-1 Feature/Weight Removal for Out-of-distribution Detectioncited by 1