Bounded-Rationality, Hedging, and Generalization
Not on the map. Why?
Abstract (arXiv)
A learner does not only fit data; it also determines how strongly the training sample may shape its output and how much distortion it can hedge. We study this relation as a bounded-rational decision problem whose primitive object is the induced channel from samples to outputs. The learner's response law determines which changes in this channel are cheap or costly, and therefore induces both a lower tradeoff curve between training loss and sample dependence and a matched upper certificate curve. When the response law is represented by an f-divergence regularizer, these curves live in the regularizer's native information geometry, with KL as the special case corresponding to Shannon mutual information. We show how the hedge and the two curves can be recovered from black-box behavior by observing responses to scaled losses and local loss perturbations. In learning, population loss is empirical loss plus the distortion induced by the particular training sample. The recovered hedge gives a practical certificate when it covers that distortion. Thus generalization is treated as a testable hedging property of the learner's own response law.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 67 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
27 of 67 references.
References · 27
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
- 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
- 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
- 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
- 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
- 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
- 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 413
- 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
- 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 299
- 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
- 2019A Theory of Regularized Markov Decision Processescited by 279
15 more
- 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
- 2020Reasoning About Generalization via Conditional Mutual Informationcited by 145
- 2022Omnigrok: Grokking Beyond Algorithmic Datacited by 142
- 2019Tightening Mutual Information Based Bounds on Generalization Errorcited by 117
- 2020Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithmscited by 89
- 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
- 2019Distributionally Robust Optimization: A Reviewcited by 75
- 2023Generalization Bounds: Perspectives from Information Theory and PAC-Bayescited by 60
- 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
- 2021Regularized Policies are Reward Robustcited by 32
- 2015Information-Theoretic Bounded Rationalitycited by 21
- 2022Your Policy Regularizer is Secretly an Adversarycited by 21
- 2022f-divergences and their applications in lossy compression and bounding generalization errorcited by 7
- 2014An Adversarial Interpretation of Information-Theoretic Bounded Rationalitycited by 6
- 2016Human Decision-Making under Limited Timecited by 4
Built on by
Not cited yet.