Bootstrap-Conditioned Action Selection with Tabular Foundation Models
On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training
Abstract (arXiv)
Contextual bandits offer a natural framework for sample-efficient personalization, but practical deployment remains difficult under sparse, biased interaction data, unreliable uncertainty estimates, and severe cold starts. We study whether pre-trained tabular foundation models with in-context learning can be turned into randomized policies for online decision making. We propose BC-ICL (Bootstrap-conditioned action selection using ICL), which at each round draws a bootstrap resample of the interaction history, conditions a frozen pre-trained ICL model on that resample, scores all actions, and selects the action with the highest sampled score. We further introduce an arm-context conditioning architecture that promotes shared statistical strength across actions and helps avoid common bootstrap failure modes of isolated-arm bandits. Empirically, this policy delivers strong early-round regret and regret performance on standard contextual bandit suites, outperforming established baselines under a strict online protocol.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 22 of its 35 references that are in the map. First posted on arXiv 6 Aug 2026.
Builds on
22 of 35 references.
References · 22
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
- 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
- 2016Deep Exploration via Bootstrapped DQNcited by 527
- 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
- 2017A Tutorial on Thompson Samplingcited by 433
- 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
- 2017On Kernelized Multi-armed Banditscited by 352
- 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
- 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
- 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
10 more
- 2013Finite-Time Analysis of Kernelised Contextual Banditscited by 177
- 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 169
- 2020Neural Thompson Samplingcited by 128
- 2020Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspectivecited by 78
- 2017Ensemble Samplingcited by 62
- 2016The End of Optimism? An Asymptotic Analysis of Finite-Armed Linear Banditscited by 49
- 2018Garbage In, Reward Out: Bootstrapping Exploration in Multi-Armed Banditscited by 34
- 2014Thompson sampling with the online bootstrapcited by 19
- 2024Active Exploration via Autoregressive Generation of Missing Datacited by 9
- 2025Contextual Thompson Sampling via Generation of Missing Datacited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021GuideBoot: Guided Bootstrap for Deep Contextual Bandits
- 2024Diffusion Models Meet Contextual Bandits
- 2023Contextual Bandits and Imitation Learning via Preference-Based Active Queries
- 2024Online Bandit Learning with Offline Preference Data for Improved RLHF
- 2025Representative Action Selection for Large Action Space: From Bandits to MDPs
- 2024Bandits with Preference Feedback: A Stackelberg Game Perspective
- 2026Offline Contextual Bandits in the Presence of New Actions
- 2022Flexible and Efficient Contextual Bandits with Heterogeneous Treatment Effect Oracles