mlsanityData to 14 Oct 2026

Reinforcement Learning Foundation Models Should Already Be A Thing

arXiv 2606.18812 ↗First posted 17 Jun 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL with diffusion and flow models

Abstract (arXiv)

Foundation models for language and vision are powered by internet-scale data, while structured domains such as tabular prediction are powered by synthetic data. This substitute shifts the challenge from collection to prior design. Such priors already exist for many structured tasks: TabPFN and its successors solve tabular classification with a transformer pretrained on a synthetic Bayesian prior. We make two points. First, reinforcement learning is the conspicuous gap: sampling a synthetic MDP is as feasible as sampling a synthetic tabular dataset, yet no in-context RL work treats prior design as a primary objective. Second, MDPs admit a fixed-size sufficient statistic, independent of the episodes observed and tabular in shape, which makes them directly amenable to the attention-based architectures used for tabular foundation models, with a policy head replacing the supervised target. Together these define the agenda for an RL foundation model. As a proof of concept, we train a Graph Attention Network entirely on synthetic MDPs and show that, with no task-specific tuning, it solves held-out tabular benchmarks in context, both online and offline: online, in far fewer episodes than UCB-VI and tabular Q-learning, and offline, competitively with VI-LCB.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 30 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

23 of 30 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  6. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  7. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  8. 2017Minimax Regret Bounds for Reinforcement Learningcited by 503
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  10. 2016Learning to reinforcement learncited by 377
  11. 2021Transformers Can Do Bayesian Inferencecited by 360
  12. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 289
11 more
  1. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 279
  2. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
  3. 2026TabICLv2: A better, faster, scalable, and open tabular foundation modelcited by 144
  4. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
  5. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 83
  6. 2022LMPriors: Pre-Trained Language Models as Task-Specific Priorscited by 56
  7. 2024Efficient Reinforcement Learning with Large Language Model Priorscited by 30
  8. 2025Position: The Future of Bayesian Prediction Is Prior-Fittedcited by 20
  9. 2025Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planningcited by 13
  10. 2025ICR-RL: Deep Reinforcement Learning via In-Context Regressioncited by 8
  11. 2026End-to-End Compression for Tabular Foundation Modelscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mind the Gap? A Distributional Comparison of Real and Synthetic Priors for Tabular Foundation Models
  2. 2026Model-Aware Data Cleaning for Tabular Foundation Models
  3. 2024TabDPT: Scaling Tabular Foundation Models on Real Data
  4. 2025Strengthening LLMs for Tabular Prediction with Structural Priors
  5. 2026TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention
  6. 2026Self-Improving Tabular Language Models via Iterative Reward-Guided Post-Training
  7. 2024Reinforcement Learning with Generative Models for Compact Support Sets
  8. 2026ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learning