mlsanityData to 13 Oct 2026

Agentic AIs Are the Missing Paradigm for Out-of-Distribution Generalization in Foundation Models

arXiv 2605.06522 ↗First posted 7 May 2026cs.LG cs.CVcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Foundation models (FMs) are increasingly deployed in open-world settings where distribution shift is the rule rather than the exception. The out-of-distribution (OOD) phenomena they face -- knowledge boundaries, capability ceilings, compositional shifts, and open-ended task variation -- differ in kind from the settings that have shaped prior OOD research, and are further complicated because the pretraining and post-training distributions of modern FMs are often only partially observed. Our position is that OOD for foundation models is a structurally distinct problem that cannot be solved within the prevailing model-centric paradigm, and that agentic systems constitute the missing paradigm required to address it. We defend this claim through four steps. First, we give a stage-aware formalization of OOD that accommodates partially observed multi-stage training distributions. Second, we prove a parameter coverage ceiling: there exist practically relevant inputs that no model-centric method (training-time or test-time) can handle within tolerance varepsilon, for reasons intrinsic to parameter-based representation. Third, we characterize agentic OOD systems by four structural properties -- perception, strategy selection, external action, and closed-loop verification -- and show that they strictly extend the reachable set beyond the ceiling. Fourth, we respond to seven counterarguments, conceding two, and outline a research agenda. We do not claim that agentic methods subsume model-centric ones; we argue that the two are complementary, and that progress on FM-OOD requires explicit recognition of the agentic paradigm as a first-class research direction.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 54 of its 66 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

54 of 66 references.

References · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2022Training language models to follow instructions with human feedbackcited by 15,856
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,072
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,221
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,502
  8. 2023Segment Anythingcited by 8,195
  9. 2020Scaling Laws for Neural Language Modelscited by 6,494
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,657
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,170
  12. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,156
38 more
  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,117
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,999
  3. 2015Domain-Adversarial Training of Neural Networkscited by 3,030
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,862
  5. 2022Emergent Abilities of Large Language Modelscited by 2,499
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,068
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,904
  8. 2019Invariant Risk Minimizationcited by 1,842
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,594
  10. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,404
  11. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,371
  12. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,242
  13. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 899
  14. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 865
  15. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 846
  16. 2023Vision-Language Models for Vision Tasks: A Surveycited by 837
  17. 2021Domain Generalization: A Surveycited by 760
  18. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 738
  19. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 697
  20. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  21. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 556
  22. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 498
  23. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 455
  24. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 434
  25. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 432
  26. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 406
  27. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 381
  28. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 335
  29. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 320
  30. 2022Confident Adaptive Language Modelingcited by 302
  31. 2020Consistent Estimators for Learning to Defer to an Expertcited by 268
  32. 2023How is ChatGPT's behavior changing over time?cited by 268
  33. 2020The Risks of Invariant Risk Minimizationcited by 262
  34. 2022RealTime QA: What's the Answer Right Now?cited by 242
  35. 2020Selective Question Answering under Domain Shiftcited by 231
  36. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 205
  37. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  38. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Single-Agent Scaling Fails Multi-Agent Intelligence: Towards Foundation Models with Native Multi-Agent Intelligence
  2. 2024Foundation Model Sherpas: Guiding Foundation Models through Knowledge and Reasoning
  3. 2025Scaling Agents via Continual Pre-training
  4. 2025Foundation Models as World Models: A Foundational Study in Text-Based GridWorlds
  5. 2026The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective
  6. 2026Position: It is Time to Virtualize Foundation Models with a Self-evolving Operating System Layer
  7. 2024A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Model
  8. 2024Position: Foundation Agents as the Paradigm Shift for Decision Making