mlsanityData to 7 Oct 2026

Adaptivity Under Realizability Constraints: Comparing In-Context and Agentic Learning

arXiv 2605.04995 ↗First posted 6 May 2026cs.LG math.ST stat.MLcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

We compare in-context learning with fixed queries and agentic learning with adaptive queries for uniform approximation of task families. We consider two settings: an unrestricted regime, where querying and approximation are arbitrary functions, and a realizable regime, where we require these operations to be implemented by ReLU neural networks. In both settings, adaptivity never hinders approximation performance. However, this advantage can change when one passes from the unrestricted regime to the realizable regime. We identify four distinct approximation scenarios, each witnessed by an explicit task family: (a) no advantage of adaptivity; (b) an advantage in the unrestricted regime that persists under ReLU realizability; (c) an advantage that arises only under realizability; and (d) an advantage that disappears under realizability. This demonstrates that representational constraints interact profoundly with the effect of adaptivity.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 28 of its 53 references that are in the map. First posted on arXiv 6 May 2026.

Builds on

28 of 53 references.

References · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  7. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  8. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  9. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  10. 2016Error bounds for approximations with deep ReLU networkscited by 567
  11. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 176
  12. 2023How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?cited by 93
16 more
  1. 2022Transformers as Meta-Learners for Implicit Neural Representationscited by 79
  2. 2024Transformers are Universal In-context Learnerscited by 42
  3. 2021Universal Approximation Theorems for Differentiable Geometric Deep Learningcited by 41
  4. 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
  5. 2023The Parametric Complexity of Operator Learningcited by 27
  6. 2024Mathematical theory of deep learningcited by 20
  7. 2025Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Timecited by 15
  8. 2024Optimal sampling for least-squares approximationcited by 6
  9. 2024Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoderscited by 5
  10. 2025Generative Neural Operators of Log-Complexity Can Simultaneously Solve Infinitely Many Convex Programscited by 5
  11. 2025Transformers Meet In-Context Learning: A Universal Approximation Theorycited by 4
  12. 2025Is In-Context Universality Enough? MLPs are Also Universal In-Contextcited by 3
  13. 2025Provable Low-Frequency Bias of In-Context Learning of Representationscited by 3
  14. 2025One model to solve them all: 2BSDE families via neural operatorscited by 3
  15. 2025Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Datacited by 1
  16. 2026Equivalence of approximation by networks of single- and multi-spike neuronscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Models
  2. 2026Everywhere Learning: Artificial Intelligence with Pointwise Constraints
  3. 2026ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?
  4. 2026Adaptive Retrieval helps Reasoning in LLMs -- but mostly if it's not used
  5. 2019Active Learning within Constrained Environments through Imitation of an Expert Questioner
  6. 2026When Does In-Context Search Help? A Sampling-Complexity Theory of Reflection-Driven Reasoning
  7. 2026Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
  8. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills