mlsanityData to 7 Oct 2026

Small Language Models are the Future of Agentic AI

arXiv 2506.02153 ↗First posted 2 Jun 2025cs.AIcited by 254

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large language models (LLMs) are often praised for exhibiting near-human performance on a wide range of tasks and valued for their ability to hold a general conversation. The rise of agentic AI systems is, however, ushering in a mass of applications in which language models perform a small number of specialized tasks repetitively and with little variation. Here we lay out the position that small language models (SLMs) are sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems, and are therefore the future of agentic AI. Our argumentation is grounded in the current level of capabilities exhibited by SLMs, the common architectures of agentic systems, and the economy of LM deployment. We further argue that in situations where general-purpose conversational abilities are essential, heterogeneous agentic systems (i.e., agents invoking multiple different models) are the natural choice. We discuss the potential barriers for the adoption of SLMs in agentic systems and outline a general LLM-to-SLM agent conversion algorithm. Our position, formulated as a value statement, highlights the significance of the operational and economic impact even a partial shift from LLMs to SLMs is to have on the AI agent industry. We aim to stimulate the discussion on the effective use of AI resources and hope to advance the efforts to lower the costs of AI of the present day. Calling for both contributions to and critique of our position, we commit to publishing all such correspondence at https://research.nvidia.com/labs/lpr/slm-agents.

Key facts. Cited by 254 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 38 of its 79 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

38 of 79 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges27 Mar
  2. 2025Agentic AI Needs a Systems Theory28 Feb
  3. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systems5 Sep
  4. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
  5. 2024The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey17 Apr

Other references · 33

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  5. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  6. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  7. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  9. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  10. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  11. 2023A Comprehensive Overview of Large Language Modelscited by 639
  12. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 295
21 more
  1. 2021Scaling Laws for Transfercited by 265
  2. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 247
  3. 2023PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPUcited by 210
  4. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  5. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 125
  6. 2024Small Language Models: Survey, Measurements, and Insightscited by 109
  7. 2024PowerInfer-2: Fast Large Language Model Inference on a Smartphonecited by 101
  8. 2024Hymba: A Hybrid-head Architecture for Small Language Modelscited by 95
  9. 2024On Protecting the Data Privacy of Large Language Models (LLMs): A Surveycited by 91
  10. 2025Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Modelscited by 88
  11. 2023MatFormer: Nested Transformer for Elastic Inferencecited by 79
  12. 2025Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-trainingcited by 62
  13. 2024Flextron: Many-in-One Flexible Large Language Modelcited by 42
  14. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 34
  15. 2024DELIFT: Data Efficient Language model Instruction Fine Tuningcited by 14
  16. 2024AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deploymentcited by 12
  17. 2024Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Modelscited by 7
  18. 2024An Open Source Python Library for Anonymizing Sensitive Datacited by 4
  19. 2024Tiny Transformers Excel at Sentence Compressioncited by 2
  20. 2025Text Compression for Efficient Language Generationcited by 1
  21. 2025Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 108; 2026: 146.

254 papers.

Sort:

By area:

  1. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 128
  2. 2025Tongyi DeepResearch Technical Reportcited by 53
  3. 2025Towards General Agentic Intelligence via Environment Scalingcited by 51
  4. 2025ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestrationcited by 44
  5. 2025The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspectivecited by 38
  6. 2025CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimizationcited by 37
  7. 2025Self-Improving LLM Agents at Test-Timecited by 30
  8. 2025ARE: Scaling Up Agent Environments and Evaluationscited by 28
  9. 2025LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarioscited by 25
  10. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  11. 2026Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baselinecited by 18
  12. 2025Virtual Agent Economiescited by 17
38 more
  1. 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMscited by 16
  2. 2025AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial Asset Operations and Maintenancecited by 15
  3. 2025Agent Exchange: Shaping the Future of AI Agent Economicscited by 15
  4. 2025MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Sciencecited by 15
  5. 2026Training Language Models via Neural Cellular Automatacited by 15
  6. 2025Ferret-UI Lite: Lessons from Building Small On-Device GUI Agentscited by 14
  7. 2026Security Threat Modeling for Emerging AI-Agent Protocols: A Comparative Analysis of MCP, A2A, Agora, and ANPcited by 14
  8. 2025Intelligence per Watt: Measuring Intelligence Efficiency of Local AIcited by 13
  9. 2025The Collaboration Gap: Exploration and Benchmarking of Open-World Agentic Cooperationcited by 11
  10. 2026Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errorscited by 10
  11. 2025AutoTool: Efficient Tool Selection for Large Language Model Agentscited by 9
  12. 2025From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulationcited by 8
  13. 2025Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5Bcited by 8
  14. 2025An Information Theoretic Perspective on Agentic System Designcited by 8
  15. 2025Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Modelscited by 8
  16. 2025Planner-R1: Reward Shaping Enables Efficient Agentic RL with Smaller LLMscited by 7
  17. 2025SpeakRL: Synergizing Reasoning, Speaking, and Acting in Language Models with Reinforcement Learningcited by 7
  18. 2026Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Usecited by 7
  19. 2026Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptationcited by 7
  20. 2025Batch Query Processing and Optimization for Agentic Workflowscited by 6
  21. 2025AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Productioncited by 6
  22. 2025An Evaluation-Centric Paradigm for Scientific Visualization Agentscited by 6
  23. 2025Federation of Agents: A Semantics-Aware Communication Fabric for Large-Scale Agentic AIcited by 6
  24. 2025Training Large Language Models To Reason In Parallel With Global Forking Tokenscited by 6
  25. 2025Don't Adapt Small Language Models for Tools; Adapt Tool Schemas to the Modelscited by 6
  26. 2025APTBench: Benchmarking Agentic Potential of Base LLMs During Pre-Trainingcited by 6
  27. 2025Klear-AgentForge: Forging Agentic Intelligence through Posttraining Scalingcited by 6
  28. 2025Towards Active Synthetic Data Generation for Finetuning Language Modelscited by 6
  29. 2025Slm-mux: Orchestrating small language models for reasoningcited by 5
  30. 2025SWEnergy: An Empirical Study on Energy Efficiency in Agentic Issue Resolution Frameworks with SLMscited by 5
  31. 2025SANet: A Semantic-aware Agentic AI Networking Framework for Cross-layer Optimization in 6Gcited by 5
  32. 2025Nested Browser-Use Learning for Agentic Information Seekingcited by 5
  33. 2026Modular Prompt Optimization: Optimizing Structured Prompts with Section-Local Textual Gradientscited by 5
  34. 2026MASEval: Extending Multi-Agent Evaluation from Models to Systemscited by 5
  35. 2026Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awarenesscited by 5
  36. 2026EmoMAS: Emotion-Aware Multi-Agent System for High-Stakes Edge-Deployable Negotiation with Bayesian Orchestrationcited by 5
  37. 2026AgentStop: Terminating Local AI Agents Early to Save Energy in Consumer Devicescited by 5
  38. 2025ShadowNPU: System and Algorithm Co-design for NPU-Centric On-Device LLM Inferencecited by 4

50 most cited of 254 All 254 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Survey
  2. 2025Small Language Models for Agentic Systems: A Survey of Architectures, Capabilities, and Deployment Trade offs
  3. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey
  4. 2023A Survey on Large Language Model based Autonomous Agents
  5. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  6. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
  7. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  8. 2024Practical Considerations for Agentic LLM Systems