mlsanityData to 14 Oct 2026

OrchSLM: Probing the Dynamics of Small Language Model Orchestration

arXiv 2609.13470 ↗First posted 11 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Although large language models (LLMs) have demonstrated remarkable capabilities, their reliance on cloud-scale infrastructure poses fundamental challenges for deployment in agentic pipelines, including latency, privacy, connectivity, and substantial computational cost. Small language models (SLMs) offer a compelling alternative: recent studies suggest that many repetitive and narrowly scoped subtasks in agentic workloads may be better served by specialized SLMs than by monolithic LLMs. However, the limited capacity and context windows of SLMs can constrain long-horizon reasoning and interaction-heavy orchestration strategies such as iterative verification and debate. This motivates a complementary, non-interactive paradigm in which heterogeneous SLMs independently generate candidate solutions and a router orchestrates their cached samples without further model interaction. To further understand the mechanisms of such orchestration, we introduce OrchSLM, a routing framework that unifies existing non-interactive orchestration methods and exposes their underlying design choices as controllable parameters. Using OrchSLM as a systematic probe, we reveal how orchestration behavior emerges from diverse knobs, including the task structure, model-pool composition, and multi-agent consensus.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 44 of its 47 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

44 of 47 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Attention-MoA: Enhancing Mixture-of-Agents via Inter-Agent Semantic Attention and Deep Residual Synthesis23 Jan
  2. 2025Small Language Models are the Future of Agentic AI2 Jun
  3. 2025RMoA: Optimizing Mixture-of-Agents through Diversity Maximization and Residual Compensation30 May
  4. 2025Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers27 Feb
  5. 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?2 Feb
  6. 2024SMoA: Improving Multi-agent Large Language Models with Sparse Mixture-of-Agents5 Nov
  7. 2024Mixture-of-Agents Enhances Large Language Model Capabilities7 Jun
  8. 2024More Agents Is All You Need3 Feb

Other references · 36

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  11. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  12. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
24 more
  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  2. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  3. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  4. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  5. 2025s1: Simple test-time scalingcited by 1,341
  6. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  7. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  8. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  9. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  10. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 335
  11. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 291
  12. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  13. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  14. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 127
  15. 2024Systematic Biases in LLM Simulations of Debatescited by 99
  16. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 88
  17. 2025Cache-to-Cache: Direct Semantic Communication Between Large Language Modelscited by 61
  18. 2025Optimizing Model Selection for Compound AI Systemscited by 36
  19. 2025Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolutioncited by 19
  20. 2025Slm-mux: Orchestrating small language models for reasoningcited by 6
  21. 2025CTTS: Collective Test-Time Scalingcited by 3
  22. 2026ReM-MoA: Reasoning Memory Sustains Mixture-of-Agents Scalingcited by 2
  23. 2026Reasoning Consensus: Structural Ensembling of LLM Reasoning via Weighted DAG Aggregationcited by 2
  24. 2025Roundtable Policy: Confidence-Weighted-Consensus Aggregation Improves Multi-Agent-System Reasoningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Small Language Models for Agentic Systems: A Survey of Architectures, Capabilities, and Deployment Trade offs
  2. 2026Can Small Agents Collaborate to Beat a Single Large Language Model?
  3. 2026Uno-Orchestra: Parsimonious Agent Routing via Selective Delegation
  4. 2026ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents
  5. 2026Rethinking Scale: Deployment Trade-offs of Small Language Models under Agent Paradigms
  6. 2026AdaptOrch: Task-Adaptive Multi-Agent Orchestration in the Era of LLM Performance Convergence
  7. 2026A Formal Hierarchical Architecture for Agentic Orchestration with Stack-Based Execution and Lazy Discovery
  8. 2026Sutradhara: An Intelligent Orchestrator-Engine Co-design for Tool-based Agentic Inference