mlsanityData to 7 Oct 2026

An LLM Compiler for Parallel Function Calling

arXiv 2312.04511 ↗First posted 7 Dec 2023cs.CLcited by 170

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

The reasoning capabilities of the recent LLMs enable them to execute external function calls to overcome their inherent limitations, such as knowledge cutoffs, poor arithmetic skills, or lack of access to private data. This development has allowed LLMs to select and coordinate multiple functions based on the context to tackle more complex problems. However, current methods for function calling often require sequential reasoning and acting for each function which can result in high latency, cost, and sometimes inaccurate behavior. To address this, we introduce LLMCompiler, which executes functions in parallel to efficiently orchestrate multiple function calls. Drawing inspiration from the principles of classical compilers, LLMCompiler enables parallel function calling with three components: (i) a Function Calling Planner, formulating execution plans for function calling; (ii) a Task Fetching Unit, dispatching function calling tasks; and (iii) an Executor, executing these tasks in parallel. LLMCompiler automatically generates an optimized orchestration for the function calls and can be used with both open-source and closed-source models. We have benchmarked LLMCompiler on a range of tasks with different patterns of function calling. We observe consistent latency speedup of up to 3.7x, cost savings of up to 6.7x, and accuracy improvement of up to ~9% compared to ReAct. Our code is available at https://github.com/SqueezeAILab/LLMCompiler.

Key facts. Cited by 170 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 41 of its 56 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

41 of 56 references.

References · 41

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  12. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
29 more
  1. 2021Program Synthesis with Large Language Modelscited by 3,509
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  5. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  6. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  8. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  9. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  10. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  11. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  13. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  14. 2021Measuring Coding Challenge Competence With APPScited by 879
  15. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  16. 2023Large Language Modelscited by 774
  17. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  18. 2022PAL: Program-aided Language Modelscited by 635
  19. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  20. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  21. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  22. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  23. 2023ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Modelscited by 174
  24. 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
  25. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 157
  26. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  27. 2023LASER: LLM Agent with State-Space Exploration for Web Navigationcited by 66
  28. 2022Is a Question Decomposition Unit All We Need?cited by 37
  29. 2023IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositionscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 31; 2025: 48; 2026: 89.

170 papers.

Sort:

By area:

  1. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  2. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  3. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  4. 2025Learning Adaptive Parallel Reasoning with Language Modelscited by 58
  5. 2024TinyAgent: Function Calling at the Edgecited by 55
  6. 2024Hammer: Robust Function-Calling for On-Device Language Models via Function Maskingcited by 53
  7. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  8. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Livecited by 43
  9. 2024RATT: A Thought Structure for Coherent and Correct LLM Reasoningcited by 39
  10. 2025LEGOMem: Modular Procedural Memory for Multi-agent LLM Systems for Workflow Automationcited by 39
  11. 2025The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspectivecited by 38
  12. 2024Teola: Towards End-to-End Optimization of LLM-based Applicationscited by 32
38 more
  1. 2025Hogwild! Inference: Parallel LLM Generation via Concurrent Attentioncited by 32
  2. 2024LAMBDA: A Large Model Based Data Agentcited by 30
  3. 2024GeoLLM-Engine: A Realistic Environment for Building Geospatial Copilotscited by 27
  4. 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunitiescited by 26
  5. 2023TypeFly: Flying Drones with Large Language Modelcited by 23
  6. 2024Efficient and Scalable Estimation of Tool Representations in Vector Spacecited by 22
  7. 2025Multi-Agent Image Restorationcited by 19
  8. 2025DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discoverycited by 18
  9. 2024Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Executioncited by 16
  10. 2024Agreement-Based Cascading for Efficient Inferencecited by 16
  11. 2025Divide-Then-Aggregate: An Efficient Tool Learning Method via Parallel Tool Invocationcited by 16
  12. 2025Pie: A Programmable Serving System for Emerging LLM Applicationscited by 15
  13. 2024An LLM-Tool Compiler for Fused Parallel Function Callingcited by 14
  14. 2024AI Metropolis: Scaling Large Language Model-based Multi-Agent Simulation with Out-of-order Executioncited by 14
  15. 2025Optimizing Sequential Multi-Step Tasks with Parallel LLM Agentscited by 14
  16. 2024Budget-Constrained Tool Learning with Planningcited by 11
  17. 2025A Survey on Parallel Reasoningcited by 11
  18. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 11
  19. 2025Exploring the Necessity of Reasoning in LLM-based Agent Scenarioscited by 10
  20. 2025SPRINT: Enabling Interleaved Planning and Parallelized Execution in Reasoning Modelscited by 10
  21. 2025TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrievalcited by 9
  22. 2025Benchmarking Failures in Tool-Augmented Language Modelscited by 9
  23. 2025Orchestrating Agents and Data for Enterprise: A Blueprint Architecture for Compound AIcited by 9
  24. 2025AutoTool: Efficient Tool Selection for Large Language Model Agentscited by 9
  25. 2026Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspectivecited by 9
  26. 2024APPL: A Prompt Programming Language for Harmonious Integration of Programs and Large Language Model Promptscited by 8
  27. 2025Template-Based Financial Report Generation in Agentic and Decomposed Information Retrievalcited by 8
  28. 2025HiRA: A Hierarchical Reasoning Framework for Decoupled Planning and Execution in Deep Searchcited by 8
  29. 2025A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QAcited by 8
  30. 2026MobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarioscited by 8
  31. 2024LLM-dCache: Improving Tool-Augmented LLMs with GPT-Driven Localized Data Cachingcited by 7
  32. 2024Improving Small-Scale Large Language Models Function Calling for Reasoning Taskscited by 7
  33. 2024Multi-Modal Data Exploration via Language Agentscited by 7
  34. 2025Step-Audio-AQAA: a Fully End-to-End Expressive Large Audio Language Modelcited by 7
  35. 2025Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigmscited by 7
  36. 2026Agentic Test-Time Scaling for WebAgentscited by 7
  37. 2026SkCC: Portable and Secure Skill Compilation for Cross-Framework LLM Agentscited by 7
  38. 2024A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWarescited by 6

50 most cited of 170 All 170 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Function Orchestration for Large Language Models
  2. 2024Asynchronous LLM Function Calling
  3. 2025LLM4VV: Evaluating Cutting-Edge LLMs for Generation and Evaluation of Directive-Based Parallel Programming Model Compiler Tests ↗
  4. 2025REASONING COMPILER: LLM-Guided Optimizations for Efficient Model Serving
  5. 2026LiteCoOp: Lightweight Multi-LLM Shared-Tree Reasoning for Model-Serving Compiler Optimizations
  6. 2024Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning
  7. 2025Unified Tool Integration for LLMs: A Protocol-Agnostic Approach to Function Calling
  8. 2024APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Serving