mlsanityData to 7 Oct 2026

OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning

arXiv 2502.11271 ↗First posted 16 Feb 2025cs.LG cs.CL cs.CVcited by 69

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Solving complex reasoning tasks may involve visual understanding, domain knowledge retrieval, numerical calculation, and multi-step reasoning. Existing methods augment large language models (LLMs) with external tools but are restricted to specialized domains, limited tool types, or require additional training data. In this paper, we introduce OctoTools, a training-free, user-friendly, and easily extensible multi-agent framework designed to tackle complex reasoning across diverse domains. OctoTools introduces standardized tool cards to encapsulate tool functionality, a planner for both high-level and low-level planning, and an executor to carry out tool usage. We validate OctoTools' generality across 16 diverse tasks (including MathVista, MMLU-Pro, MedQA, and GAIA-Text), achieving substantial average accuracy gains of 9.3% over GPT-4o. Furthermore, OctoTools also outperforms AutoGen, GPT-Functions, and LangChain by up to 10.6% when given the same set of tools. Through comprehensive analysi, ablations, and robustness tests with compact backbones and noisy tool environments, OctoTools demonstrates advantages in task planning, effective tool usage, and multi-step problem solving. Code, demos, and visualization are publicly available at https://octotools.github.io/.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 35 of its 67 references that are in the map. First posted on arXiv 16 Feb 2025.

Builds on

35 of 67 references.

References · 35

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  8. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  11. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  12. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
23 more
  1. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  2. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  3. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  4. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 277
  5. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
  6. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 215
  7. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  8. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 188
  9. 2022Internet-augmented language models through few-shot prompting for open-domain question answeringcited by 131
  10. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoningcited by 119
  11. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  12. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoningcited by 110
  13. 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agentcited by 108
  14. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 85
  15. 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
  16. 2024PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patternscited by 54
  17. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 47
  18. 2024Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agentscited by 39
  19. 2024Are Language Models Puzzle Prodigies? Algorithmic Puzzles Unveil Serious Challenges in Multimodal Reasoningcited by 23
  20. 2024Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Basescited by 23
  21. 2024Testing and Understanding Erroneous Planning in LLM Agents through Synthesized User Inputscited by 22
  22. 2023WebWISE: Web Interface Control and Sequential Exploration with Large Language Modelscited by 11
  23. 2024Less is More: Optimizing Function Calling for LLM Execution on Edge Devicescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 39; 2026: 30.

69 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  3. 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolutioncited by 114
  4. 2025PyVision: Agentic Vision with Dynamic Toolingcited by 71
  5. 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Usecited by 67
  6. 2025Paper2Poster: Towards Multimodal Poster Automation from Scientific Paperscited by 56
  7. 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agentcited by 53
  8. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  9. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  10. 2025El Agente: An Autonomous Agent for Quantum Chemistrycited by 43
  11. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 43
  12. 2025VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selectioncited by 34
38 more
  1. 2025AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxescited by 28
  2. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  3. 2025MLZero: A Multi-Agent System for End-to-end Machine Learning Automationcited by 22
  4. 2025Position: Agent Should Invoke External Tools ONLY When Epistemically Necessarycited by 22
  5. 2025Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executorscited by 20
  6. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 18
  7. 2026ToolTree: Efficient LLM Agent Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruningcited by 18
  8. 2025Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learningcited by 17
  9. 2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Usecited by 16
  10. 2025E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testingcited by 14
  11. 2025Xolver: Multi-Agent Reasoning with Holistic Experience Learning Just Like an Olympiad Teamcited by 13
  12. 2025DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoningcited by 13
  13. 2026OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agentscited by 13
  14. 2026Inference-Time Budget Control for LLM Search Agentscited by 13
  15. 2026SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoningcited by 13
  16. 2026SmartSearch: Process Reward-Guided Query Refinement for Search Agentscited by 12
  17. 2025ToolACE-MT: Non-Autoregressive Generation for Agentic Multi-Turn Interactioncited by 11
  18. 2026Sponge Tool Attack: Stealthy Denial-of-Efficiency against Tool-Augmented Agentic Reasoningcited by 11
  19. 2025Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Taskcited by 10
  20. 2026Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agentscited by 10
  21. 2025Look Less, Reason More: Rollout-Guided Adaptive Pixel-Space Reasoningcited by 9
  22. 2025RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflowscited by 8
  23. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  24. 2025PDE-Agent: A toolchain-augmented multi-agent framework for PDE solvingcited by 7
  25. 2025Open Agent Specification (Agent Spec): A Unified Representation for AI Agentscited by 5
  26. 2025TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoningcited by 5
  27. 2025AlphaApollo: A System for Deep Agentic Reasoningcited by 5
  28. 2026Any2Poster: Any-Source Poster Generation Across Modalities and Domainscited by 5
  29. 2025T^2Agent A Tool-augmented Multimodal Misinformation Detection Agent with Monte Carlo Tree Searchcited by 4
  30. 2025SIRI-Bench: Challenging VLMs' Spatial Intelligence through Complex Reasoning Taskscited by 4
  31. 2026ET-Agent: Incentivizing Effective Tool-Integrated Reasoning Agent via Behavior Calibrationcited by 4
  32. 2026KLong: Training LLM Agent for Extremely Long-horizon Taskscited by 4
  33. 2026AI Organizations are More Effective but Less Aligned than Individual Agentscited by 4
  34. 2025Agent WARPP: Workflow Adherence via Runtime Parallel Personalizationcited by 3
  35. 2025RefTool: Reference-Guided Tool Creation for Knowledge-Intensive Reasoningcited by 2
  36. 2025Radiologist Copilot: An Agentic Framework Orchestrating Specialized Tools for Reliable Radiology Reportingcited by 2
  37. 2026Cognitively Diverse Multiple-Choice Question Generation: A Hybrid Multi-Agent Framework with Large Language Modelscited by 2
  38. 2026GLANCE: A Global-Local Coordination Multi-Agent Framework for Music-Grounded Non-Linear Video Editingcited by 2

50 most cited of 69 All 69 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Modeling Complex Mathematical Reasoning via Large Language Model based MathAgent
  2. 2025AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent
  3. 2024Sibyl: Simple yet Effective Agent Framework for Complex Real-world Reasoning
  4. 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineering
  5. 2024Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation
  6. 2025AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
  7. 2024Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning
  8. 2024Infant Agent: A Tool-Integrated, Logic-Driven Agent with Cost-Effective API Usage