mlsanityData to 14 Oct 2026

Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration

arXiv 2511.15351 ↗First posted 19 Nov 2025cs.AI cs.CVcited by 9

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Existing multimodal reasoning models and frameworks suffer from fundamental architectural limitations: most lack the human-like ability to autonomously explore diverse reasoning pathways-whether in direct inference, tool-driven visual exploration, programmatic visual manipulation, or intrinsic visual imagination. Consequently, they struggle to adapt to dynamically changing capability requirements in real-world tasks. Meanwhile, humans exhibit a complementary set of thinking abilities when addressing such tasks, whereas existing methods typically cover only a subset of these dimensions. Inspired by this, we propose Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration, a new paradigm for multimodal agentic reasoning. We define six core capabilities essential for multimodal reasoning and organize a comprehensive evaluation benchmark, Octopus-Bench, accordingly. Octopus is capable of autonomously exploring during reasoning and dynamically selecting the most appropriate capability based on the current state. Experimental results show that Octopus achieves the best performance on the vast majority of tasks in Octopus-Bench, highlighting the crucial role of capability coordination in agentic multimodal reasoning.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 57 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

46 of 57 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoning3 Nov
  2. 2025DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning30 Sep
  3. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens20 Jun
  4. 2025Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification8 Jun
  5. 2025Thinking with Generated Images28 May
  6. 2024CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models17 Dec
  7. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
  8. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
  9. 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models22 May
  10. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  11. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  12. 2023ViperGPT: Visual Inference via Python Execution for Reasoning14 Mar
1 more
  1. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb

Other references · 33

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  12. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
21 more
  1. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  2. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  3. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  4. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  5. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
  6. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 299
  7. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 136
  8. 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 102
  9. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  10. 2025Visual Planning: Let's Think Only with Imagescited by 85
  11. 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agentscited by 69
  12. 2024Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Modelscited by 66
  13. 2024DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLMcited by 46
  14. 2024IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representationscited by 38
  15. 2025GraphMaster: Automated Graph Synthesis via LLM Agents in Data-Limited Environmentscited by 19
  16. 2025WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generationcited by 16
  17. 2024MMFactory: A Universal Solution Search Engine for Vision-Language Taskscited by 7
  18. 2025GraphOracle: Efficient Fully-Inductive Knowledge Graph Reasoning via Relation-Dependency Graphscited by 7
  19. 2025Mixture of Length and Pruning Experts for Knowledge Graphs Reasoningcited by 5
  20. 2025How Many Code and Test Cases Are Enough? Evaluating Test Cases Generation from a Binary-Matrix Perspectivecited by 4
  21. 2025Decoupling Continual Semantic Segmentationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Agent-ScanKit: Unraveling Memory and Reasoning of Multimodal Agents via Sensitivity Perturbations
  2. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models
  3. 2025Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks
  4. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning
  5. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoning
  6. 2024MageBench: Bridging Large Multimodal Models to Agents
  7. 2025MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models
  8. 2026A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applications