Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration
On the map: LLM reasoning → Multimodal reasoning
also in 1 more
Abstract (arXiv)
Existing multimodal reasoning models and frameworks suffer from fundamental architectural limitations: most lack the human-like ability to autonomously explore diverse reasoning pathways-whether in direct inference, tool-driven visual exploration, programmatic visual manipulation, or intrinsic visual imagination. Consequently, they struggle to adapt to dynamically changing capability requirements in real-world tasks. Meanwhile, humans exhibit a complementary set of thinking abilities when addressing such tasks, whereas existing methods typically cover only a subset of these dimensions. Inspired by this, we propose Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration, a new paradigm for multimodal agentic reasoning. We define six core capabilities essential for multimodal reasoning and organize a comprehensive evaluation benchmark, Octopus-Bench, accordingly. Octopus is capable of autonomously exploring during reasoning and dynamically selecting the most appropriate capability based on the current state. Experimental results show that Octopus achieves the best performance on the vast majority of tasks in Octopus-Bench, highlighting the crucial role of capability coordination in agentic multimodal reasoning.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 57 references that are in the map. First posted on arXiv 19 Nov 2025.
Builds on
46 of 57 references.
Closest ancestors · 13
Same direction, newest first
- 2025TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoning3 Nov
- 2025DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning30 Sep
- 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens20 Jun
- 2025Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification8 Jun
- 2025Thinking with Generated Images28 May
- 2024CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models17 Dec
- 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
- 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
- 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models22 May
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
- 2023ViperGPT: Visual Inference via Python Execution for Reasoning14 Mar
1 more
Other references · 33
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
21 more
- 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
- 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
- 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
- 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 299
- 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 136
- 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 102
- 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
- 2025Visual Planning: Let's Think Only with Imagescited by 85
- 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agentscited by 69
- 2024Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Modelscited by 66
- 2024DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLMcited by 46
- 2024IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representationscited by 38
- 2025GraphMaster: Automated Graph Synthesis via LLM Agents in Data-Limited Environmentscited by 19
- 2025WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generationcited by 16
- 2024MMFactory: A Universal Solution Search Engine for Vision-Language Taskscited by 7
- 2025GraphOracle: Efficient Fully-Inductive Knowledge Graph Reasoning via Relation-Dependency Graphscited by 7
- 2025Mixture of Length and Pruning Experts for Knowledge Graphs Reasoningcited by 5
- 2025How Many Code and Test Cases Are Enough? Evaluating Test Cases Generation from a Binary-Matrix Perspectivecited by 4
- 2025Decoupling Continual Semantic Segmentationcited by 2
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 75
- 2026RealMem: Benchmarking LLMs in Real-World Memory-Driven Interactioncited by 20
- 2026VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chainingcited by 12
- 2026ACE-Router: Generalizing History-Aware Routing from MCP Tools to the Agent Webcited by 7
- 2026TIR-Agent: Training an Explorative and Efficient Agent for Image Restorationcited by 5
- 2026LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledgercited by 5
- 2026Chain of Mindset: Reasoning with Adaptive Cognitive Modescited by 2
- 2026Learning Multimodal Embeddings with Evidence-Aligned Readoutcited by 1
- 2026CytoSPM: Open-Vocabulary Cytopathology Detection with Structured Prompt Bankcited by 0
- 2026Learning Multimodal Embeddings with Evidence-Aligned Readout27 Sep 2026
- 2026CytoSPM: Open-Vocabulary Cytopathology Detection with Structured Prompt Bank25 Sep 2026
- 2026LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger30 Jul 2026
- 2026TIR-Agent: Training an Explorative and Efficient Agent for Image Restoration29 Mar 2026
- 2026VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chaining16 Mar 2026
- 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agents12 Mar 2026
- 2026Chain of Mindset: Reasoning with Adaptive Cognitive Modes10 Feb 2026
- 2026ACE-Router: Generalizing History-Aware Routing from MCP Tools to the Agent Web13 Jan 2026
- 2026RealMem: Benchmarking LLMs in Real-World Memory-Driven Interaction11 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025Agent-ScanKit: Unraveling Memory and Reasoning of Multimodal Agents via Sensitivity Perturbations
- 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models
- 2025Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks
- 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning
- 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoning
- 2024MageBench: Bridging Large Multimodal Models to Agents
- 2025MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models
- 2026A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applications