mlsanityData to 14 Oct 2026

Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agents

arXiv 2607.11433 ↗First posted 13 Jul 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Omni-modal agents must seek evidence across video, audio, web pages, and computation to answer questions. Their main bottleneck is planning: noisy multimodal observations accumulate in conversation history and disrupt later decisions, while multimodal models have limited capacity for multi-step planning. Controlled backend replacements support this diagnosis: replacing the planner causes a much larger performance loss than replacing the perception backend. We present Omni-Decision, an omni-modal agent built on evidence-ledger planning: it replaces the growing dialogue history with an explicit evidence ledger that records what evidence is still missing, what has been confirmed, and where records conflict. A critic reads each noisy observation and passes only the usable content to the ledger, discarding the rest, so the planner works from a compact context throughout the task. Each run records the state, action, and verdict at every step, and supervised fine-tuning and decision-level reinforcement learning on these trajectories further improve the planner. Omni-Decision achieves state-of-the-art accuracy of 81.4% on OmniGAIA at approximately 43% of Gemini-3.1-Pro's cost per question, and 65.0% on WorldSense long-video understanding, level with the strongest end-to-end model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 32 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

30 of 32 references.

References · 30

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  6. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  7. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  10. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 155
  11. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 136
  12. 2025Where LLM Agents Fail and How They can Learn From Failurescited by 133
18 more
  1. 2025VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videoscited by 77
  2. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 77
  3. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 61
  4. 2025The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Managementcited by 48
  5. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 43
  6. 2026Stop Comparing LLM Agents Without Disclosing the Harnesscited by 34
  7. 2025LongVideoAgent: Multi-Agent Reasoning with Long Videoscited by 25
  8. 2025DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systemscited by 24
  9. 2026OmniGAIA: Towards Native Omni-Modal AI Agentscited by 22
  10. 2024MMCTAgent: Multi-modal Critical Thinking Agent Framework for Complex Visual Reasoningcited by 13
  11. 2025Active Perception Agent for Omnimodal Audio-Video Understandingcited by 13
  12. 2025VerifiAgent: a Unified Verification Agent in Language Model Reasoningcited by 10
  13. 2025VITED: Video Temporal Evidence Distillationcited by 5
  14. 2026OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answeringcited by 5
  15. 2026LongVidSearch: An Agentic Benchmark for Multi-hop Evidence Retrieval Planning in Long Videoscited by 4
  16. 2026Native Active Perception as Reasoning for Omni-Modal Understandingcited by 4
  17. 2026Sandboxed Coding Agents are Competitive Omni-modal Task Solverscited by 3
  18. 2026Orchestra-o1: Omnimodal Agent Orchestrationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
  2. 2025Omni-AutoThink: Adaptive Multimodal Reasoning via Reinforcement Learning
  3. 2026OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reordering
  4. 2025Agent-Omni: Test-Time Multimodal Reasoning via Model Coordination for Understanding Anything
  5. 2026OmniReasoner: Thinking with Long Audio-Video via Native Tool Use
  6. 2025A Benchmark for Omni-Modal Reasoning in Long Videos
  7. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaboration
  8. 2026Omni-Perception Policy Optimization for Multimodal Emotion Reasoning