# Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agents

arXiv: https://arxiv.org/abs/2607.11433 · HTML: https://mlsanity.com/paper/2607.11433/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 32 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Builds on (most cited first, 30 of 30)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs](https://mlsanity.com/paper/2502.04326/) (2025)
- [ACON: Optimizing Context Compression for Long-horizon LLM Agents](https://mlsanity.com/paper/2510.00615/) (2025)
- [Where LLM Agents Fail and How They can Learn From Failures](https://mlsanity.com/paper/2509.25370/) (2025)
- [VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos](https://mlsanity.com/paper/2502.01549/) (2025)
- [AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress](https://mlsanity.com/paper/2511.08325/) (2025)
- [LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling](https://mlsanity.com/paper/2511.20785/) (2025)
- [The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management](https://mlsanity.com/paper/2508.21433/) (2025)
- [VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning](https://mlsanity.com/paper/2503.13444/) (2025)
- [Stop Comparing LLM Agents Without Disclosing the Harness](https://mlsanity.com/paper/2605.23950/) (2026)
- [LongVideoAgent: Multi-Agent Reasoning with Long Videos](https://mlsanity.com/paper/2512.20618/) (2025)
- [DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems](https://mlsanity.com/paper/2512.06749/) (2025)
- [OmniGAIA: Towards Native Omni-Modal AI Agents](https://mlsanity.com/paper/2602.22897/) (2026)
- [MMCTAgent: Multi-modal Critical Thinking Agent Framework for Complex Visual Reasoning](https://mlsanity.com/paper/2405.18358/) (2024)
- [Active Perception Agent for Omnimodal Audio-Video Understanding](https://mlsanity.com/paper/2512.23646/) (2025)
- [VerifiAgent: a Unified Verification Agent in Language Model Reasoning](https://mlsanity.com/paper/2504.00406/) (2025)
- [VITED: Video Temporal Evidence Distillation](https://mlsanity.com/paper/2503.12855/) (2025)
- [OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answering](https://mlsanity.com/paper/2602.03707/) (2026)
- [LongVidSearch: An Agentic Benchmark for Multi-hop Evidence Retrieval Planning in Long Videos](https://mlsanity.com/paper/2603.14468/) (2026)
- [Native Active Perception as Reasoning for Omni-Modal Understanding](https://mlsanity.com/paper/2606.19341/) (2026)
- [Sandboxed Coding Agents are Competitive Omni-modal Task Solvers](https://mlsanity.com/paper/2606.00579/) (2026)
- [Orchestra-o1: Omnimodal Agent Orchestration](https://mlsanity.com/paper/2606.13707/) (2026)

## Abstract (arXiv)

Omni-modal agents must seek evidence across video, audio, web pages, and computation to answer questions. Their main bottleneck is planning: noisy multimodal observations accumulate in conversation history and disrupt later decisions, while multimodal models have limited capacity for multi-step planning. Controlled backend replacements support this diagnosis: replacing the planner causes a much larger performance loss than replacing the perception backend. We present Omni-Decision, an omni-modal agent built on evidence-ledger planning: it replaces the growing dialogue history with an explicit evidence ledger that records what evidence is still missing, what has been confirmed, and where records conflict. A critic reads each noisy observation and passes only the usable content to the ledger, discarding the rest, so the planner works from a compact context throughout the task. Each run records the state, action, and verdict at every step, and supervised fine-tuning and decision-level reinforcement learning on these trajectories further improve the planner. Omni-Decision achieves state-of-the-art accuracy of 81.4% on OmniGAIA at approximately 43% of Gemini-3.1-Pro's cost per question, and 65.0% on WorldSense long-video understanding, level with the strongest end-to-end model.