mlsanityData to 7 Oct 2026

OmniGAIA: Towards Native Omni-Modal AI Agents

arXiv 2602.22897 ↗First posted 26 Feb 2026cs.AI cs.CL cs.CVcited by 21

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Human intelligence naturally intertwines omni-modal perception -- spanning vision, audio, and language -- with complex reasoning and tool usage to interact with the world. However, current multi-modal LLMs are primarily confined to bi-modal interactions (e.g., vision-language), lacking the unified cognitive capabilities required for general AI assistants. To bridge this gap, we introduce OmniGAIA, a comprehensive benchmark designed to evaluate omni-modal agents on tasks necessitating deep reasoning and multi-turn tool execution across video, audio, and image modalities. Constructed via a novel omni-modal event graph approach, OmniGAIA synthesizes complex, multi-hop queries derived from real-world data that require cross-modal reasoning and external tool integration. Furthermore, we propose OmniAtlas, a native omni-modal foundation agent under tool-integrated reasoning paradigm with active omni-modal perception. Trained on trajectories synthesized via a hindsight-guided tree exploration strategy and OmniDPO for fine-grained error correction, OmniAtlas effectively enhances the tool-use capabilities of existing open-source models. This work marks a step towards next-generation native omni-modal AI assistants for real-world scenarios.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 64 of its 70 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

64 of 70 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answering3 Feb
  2. 2026Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning11 Jan
  3. 2025Active Perception Agent for Omnimodal Audio-Video Understanding29 Dec
  4. 2025VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understanding13 Dec
  5. 2025Agent-Omni: Test-Time Multimodal Reasoning via Model Coordination for Understanding Anything4 Nov
  6. 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents14 Aug
  7. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory13 Aug
  8. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent7 Aug
  9. 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding23 May
  10. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agent15 Mar

Other references · 54

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  6. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  7. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  8. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  9. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  10. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  11. 2025Qwen2.5-Omni Technical Reportcited by 704
  12. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
38 more
  1. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  2. 2024Qwen2-Audio Technical Reportcited by 543
  3. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  4. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  5. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  6. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
  7. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  8. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 303
  9. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 283
  10. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  11. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 180
  12. 2025ToRL: Scaling Tool-Integrated RLcited by 166
  13. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 147
  14. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 120
  15. 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Modelcited by 101
  16. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 101
  17. 2024OmniBench: Towards The Future of Universal Omni-Language Modelscited by 100
  18. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 69
  19. 2025Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learningcited by 67
  20. 2025In-the-Flow Agentic System Optimization for Effective Planning and Tool Usecited by 67
  21. 2025Ming-Omni: A Unified Multimodal Model for Perception and Generationcited by 62
  22. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMscited by 54
  23. 2024AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?cited by 46
  24. 2026Dr. Zero: Self-Evolving Search Agents without Training Datacited by 39
  25. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 38
  26. 2025MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Usecited by 37
  27. 2025VideoExplorer: Think With Videos For Agentic Long-Video Understandingcited by 29
  28. 2025LongCat-Flash-Omni Technical Reportcited by 23
  29. 2026OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Modelscited by 23
  30. 2025NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matchingcited by 19
  31. 2024OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalitiescited by 13
  32. 2025Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Visioncited by 12
  33. 2025UNO-Bench: A Unified Benchmark for Exploring the Compositional Law Between Uni-modal and Omni-modal in Omni Modelscited by 12
  34. 2025JavisGPT: A Unified Multi-modal LLM for Sounding-Video Comprehension and Generationcited by 12
  35. 2025Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferencescited by 11
  36. 2026PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarioscited by 10
  37. 2026OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attentioncited by 9
  38. 2025HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searchescited by 7

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 21.

21 papers.

Sort:

By area:

  1. 2026Qwen3.5-Omni Technical Reportcited by 146
  2. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 35
  3. 2026MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluationcited by 23
  4. 2026LaSER: Internalizing Explicit Reasoning into Latent Space for Dense Retrievalcited by 8
  5. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  6. 2026CausalRM: Causal-Theoretic Reward Modeling for RLHF from Observational User Feedbackscited by 6
  7. 2026OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Modelscited by 3
  8. 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoningcited by 2
  9. 2026Sandboxed Coding Agents are Competitive Omni-modal Task Solverscited by 2
  10. 2026Orchestra-o1: Omnimodal Agent Orchestrationcited by 2
  11. 2026OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Modelscited by 2
  12. 2026Qwen3.8-Omni: Towards Native Omni-Modal Agentscited by 2
9 more
  1. 2026TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videoscited by 1
  2. 2026Omni-DeepSearch: A Benchmark for Audio-Driven Omni-Modal Deep Searchcited by 1
  3. 2026OmniReasoner: Thinking with Long Audio-Video via Native Tool Usecited by 1
  4. 2026OPOD: On-Policy Omni Distillationcited by 1
  5. 2026Chain of Modality: From Static Fusion to Dynamic Orchestration in Omni-MLLMscited by 0
  6. 2026TOBench: A Task-Oriented Omni-Modal Benchmark for Real-World Tool-Using Agentscited by 0
  7. 2026Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agentscited by 0
  8. 2026Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streamscited by 0
  9. 2026Omni-IO Skills: Harnessing Your Agent Omni-Nativecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
  2. 2025OMHBench: Benchmarking Balanced and Grounded Omni-Modal Multi-Hop Reasoning
  3. 2026ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decoding
  4. 2026Native Active Perception as Reasoning for Omni-Modal Understanding
  5. 2025OmniGuard: Unified Omni-Modal Guardrails with Deliberate Reasoning
  6. 2025Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception
  7. 2025A Benchmark for Omni-Modal Reasoning in Long Videos
  8. 2025OmniPlay: Benchmarking Omni-Modal Models on Omni-Modal Game Playing