mlsanityData to 7 Oct 2026

AgentOmnia: Scaling Agentic Models for Full-Scenario Applications

arXiv 2607.23124 ↗First posted 25 Jul 2026cs.AI cs.CLcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large language model agents have advanced rapidly, yet progress remains fragmented across domains, capabilities, task difficulty, and interaction settings. We frame this as full-scenario agentic scaling and present AgentOmnia, a framework coordinating task-space definition, data synthesis, post-training, evaluation, and improvement across To-Consumer (ToC), To-Business (ToB), and To-Employee (ToE) applications. An extensible Domain x Capability x Atomic Difficulty taxonomy aligns these stages and enables fine-grained diagnosis with OmniaBench. AgentOmnia combines bidirectional environment-task synthesis with tool-dependency, program-structured, and solver-based pipelines, constructing 5,018 stateful environments with 255,375 tools and 52,361 tasks. Programs, solvers, and verifiers provide correctness signals, while supervised fine-tuning, online agentic reinforcement learning, and a rollback curriculum support post-training. Evaluation failures translate into Product Requirement Documents (PRDs) for targeted self-evolution. Starting from Qwen3-30B-A3B-Thinking-2507, AgentOmnia raises the pass rate on the OmniaBench challenging subset from 9.16% to 37.11% and the macro-average across OmniaBench, τ2-Bench, DeepPlanning, and VitaBench from 22.86% to 41.69%. Under a unified protocol,it leads the evaluated agentic post-trained baselines on OmniaBench and retains the highest four-benchmark macro-average. It also surpasses Qwen3-235B-A22B-Thinking-2507 on all four benchmarks and exceeds Qwen3.5-35B-A3B on the macro-average. Gains span three application splits, ten capability dimensions, eight atomic-difficulty factors, and 76 of 90 level-1 domains, indicating broad rather than category-specific improvement. A one-round study provides initial evidence for PRD-guided self-evolution, motivating validation at larger scales and in industrial settings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 77 of its 119 references that are in the map. First posted on arXiv 25 Jul 2026.

Builds on

77 of 119 references.

Closest ancestors · 21

Same direction, newest first

  1. 2026OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios16 Jul
  2. 2026Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent29 Jun
  3. 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environments23 May
  4. 2026Hindsight Hint Distillation: Scaffolded Reasoning for SWE Agents from CoT-free Answers12 May
  5. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence20 Apr
  6. 2026Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments12 Feb
  7. 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning10 Feb
  8. 2026AgentSkiller: Scaling Generalist Agent Intelligence through Semantically Integrated Cross-Domain Data Synthesis10 Feb
  9. 2026DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints26 Jan
  10. 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators22 Dec
  11. 2025Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction4 Dec
  12. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning20 Nov
9 more
  1. 2025AgentEvolver: Towards Efficient Self-Evolving Agent System13 Nov
  2. 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution29 Oct
  3. 2025VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications30 Sep
  4. 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents17 Jun
  5. 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments4 Nov
  6. 2024OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation26 Jul
  7. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
  8. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environments6 Jun
  9. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep

Other references · 56

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  7. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  9. 2025Group Sequence Policy Optimizationcited by 788
  10. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  11. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  12. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
38 more
  1. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
  2. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  3. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  4. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
  5. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 385
  6. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 383
  7. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
  8. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
  9. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
  10. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  11. 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 248
  12. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  13. 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
  14. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performancecited by 102
  15. 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 100
  16. 2025Soft Adaptive Policy Optimizationcited by 89
  17. 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 83
  18. 2025Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routerscited by 67
  19. 2025Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMscited by 61
  20. 2025TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environmentscited by 61
  21. 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 46
  22. 2024VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Taskscited by 45
  23. 2023ToolTalk: Evaluating Tool-Usage in a Conversational Settingcited by 42
  24. 2025MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool usecited by 30
  25. 2025Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystemcited by 28
  26. 2026Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learningcited by 26
  27. 2023SELF: Self-Evolution with Language Feedbackcited by 24
  28. 2026ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Trainingcited by 23
  29. 2026Composer 2 Technical Reportcited by 20
  30. 2026From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Modelscited by 19
  31. 2026From Self-Evolving Synthetic Data to Verifiable-Reward RL: Post-Training Multi-turn Interactive Tool-Using Agentscited by 17
  32. 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learningcited by 14
  33. 2026ECHO: Terminal Agents Learn World Models for Freecited by 13
  34. 2025Data Mixing Optimization for Supervised Fine-Tuning of Large Language Modelscited by 12
  35. 2026EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RLcited by 11
  36. 2025Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playingcited by 9
  37. 2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Usecited by 9
  38. 2026Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learningcited by 9

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Hierarchy of Agentic Capabilities: Evaluating Frontier Models on Realistic RL Environments
  2. 2025Towards a Science of Scaling Agent Systems
  3. 2026Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback
  4. 2026OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions
  5. 2026AgentFactory: Towards Automated Agentic System Design and Optimization
  6. 2025Towards General Agentic Intelligence via Environment Scaling
  7. 2026Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
  8. 2026T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains