AgentOmnia: Scaling Agentic Models for Full-Scenario Applications
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Large language model agents have advanced rapidly, yet progress remains fragmented across domains, capabilities, task difficulty, and interaction settings. We frame this as full-scenario agentic scaling and present AgentOmnia, a framework coordinating task-space definition, data synthesis, post-training, evaluation, and improvement across To-Consumer (ToC), To-Business (ToB), and To-Employee (ToE) applications. An extensible Domain x Capability x Atomic Difficulty taxonomy aligns these stages and enables fine-grained diagnosis with OmniaBench. AgentOmnia combines bidirectional environment-task synthesis with tool-dependency, program-structured, and solver-based pipelines, constructing 5,018 stateful environments with 255,375 tools and 52,361 tasks. Programs, solvers, and verifiers provide correctness signals, while supervised fine-tuning, online agentic reinforcement learning, and a rollback curriculum support post-training. Evaluation failures translate into Product Requirement Documents (PRDs) for targeted self-evolution. Starting from Qwen3-30B-A3B-Thinking-2507, AgentOmnia raises the pass rate on the OmniaBench challenging subset from 9.16% to 37.11% and the macro-average across OmniaBench, τ2-Bench, DeepPlanning, and VitaBench from 22.86% to 41.69%. Under a unified protocol,it leads the evaluated agentic post-trained baselines on OmniaBench and retains the highest four-benchmark macro-average. It also surpasses Qwen3-235B-A22B-Thinking-2507 on all four benchmarks and exceeds Qwen3.5-35B-A3B on the macro-average. Gains span three application splits, ten capability dimensions, eight atomic-difficulty factors, and 76 of 90 level-1 domains, indicating broad rather than category-specific improvement. A one-round study provides initial evidence for PRD-guided self-evolution, motivating validation at larger scales and in industrial settings.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 77 of its 119 references that are in the map. First posted on arXiv 25 Jul 2026.
Builds on
77 of 119 references.
Closest ancestors · 21
Same direction, newest first
- 2026OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios16 Jul
- 2026Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent29 Jun
- 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environments23 May
- 2026Hindsight Hint Distillation: Scaffolded Reasoning for SWE Agents from CoT-free Answers12 May
- 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence20 Apr
- 2026Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments12 Feb
- 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning10 Feb
- 2026AgentSkiller: Scaling Generalist Agent Intelligence through Semantically Integrated Cross-Domain Data Synthesis10 Feb
- 2026DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints26 Jan
- 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators22 Dec
- 2025Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction4 Dec
- 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning20 Nov
9 more
- 2025AgentEvolver: Towards Efficient Self-Evolving Agent System13 Nov
- 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution29 Oct
- 2025VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications30 Sep
- 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents17 Jun
- 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments4 Nov
- 2024OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation26 Jul
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
- 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environments6 Jun
- 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
Other references · 56
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3 Technical Reportcited by 8,195
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2025Group Sequence Policy Optimizationcited by 788
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
38 more
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
- 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
- 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 385
- 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 383
- 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
- 2025ToolRL: Reward is All Tool Learning Needscited by 342
- 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 248
- 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 142
- 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performancecited by 102
- 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 100
- 2025Soft Adaptive Policy Optimizationcited by 89
- 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 83
- 2025Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routerscited by 67
- 2025Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMscited by 61
- 2025TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environmentscited by 61
- 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 46
- 2024VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Taskscited by 45
- 2023ToolTalk: Evaluating Tool-Usage in a Conversational Settingcited by 42
- 2025MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool usecited by 30
- 2025Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystemcited by 28
- 2026Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learningcited by 26
- 2023SELF: Self-Evolution with Language Feedbackcited by 24
- 2026ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Trainingcited by 23
- 2026Composer 2 Technical Reportcited by 20
- 2026From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Modelscited by 19
- 2026From Self-Evolving Synthetic Data to Verifiable-Reward RL: Post-Training Multi-turn Interactive Tool-Using Agentscited by 17
- 2025AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learningcited by 14
- 2026ECHO: Terminal Agents Learn World Models for Freecited by 13
- 2025Data Mixing Optimization for Supervised Fine-Tuning of Large Language Modelscited by 12
- 2026EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RLcited by 11
- 2025Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playingcited by 9
- 2026DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Usecited by 9
- 2026Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learningcited by 9
50 most cited of 56 All 56 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026The Hierarchy of Agentic Capabilities: Evaluating Frontier Models on Realistic RL Environments
- 2025Towards a Science of Scaling Agent Systems
- 2026Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback
- 2026OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions
- 2026AgentFactory: Towards Automated Agentic System Design and Optimization
- 2025Towards General Agentic Intelligence via Environment Scaling
- 2026Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
- 2026T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains