# Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

arXiv: https://arxiv.org/abs/2606.30616 · HTML: https://mlsanity.com/paper/2606.30616/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)
- Mixture-of-experts / [MoE in reinforcement learning and robotics](https://mlsanity.com/moe/moe-in-reinforcement-learning-and-robotics/)

## Builds on (most cited first, 29 of 29)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024)
- [Generalizing Verifiable Instruction Following](https://mlsanity.com/paper/2507.02833/) (2025)
- [SciCode: A Research Coding Benchmark Curated by Scientists](https://mlsanity.com/paper/2407.13168/) (2024)
- [Scaling Long-Horizon LLM Agent via Context-Folding](https://mlsanity.com/paper/2510.11967/) (2025)
- [Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes](https://mlsanity.com/paper/2603.25562/) (2026)
- [xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations](https://mlsanity.com/paper/2506.13651/) (2025)
- [CWM: An Open-Weights LLM for Research on Code Generation with World Models](https://mlsanity.com/paper/2510.02387/) (2025)
- [SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models](https://mlsanity.com/paper/2506.01062/) (2025)
- [VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications](https://mlsanity.com/paper/2509.26490/) (2025)
- [MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering](https://mlsanity.com/paper/2505.07782/) (2025)
- [MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery](https://mlsanity.com/paper/2606.06473/) (2026)
- [HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark?](https://mlsanity.com/paper/2509.07894/) (2025)
- [MatTools: Benchmarking Large Language Models for Materials Science Tools](https://mlsanity.com/paper/2505.10852/) (2025)
- [PAPO: Stabilizing Rubric Integration Training via Decoupled Advantage Normalization](https://mlsanity.com/paper/2603.26535/) (2026)
- [Towards Long-horizon Agentic Multimodal Search](https://mlsanity.com/paper/2604.12890/) (2026)
- [MolClaw: An Autonomous Agent with Hierarchical Skills for Drug Molecule Evaluation, Screening, and Optimization](https://mlsanity.com/paper/2604.21937/) (2026)
- [Agents-K1: Towards Agent-native Knowledge Orchestration](https://mlsanity.com/paper/2606.13669/) (2026)

## Built on by (most cited first, 22 of 22)

- [Open-MOPD: Diagnosing and Fixing Capability Imbalance in Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2608.19098/) (2026)
- [NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness](https://mlsanity.com/paper/2609.08183/) (2026)
- [The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation](https://mlsanity.com/paper/2607.24720/) (2026)
- [Occamy-1.0: Open Pareto-frontier 35B Intelligence for Co-work](https://mlsanity.com/paper/2609.11977/) (2026)
- [Proxy OPD: On-Policy Distillation with Transferable Relative Proxy Update](https://mlsanity.com/paper/2607.11505/) (2026)
- [AgentOmnia: Scaling Agentic Models for Full-Scenario Applications](https://mlsanity.com/paper/2607.23124/) (2026)
- [GraphRareBench: An Auditable Graph-Evidence Benchmark for Phenotype-Driven Rare-Disease Diagnosis](https://mlsanity.com/paper/2607.24878/) (2026)
- [SciDataSailor: Deep Scientific Data Exploring](https://mlsanity.com/paper/2607.28098/) (2026)
- [SeekBrain: An Autonomous Multi-Agent System for Accelerating Neuroscience Discovery](https://mlsanity.com/paper/2607.29347/) (2026)
- [Towards Physics-Faithful Generation of Scientific Diagrams](https://mlsanity.com/paper/2608.13112/) (2026)
- [Mint-Agent: Introducing Finance-Native Agentic Foundation Models](https://mlsanity.com/paper/2608.16386/) (2026)
- [Iris: Climbing to the Search Frontier](https://mlsanity.com/paper/2609.04304/) (2026)
- [SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving](https://mlsanity.com/paper/2609.27717/) (2026)
- [CompoWorld: Compositional Environment Scaling for General Agents](https://mlsanity.com/paper/2609.33665/) (2026)
- [WorkGenesis: Building the Worlds That Teach Agents to Work](https://mlsanity.com/paper/2609.39325/) (2026)
- [How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?](https://mlsanity.com/paper/2609.40303/) (2026)
- [From Gradients to Capabilities: Understanding Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2610.02179/) (2026)
- [SAIL: Scientific Agentic Intelligence via a Science-Aware Loop](https://mlsanity.com/paper/2610.11451/) (2026)
- [JoyInside-VLM-Agent: Product-Conditioned Multimodal Policy Learning for Embodied Agents](https://mlsanity.com/paper/2610.12957/) (2026)
- [HyWe: Unifying Search and Operation for Real-World Hybrid Web Tasks](https://mlsanity.com/paper/2610.16319/) (2026)
- [Glasseek: A Transparent Recipe for Training and Trustworthy Evaluation of DeepSearch Agents](https://mlsanity.com/paper/2610.16998/) (2026)
- [Cowork-Gym: Synthesizing Interactive Work Environments at Scale](https://mlsanity.com/paper/2610.17670/) (2026)

## Abstract (arXiv)

We introduce Agents-A1, a 35B Mixture-of-Experts Agentic Model that reaches trillion-parameter-level performance by scaling the agent horizon. We investigate agent-horizon scaling from two perspectives: scaling long-horizon trajectories and scaling heterogeneous agent abilities. To support this goal, we build a long-horizon knowledge-action infrastructure that connects external knowledge, actions, observations, and verifier outcomes, producing agentic trajectories with an average length of 45K tokens. Based on this, we train Agents-A1 with a three-stage recipe. First, we perform full-domain supervised fine-tuning to align the base model with broad agentic behaviors. Second, we train domain-level teacher models to capture specialized expertise in each domain. Third, we propose a multi-teacher domain-routed on-policy distillation with salient vocabulary alignment to improve knowledge transfer efficiency across different domains, unifying six heterogeneous domains into one deployable student model. Agents-A1 achieves strong and broad performance for long-horizon agent benchmarks. Compared with 1T-parameter model such as Kimi-K2.6 and DeepSeek-V4-pro, Agents-A1 achieves leading results on SEAL-0 (56.4), IFBench (80.6), HiPhO (46.4), FrontierScience-Olympiad (79.0), and MolBench-Bind (56.8), and remains highly competitive on SciCode (44.3), HLE (47.6) and BrowseComp (75.5). We hope this work provides the community with a practical path for scaling the horizon using a 35B agent that can reach or match the performance of 1T models on long-horizon tasks.