ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search
On the map: LLM agents and tool use → Educational agents and math problem solving
Abstract (arXiv)
In this work, we present ZGCM-1, a fully open 7B dense foundation model trained from scratch with extreme data, system, and algorithmic efficiency. ZGCM-1 is founded on a core premise: compact models cannot passively memorize the open web, but can overcome parametric capacity limits by coupling deliberate internal thinking with active external tool use. To support this paradigm across a 256K context, we develop an end-to-end, high-efficiency open training recipe: Architecture & System Co-design: interleaved gated sliding-window and full attention, and a stable FP8 Muon optimizer; Progressive Curriculum & MDP Mid-Training: context scaling across 16K, 64K, and 256K, and the reformulation of interaction traces into Markov Decision Processes. Furthermore, we establish an AI-native R&D workflow where agent swarms autonomously manage cluster operations, data curation, and rapid diagnostic evaluation. Extensive evaluations show that ZGCM-1-7B is competitive across 7B model family on general benchmarks. On several challenging mathematical reasoning and agentic search suites, it remains competitive with frontier models orders of magnitude larger, such as Qwen3-235B-A22B and GLM-5.1. We also show that our pre-training design offers a ~4.2x efficiency improvement in 16K pre-training time-to-loss. Across the full development lifecycle, we distill eight actionable empirical findings-spanning architectural scaling, SFT quality pruning, long-context generalization, and agentic co-training dynamics. To facilitate community research, we open-source model weights from the pre-training, mid-training, and post-training stages, intermediate checkpoints, training code, per-stage data and data recipes, and W&B logs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use. Builds on 52 of its 97 references that are in the map. First posted on arXiv 11 Sep 2026.
Builds on
52 of 97 references.
References · 52
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2023Let's Verify Step by Stepcited by 3,789
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
38 more
- 2025Gemma 3 Technical Reportcited by 1,824
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2020GLU Variants Improve Transformercited by 1,804
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2019Generating Long Sequences with Sparse Transformerscited by 1,546
- 2019Root Mean Square Layer Normalizationcited by 1,480
- 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
- 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
- 2021Deduplicating Training Data Makes Language Models Bettercited by 714
- 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
- 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2025Muon is Scalable for LLM Trainingcited by 428
- 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 391
- 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
- 2022FP8 Formats for Deep Learningcited by 196
- 2025Generalizing Verifiable Instruction Followingcited by 194
- 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
- 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 177
- 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 175
- 2025Qwen2.5-1M Technical Reportcited by 164
- 2026Kimi K3: Open Frontier Intelligencecited by 138
- 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 125
- 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 121
- 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 89
- 2025Towards Robust Mathematical Reasoningcited by 73
- 2025Scaling Agents via Continual Pre-trainingcited by 40
- 2025Ultra-FineWeb: Efficient Data Filtering and Verification for High-Quality LLM Training Datacited by 38
- 2024HARDMath: A Benchmark Dataset for Challenging Problems in Applied Mathematicscited by 32
50 most cited of 52 All 52 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ResearchMath-14K: Scaling Research-Level Mathematics via Agents
- 2025SIGMA: Search-Augmented On-Demand Knowledge Integration for Agentic Mathematical Reasoning
- 2026Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search
- 2026Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model
- 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents
- 2026Iteris: Agentic Research Loops for Computational Mathematics
- 2026Advancing Mathematics Research with AI-Driven Formal Proof Search
- 2023Modeling Complex Mathematical Reasoning via Large Language Model based MathAgent