mlsanityData to 14 Oct 2026

ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search

arXiv 2609.13356 ↗First posted 11 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → Educational agents and math problem solving

Abstract (arXiv)

In this work, we present ZGCM-1, a fully open 7B dense foundation model trained from scratch with extreme data, system, and algorithmic efficiency. ZGCM-1 is founded on a core premise: compact models cannot passively memorize the open web, but can overcome parametric capacity limits by coupling deliberate internal thinking with active external tool use. To support this paradigm across a 256K context, we develop an end-to-end, high-efficiency open training recipe: Architecture & System Co-design: interleaved gated sliding-window and full attention, and a stable FP8 Muon optimizer; Progressive Curriculum & MDP Mid-Training: context scaling across 16K, 64K, and 256K, and the reformulation of interaction traces into Markov Decision Processes. Furthermore, we establish an AI-native R&D workflow where agent swarms autonomously manage cluster operations, data curation, and rapid diagnostic evaluation. Extensive evaluations show that ZGCM-1-7B is competitive across 7B model family on general benchmarks. On several challenging mathematical reasoning and agentic search suites, it remains competitive with frontier models orders of magnitude larger, such as Qwen3-235B-A22B and GLM-5.1. We also show that our pre-training design offers a ~4.2x efficiency improvement in 16K pre-training time-to-loss. Across the full development lifecycle, we distill eight actionable empirical findings-spanning architectural scaling, SFT quality pruning, long-context generalization, and agentic co-training dynamics. To facilitate community research, we open-source model weights from the pre-training, mid-training, and post-training stages, intermediate checkpoints, training code, per-stage data and data recipes, and W&B logs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use. Builds on 52 of its 97 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

52 of 97 references.

References · 52

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2020Longformer: The Long-Document Transformercited by 3,413
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  10. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  11. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  12. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
38 more
  1. 2025Gemma 3 Technical Reportcited by 1,824
  2. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  3. 2020GLU Variants Improve Transformercited by 1,804
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  5. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  6. 2019Root Mean Square Layer Normalizationcited by 1,480
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  8. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  9. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  10. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  11. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  12. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  13. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  14. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  15. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  16. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  17. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  18. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  19. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  20. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  21. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  22. 2025Muon is Scalable for LLM Trainingcited by 428
  23. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 391
  24. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
  25. 2022FP8 Formats for Deep Learningcited by 196
  26. 2025Generalizing Verifiable Instruction Followingcited by 194
  27. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
  28. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 177
  29. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 175
  30. 2025Qwen2.5-1M Technical Reportcited by 164
  31. 2026Kimi K3: Open Frontier Intelligencecited by 138
  32. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 125
  33. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 121
  34. 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 89
  35. 2025Towards Robust Mathematical Reasoningcited by 73
  36. 2025Scaling Agents via Continual Pre-trainingcited by 40
  37. 2025Ultra-FineWeb: Efficient Data Filtering and Verification for High-Quality LLM Training Datacited by 38
  38. 2024HARDMath: A Benchmark Dataset for Challenging Problems in Applied Mathematicscited by 32

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ResearchMath-14K: Scaling Research-Level Mathematics via Agents
  2. 2025SIGMA: Search-Augmented On-Demand Knowledge Integration for Agentic Mathematical Reasoning
  3. 2026Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search
  4. 2026Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model
  5. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents
  6. 2026Iteris: Agentic Research Loops for Computational Mathematics
  7. 2026Advancing Mathematics Research with AI-Driven Formal Proof Search
  8. 2023Modeling Complex Mathematical Reasoning via Large Language Model based MathAgent