mlsanityData to 7 Oct 2026

AlphaApollo: A System for Deep Agentic Reasoning

arXiv 2510.06261 ↗First posted 5 Oct 2025cs.AI cs.CL cs.LGcited by 5

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

We present AlphaApollo, an agentic reasoning system that targets two bottlenecks in foundation-model reasoning: (1) limited reasoning capacity for complex, long-horizon problem solving and (2) unreliable test-time evolution without trustworthy verification. AlphaApollo orchestrates models and tools via three components: (i) multi-turn agentic reasoning, which formalizes model-environment interaction with structured tool calls and responses; (ii) multi-turn agentic learning, which applies turn-level reinforcement learning to optimize tool-use reasoning while decoupling actions from tool responses for stable training; and (iii) multi-round agentic evolution, which refines solutions through a propose-judge-update loop with tool-assisted verifications and long-horizon memory. Across seven math reasoning benchmarks and multiple model scales, AlphaApollo improves performance through reliable tool use (> 85% tool-call success), substantial gains from multi-turn RL (Avg@32: Qwen2.5-1.5B-Instruct 1.07% -> 9.64%, Qwen2.5-7B-Instruct 8.77% -> 20.35%), and improvements from evolution (e.g., Qwen2.5-3B-Instruct 5.27% -> 7.70%, Qwen2.5-14B-Instruct 16.53% -> 21.08%). This project is still ongoing. We welcome feedback from the community and will frequently update the source code and technical report.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 68 of its 82 references that are in the map. First posted on arXiv 5 Oct 2025.

Builds on

68 of 82 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution26 May
  2. 2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning16 Feb
  3. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improve25 Jul
  4. 2024Improving Multi-Agent Debate with Sparse Communication Topology17 Jun
  5. 2024Small LLMs Are Weak Tool Learners: A Multi-LLM Agent14 Jan
  6. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs19 Oct
  7. 2023FireAct: Toward Language Agent Fine-tuning9 Oct
  8. 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models5 Oct
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug

Other references · 59

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2025Qwen2.5-VL Technical Reportcited by 5,418
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2024Qwen2.5 Technical Reportcited by 4,713
  11. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
38 more
  1. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  10. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  11. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  12. 2025s1: Simple test-time scalingcited by 1,311
  13. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  14. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  15. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  16. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  17. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  18. 2023Teaching Large Language Models to Self-Debugcited by 963
  19. 2023Large Language Models as Optimizerscited by 830
  20. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  21. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  22. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  23. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  24. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 442
  25. 2023Language Models can Solve Computer Taskscited by 371
  26. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  27. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  28. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 188
  29. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 171
  30. 2025Hierarchical Reasoning Modelcited by 167
  31. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 156
  32. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoningcited by 145
  33. 2025LIMR: Less is More for RL Scalingcited by 136
  34. 2025Confidence Improves Self-Consistency in LLMscited by 111
  35. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
  36. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 90
  37. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  38. 2023Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculumcited by 64

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learningcited by 4
  2. 2026Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMscited by 4
  3. 2026What Do Evolutionary Coding Agents Evolve?cited by 3
  4. 2026The Easy, the Hard, and the Learnable: Confidence and Difficulty-Adaptive Policy Optimization for LLM Reasoningcited by 0
  5. 2026DICE: Entropy-Regularized Equilibrium Selection for Stable Multi-Agent LLM Coordinationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning
  2. 2025AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
  3. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents
  4. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsets
  5. 2026Don't Overthink, Don't Underthink: Toward Adaptive Reasoning in Agentic AI
  6. 2025AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent
  7. 2026M2A: Synergizing Mathematical and Agentic Reasoning in Large Language Models
  8. 2026Scaling the Scaling Logic: Agentic Meta-Synthesis of Logic Reasoning