mlsanityData to 14 Oct 2026

More Agents Improve Math Problem Solving but Adversarial Robustness Gap Persists

arXiv 2511.07112 ↗First posted 10 Nov 2025cs.CL cs.AIcited by 1

On the map: LLM agents and tool use → Educational agents and math problem solving

Abstract (arXiv)

When LLM agents work together, they seem to be more powerful than a single LLM in mathematical question answering. However, are they also more robust to adversarial inputs? We investigate this question using adversarially perturbed math questions. These perturbations include punctuation noise with three intensities (10%, 30%, 50%), plus real-world and human-like typos (WikiTypo, R2ATA). Using a unified sampling-and-voting framework (Agent Forest), we evaluate six open-source models (Qwen3-4B/14B, Llama3.1-8B, Mistral-7B, Gemma3-4B/12B) across four benchmarks (GSM8K, MATH, MMLU-Math, MultiArith), with various numbers of agents n = 1,2,5,10,15,20,25. Our findings show that 1) Noise type matters: punctuation noise harm scales with its severity, and the human typos remain the dominant bottleneck, yielding the largest gaps to Clean accuracy and the highest attack success rate (ASR) even with a large number of agents; 2) Collaboration reliably improves accuracy as the number of agents, n, increases, with the largest gains from n=1 to n=5 and diminishing returns beyond n≈10. However, the adversarial robustness gap persists regardless of the agent count.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use. Builds on 25 of its 34 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

25 of 34 references.

References · 25

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  5. 2025Gemma 3 Technical Reportcited by 1,824
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  7. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  8. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
  9. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
  10. 2016Solving General Arithmetic Word Problemscited by 360
  11. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 345
  12. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 333
13 more
  1. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  2. 2024Improving Multi-Agent Debate with Sparse Communication Topologycited by 107
  3. 2021Evaluating the Robustness of Neural Language Models to Input Perturbationscited by 87
  4. 2024TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generationcited by 51
  5. 2020From Hero to Z'eroe: A Benchmark of Low-Level Adversarial Attackscited by 36
  6. 2023Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Taskcited by 23
  7. 2025MALLM: Multi-Agent Large Language Models Frameworkcited by 15
  8. 2025ArithmAttack: Evaluating Robustness of LLMs to Noisy Context in Math Problem Solvingcited by 13
  9. 2025SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Sciencecited by 7
  10. 2025RedDebate: Safer Responses Through Multi-Agent Red Teaming Debatescited by 7
  11. 2025Exploring Robustness of Multilingual LLMs on Real-World Noisy Datacited by 6
  12. 2025Modeling Response Consistency in Multi-Agent LLM Systems: A Comparative Analysis of Shared and Separate Context Approachescited by 3
  13. 2025Exploring Robustness of LLMs to Paraphrasing Based on Sociodemographic Factorscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Robust Reasoning Benchmarkcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attacks
  2. 2023Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5?
  3. 2024GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers
  4. 2025RIDE: Difficulty Evolving Perturbation with Item Response Theory for Mathematical Reasoning
  5. 2025MSCR: Exploring the Vulnerability of LLMs' Mathematical Reasoning Abilities Using Multi-Source Candidate Replacement
  6. 2025Toward Automated Robustness Evaluation of Mathematical Reasoning
  7. 2026MathDuels: A Self-Play Benchmark That Grows
  8. 2025Numerical Sensitivity and Robustness: Exploring the Flaws of Mathematical Reasoning in Large Language Models