mlsanityData to 7 Oct 2026

MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning

arXiv 2607.07391 ↗First posted 8 Jul 2026cs.AIcited by 0

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Mathematical reasoning benchmarks typically provide all facts needed to solve each problem, while interactive benchmarks often mix reasoning with tools, retrieval, and long-horizon dialogue. We introduce MIRA-Math, a benchmark for a narrower diagnostic capability: solving mathematical problems whose full latent state has a unique answer, but whose solver-facing view is missing exactly one necessary atomic fact. The solver must request the missing information in natural language under a strict budget and then integrate the returned fact into an exact final answer. A fixed constrained LLM responder sees only the dataset-provided atomic fact and must either offer the quoted fact when the request matches it, or decline otherwise. Thus, instance generation, typed hint specifications, validation, and final-answer verification are deterministic, while request metrics are measured under a fixed LLM-mediated responder channel. MIRA-Math contains 2{,}310 generated instances from 22 typed mathematical families spanning algebra, probability, linear systems, discrete structures, signal processing, Markov chains, circuits, interpolation, and numerical boundary-value problems. Experiments across frontier and small models show that request success and final-answer accuracy are separable: models may ask for the right fact yet fail the downstream computation, or fail before obtaining the canonical hint. We release generators, verifiers, prompts, run metadata, and dataset documentation to support reproducible evaluation of minimal information requesting in mathematical reasoning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 30 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

16 of 30 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning Models to Ask for Information15 Aug
  2. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitions29 May

Other references · 14

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  6. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  8. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  9. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  10. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  11. 2016Learning Multiagent Communication with Backpropagationcited by 355
  12. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 56
2 more
  1. 2024ClarQ-LLM: A Benchmark for Models Clarifying and Requesting Information in Task-Oriented Dialogcited by 20
  2. 2026When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarificationcited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
  2. 2025ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Models
  3. 2025Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs
  4. 2025HERMES: Towards Efficient and Verifiable Mathematical Reasoning in LLMs
  5. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  6. 2026EternalMath: A Living Benchmark of Frontier Mathematics that Evolves with Human Discovery
  7. 2026More Accurate or More Efficient? Evaluating Locally Deployed Compact Open-Weight Language Models for Mathematical Reasoning
  8. 2026Rethinking Math Reasoning Evaluation: A Robust LLM-as-a-Judge Framework Beyond Symbolic Rigidity