mlsanityData to 14 Oct 2026

When Verification Hurts: Asymmetric Effects of Multi-Agent Feedback in Logic Proof Tutoring

arXiv 2603.27076 ↗First posted 28 Mar 2026cs.AIcited by 2

On the map: LLM agents and tool use → Educational agents and math problem solving

Abstract (arXiv)

Large language models (LLMs) are increasingly used for automated tutoring, but their reliability in structured symbolic domains remains unclear. We study step-level feedback for propositional logic proofs, which require precise symbolic reasoning aligned with a learner's current proof state. We introduce a knowledge-graph-grounded benchmark of 516 unique proof states with step-level annotations and difficulty metrics. Unlike prior tutoring evaluations that rely on model self-assessment or binary correctness, our framework enables fine-grained analysis of feedback quality against verified solution paths. We evaluate three role-specialized pipelines with varying solution access: Tutor (partial solution access), Teacher (full derivation access), and Judge (verification of Tutor feedback). Our results reveal a striking asymmetry: verification improves outcomes when upstream feedback is error-prone (<70% accuracy), but degrades performance by 4-6 percentage points through over-specification when feedback is already reliable (>85%). Critically, we identify a shared complexity ceiling; no model or pipeline reliably succeeds on proof states exceeding complexity 4-5. These findings challenge the assumption that adding verifiers or richer context universally improves tutoring, motivating adaptive, difficulty-aware architectures that route problems by estimated complexity and upstream reliability.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use. Builds on 21 of its 41 references that are in the map. First posted on arXiv 28 Mar 2026.

Builds on

21 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025FACET: Teacher-Centred LLM-Based Multi-Agent Systems-Towards Personalized Educational Worksheets15 Aug
  2. 2025EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Framework21 Apr
  3. 2025EduPlanner: LLM-Based Multi-Agent Systems for Customized and Intelligent Instructional Design7 Apr

Other references · 18

  1. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  2. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 337
  3. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 324
  4. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  5. 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 131
  6. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 86
  7. 2025MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutorscited by 49
  8. 2024Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutorscited by 41
  9. 2025Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluationcited by 30
  10. 2023Automating Human Tutor-Style Programming Feedback: Leveraging GPT-4 Tutor Model for Hint Generation and GPT-3.5 Student Model for Hint Validationcited by 29
  11. 2024LLMs can Find Mathematical Reasoning Mistakes by Pedagogical Chain-of-Thoughtcited by 28
  12. 2024Supporting Self-Reflection at Scale with Large Language Models: Insights from Randomized Field Experiments in Classroomscited by 20
6 more
  1. 2020Automated Personalized Feedback Improves Learning Gains in an Intelligent Tutoring Systemcited by 16
  2. 2025LLMs cannot spot math errors, even when allowed to peek into the solutioncited by 7
  3. 2025AI-Powered Math Tutoring: Platform for Personalized and Adaptive Educationcited by 6
  4. 2025A Comprehensive Review of AI-based Intelligent Tutoring Systems: Applications and Challengescited by 6
  5. 2025Partnering with AI: A Pedagogical Feedback System for LLM Integration into Programming Educationcited by 5
  6. 2025LogicLearner: A Tool for the Guided Practice of Propositional Logic Proofscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experience
  2. 2026Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification
  3. 2025CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization
  4. 2026LEAP: Supercharging LLMs for Formal Mathematics with Agentic Frameworks
  5. 2025Theorem Prover as a Judge for Synthetic Data Generation
  6. 2025Hilbert: Recursively Building Formal Proofs with Informal Reasoning
  7. 2025ProofNet++: A Neuro-Symbolic System for Formal Proof Verification with Self-Correction
  8. 2026AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification