# A Multi-Agent Approach to Validate and Refine LLM-Generated Personalized Math Problems

arXiv: https://arxiv.org/abs/2604.05160 · HTML: https://mlsanity.com/paper/2604.05160/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Placed in

- LLM agents and tool use / [Educational agents and math problem solving](https://mlsanity.com/agents/educational-agents-and-math-problem-solving/)

## Builds on (most cited first, 11 of 11)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction](https://mlsanity.com/paper/2506.17784/) (2025)
- [Improving the Validity of Automatically Generated Feedback via Reinforcement Learning](https://mlsanity.com/paper/2403.01304/) (2024)
- [Traceability and Accountability in Role-Specialized Multi-Agent LLM Pipelines](https://mlsanity.com/paper/2510.07614/) (2025)
- [MATHWELL: Generating Educational Math Word Problems Using Teacher Annotations](https://mlsanity.com/paper/2402.15861/) (2024)
- [Content Knowledge Identification with Multi-Agent Large Language Models (LLMs)](https://mlsanity.com/paper/2404.07960/) (2024)
- [Multi-Agent Collaborative Framework For Math Problem Generation](https://mlsanity.com/paper/2511.03958/) (2025)
- [Classroom AI: Large Language Models as Grade-Specific Teachers](https://mlsanity.com/paper/2601.06225/) (2026)
- [Personalizing explanations of AI-driven hints to users' characteristics: an empirical evaluation](https://mlsanity.com/paper/2403.04035/) (2024)

## Abstract (arXiv)

Students benefit from math problems contextualized to their interests. Large language models (LLMs) offer promise for efficient personalization at scale. However, LLM-generated personalized problems may often have problems such as unrealistic quantities and contexts, poor readability, limited authenticity with respect to students' experiences, and occasional mathematical inconsistencies. To alleviate these problems, we propose a multi-agent framework that formalizes personalization as an iterative generate--validate--revise process; we use four specialized validator agents targeting the criteria of solvability, realism, readability, and authenticity, respectively. We evaluate our framework on 600 problems drawn from a popular online mathematics homework platform, ASSISTments, personalizing each problem to a fixed set of 20 student interest topics. We compare three refinement strategies that differ in how validation feedback is coordinated into revisions. Results show that authenticity and realism are the most frequent failure modes in initial LLM-personalized problems, but that a single refinement iteration substantially reduces these failures. We further find that different refinement strategies have different strengths on different criteria. We also assess validator reliability via human evaluation. Results show that reliability is highest on realism and lowest on authenticity, highlighting the need for better evaluation protocols that consider teachers' and students' personal characteristics.