# Application-Driven Pedagogical Knowledge Optimization of Open-Source LLMs via Reinforcement Learning and Supervised Fine-Tuning

arXiv: https://arxiv.org/abs/2604.06385 · HTML: https://mlsanity.com/paper/2604.06385/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 7 Apr 2026.

## Builds on (most cited first, 14 of 14)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap](https://mlsanity.com/paper/2306.01941/) (2023)
- [Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review](https://mlsanity.com/paper/2303.13379/) (2023)
- [Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure](https://mlsanity.com/paper/2410.03781/) (2024)
- [Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models](https://mlsanity.com/paper/2407.01906/) (2024)
- [OpenMedLM: Prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models](https://mlsanity.com/paper/2402.19371/) (2024)
- [The Open Source Advantage in Large Language Models (LLMs)](https://mlsanity.com/paper/2412.12004/) (2024)
- [Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage Policy Optimization](https://mlsanity.com/paper/2412.18279/) (2024)
- [FineTuneBench: How well do commercial fine-tuning APIs infuse knowledge into LLMs?](https://mlsanity.com/paper/2411.05059/) (2024)
- [TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Models](https://mlsanity.com/paper/2510.02663/) (2025)
- [TeachLM: Post-Training LLMs for Education Using Authentic Learning Data](https://mlsanity.com/paper/2510.05087/) (2025)
- [Defending MoE LLMs against Harmful Fine-Tuning via Safety Routing Alignment](https://mlsanity.com/paper/2509.22745/) (2025)
- [Benchmarking the Pedagogical Knowledge of Large Language Models](https://mlsanity.com/paper/2506.18710/) (2025)

## Abstract (arXiv)

We present an innovative multi-stage optimization strategy combining reinforcement learning (RL) and supervised fine-tuning (SFT) to enhance the pedagogical knowledge of large language models (LLMs), as illustrated by EduQwen 32B-RL1, EduQwen 32B-SFT, and an optional third-stage model EduQwen 32B-SFT-RL2: (1) RL optimization that implements progressive difficulty training, focuses on challenging examples, and employs extended reasoning rollouts; (2) a subsequent SFT phase that leverages the RL-trained model to synthesize high-quality training data with difficulty-weighted sampling; and (3) an optional second round of RL optimization. EduQwen 32B-RL1, EduQwen 32B-SFT, and EduQwen 32B-SFT-RL2 are an application-driven family of open-source pedagogical LLMs built on a dense Qwen3-32B backbone. These models remarkably achieve high enough accuracy on the Cross-Domain Pedagogical Knowledge (CDPK) Benchmark to establish new state-of-the-art (SOTA) results across the interactive Pedagogy Benchmark Leaderboard and surpass significantly larger proprietary systems such as the previous benchmark leader Gemini-3 Pro. These dense 32-billion-parameter models demonstrate that domain-specialized optimization can transform mid-sized open-source LLMs into true pedagogical domain experts that outperform much larger general-purpose systems, while preserving the transparency, customizability, and cost-efficiency required for responsible educational AI deployment.