Mitigating Scaffolding Collapse in Socratic Tutors via Representation Alignment
On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning
also in 1 more
Abstract (arXiv)
Large language model (LLM)-based Socratic tutors increasingly guide students through multi-turn questioning, but they can suffer from scaffolding collapse: under sustained student pressure, a tutor gradually abandons guided inquiry and reveals solutions directly. Prior defenses primarily constrain observable responses through prompting, preference optimization, or filtering, leaving the internal representation drift that precedes trajectory-level collapse largely unaddressed. We propose Scaffold-Preserving Representation Alignment, a two-stage framework that first warms up a Socratic tutor with supervised fine-tuning, then combines trajectory-weighted direct preference optimization with a margin-preserving representation loss anchored to frozen reference states. Our method is designed to maintain separation between scaffold-preserving and collapse-inducing hidden states across dialogue turns. We evaluate our method across five STEM disciplines and five red-teaming attack strategies. On Qwen3-8B, our method lowers Collapse Rate to 32%, delays average collapse onset beyond nine turns, and keeps over-refusal low, suggesting that representation-level alignment can improve the robustness of long-horizon Socratic tutoring under our red-teaming protocol.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 26 of its 36 references that are in the map. First posted on arXiv 15 Jun 2026.
Builds on
26 of 36 references.
References · 26
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
- 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
- 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
14 more
- 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
- 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
- 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
- 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
- 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 131
- 2023EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Educationcited by 80
- 2024Direct Multi-Turn Preference Optimization for Language Agentscited by 64
- 2025MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutorscited by 49
- 2025Representation Bending for Large Language Model Safetycited by 30
- 2024Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failurecited by 24
- 2023Can Language Models Employ the Socratic Method? Experiments with Code Debuggingcited by 15
- 2025Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMscited by 5
- 2026State-Dependent Safety Failures in Multi-Turn Language Model Interactioncited by 4
- 2026SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning
- 2025Socratic Students: Teaching Language Models to Learn by Asking Questions
- 2025SocraticAI: Transforming LLMs into Guided CS Tutors Through Scaffolded Interaction
- 2026Teaching a Large Language Model Tutor to Withhold the Answer: A Supervisor Architecture and an Evidence-Driven Method for Tuning Socratic Behavior
- 2025Fuzzy, Symbolic, and Contextual: Enhancing LLM Instruction via Cognitive Scaffolding
- 2026PIVOT: Preference-based Intervention Vectors for Pedagogical Tutor Steering
- 2026Letting Tutor Personas Speak Up for LLMs: Learning Steering Vectors from Dialogue via Preference Optimization
- 2026PEARL: Training Socratic Tutors with Pedagogically Aligned Reinforcement Learning