mlsanityData to 14 Oct 2026

Mitigating Scaffolding Collapse in Socratic Tutors via Representation Alignment

arXiv 2607.19371 ↗First posted 15 Jun 2026cs.AI cs.CL cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

also in 1 more
Abstract (arXiv)

Large language model (LLM)-based Socratic tutors increasingly guide students through multi-turn questioning, but they can suffer from scaffolding collapse: under sustained student pressure, a tutor gradually abandons guided inquiry and reveals solutions directly. Prior defenses primarily constrain observable responses through prompting, preference optimization, or filtering, leaving the internal representation drift that precedes trajectory-level collapse largely unaddressed. We propose Scaffold-Preserving Representation Alignment, a two-stage framework that first warms up a Socratic tutor with supervised fine-tuning, then combines trajectory-weighted direct preference optimization with a margin-preserving representation loss anchored to frozen reference states. Our method is designed to maintain separation between scaffold-preserving and collapse-inducing hidden states across dialogue turns. We evaluate our method across five STEM disciplines and five red-teaming attack strategies. On Qwen3-8B, our method lowers Collapse Rate to 32%, delays average collapse onset beyond nine turns, and keeps over-refusal low, suggesting that representation-level alignment can improve the robustness of long-horizon Socratic tutoring under our red-teaming protocol.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 26 of its 36 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

26 of 36 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  8. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  9. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  10. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  11. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
  12. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
14 more
  1. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
  2. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  3. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
  4. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
  5. 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 131
  6. 2023EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Educationcited by 80
  7. 2024Direct Multi-Turn Preference Optimization for Language Agentscited by 64
  8. 2025MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutorscited by 49
  9. 2025Representation Bending for Large Language Model Safetycited by 30
  10. 2024Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failurecited by 24
  11. 2023Can Language Models Employ the Socratic Method? Experiments with Code Debuggingcited by 15
  12. 2025Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMscited by 5
  13. 2026State-Dependent Safety Failures in Multi-Turn Language Model Interactioncited by 4
  14. 2026SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning
  2. 2025Socratic Students: Teaching Language Models to Learn by Asking Questions
  3. 2025SocraticAI: Transforming LLMs into Guided CS Tutors Through Scaffolded Interaction
  4. 2026Teaching a Large Language Model Tutor to Withhold the Answer: A Supervisor Architecture and an Evidence-Driven Method for Tuning Socratic Behavior
  5. 2025Fuzzy, Symbolic, and Contextual: Enhancing LLM Instruction via Cognitive Scaffolding
  6. 2026PIVOT: Preference-based Intervention Vectors for Pedagogical Tutor Steering
  7. 2026Letting Tutor Personas Speak Up for LLMs: Learning Steering Vectors from Dialogue via Preference Optimization
  8. 2026PEARL: Training Socratic Tutors with Pedagogically Aligned Reinforcement Learning