mlsanityData to 14 Oct 2026

Guardrails or Roadblocks? Effects of Pedagogical Style and Context Awareness in AI Teaching Assistants for Programming

arXiv 2609.29995 ↗First posted 24 Sep 2026cs.HC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

AI teaching assistants (AI TAs) backed by large language models (LLMs) and pedagogical guardrails are increasingly being integrated into programming courses, providing students with scalable access to hints, conceptual explanations, and code-level feedback. However, guardrails may also create friction. If students feel that the support provided is overly restrictive or poorly contextualized to their current progress, they may bypass approved tools for general-purpose LLMs. To investigate how AI TA design affects students' learning experiences, we conducted a randomized controlled trial with 132 students in an introductory programming course. Students completed three tasks related to code-writing and debugging and were randomly assigned to one of four AI TAs varied across two dimensions: pedagogical guidance style (Socratic vs. Direct instruction) and context awareness (no context vs. full context of the problem and student solution). We examined students' perceptions, interaction behaviors, and evidence of post-task comprehension. Students rated the Socratic AI TA with full context least favorably, reporting significantly lower perceived support for task completion. Descriptively, this condition also showed the highest observed interaction stress, the highest rate of external LLM use, and the lowest proportion of post-task explanations demonstrating full comprehension, though these differences were not statistically significant. These findings suggest that guardrailed AI TAs are not automatically better for learning. Instead, their effectiveness depends on how pedagogical guidance and contextual awareness are balanced in ways that students experience as useful, supportive, and worth continuing to use.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 35 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

6 of 35 references.

References · 6

  1. 2023CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classescited by 72
  2. 2023Exploring the Responses of Large Language Models to Beginner Programmers' Help Requestscited by 46
  3. 2023Patterns of Student Help-Seeking When Using a Large Language Model-Powered Programming Assistantcited by 13
  4. 202461A Bot Report: AI Assistants in CS1 Save Students Homework Time and Reduce Demands on Staff. (Now What?)cited by 7
  5. 2025Exploring Student Behaviors and Motivations using AI TAs with Optional Guardrailscited by 3
  6. 2026"Bespoke Bots": Diverse Instructor Needs for Customizing Generative AI Classroom Chatbotscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Desirable Characteristics for AI Teaching Assistants in Programming Education
  2. 2025Building Effective Safety Guardrails in AI Education Tools
  3. 2026CodeGuard: Improving LLM Guardrails in CS Education
  4. 2026Will It Teach as Intended? How Teachers Configure Educational AI Chatbots ↗
  5. 2026AI Assistants Overassist
  6. 2026Teacher-Authored Prompts for Configuring Student-AI Dialogue: K-12 Classroom Implementation
  7. 2026An Explainable AI Assistant for Introductory Programming Education: Improving Feedback Reliability with Instructor-AI Collaboration
  8. 2026Evaluating Epistemic Guardrails in AI Reading Assistants: A Behavioral Audit of a Minimal Prototype