# A Scaffolded GenAI Lab in Early Undergraduate CS: A Mixed-Methods, Multi-Course Evaluation

arXiv: https://arxiv.org/abs/2505.00100 · HTML: https://mlsanity.com/paper/2505.00100/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 32 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Builds on (most cited first, 7 of 7)

- [GitHub Copilot AI pair programmer: Asset or Liability?](https://mlsanity.com/paper/2206.15331/) (2022)
- [Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation](https://mlsanity.com/paper/2212.01020/) (2022)
- [CS1-LLM: Integrating LLMs into CS1 Instruction](https://mlsanity.com/paper/2406.15379/) (2024)
- [Evaluating Contextually Personalized Programming Exercises Created with Generative AI](https://mlsanity.com/paper/2407.11994/) (2024)
- [Generative AI Assistants in Software Development Education: A vision for integrating Generative AI into educational practice, not instinctively defending against it](https://mlsanity.com/paper/2303.13936/) (2023)
- [Early Adoption of Generative Artificial Intelligence in Computing Education: Emergent Student Use Cases and Perspectives in 2023](https://mlsanity.com/paper/2411.11166/) (2024)
- [AI Literacy in K-12 and Higher Education in the Wake of Generative AI: An Integrative Review](https://mlsanity.com/paper/2503.00079/) (2025)

## Abstract (arXiv)

Background and Context. Generative AI (GenAI) tools are increasingly used in programming courses, but we have limited evidence about how brief instruction can foster responsible, learning-oriented use. Objectives. We evaluate "AI-Lab", a scaffolded GenAI literacy intervention, asking how students' self-reported GenAI usage and their openness and comfort using GenAI for conceptual, debugging, and homework tasks change after participation. Methods. Across two semesters in three CS courses and one first-year engineering course at a U.S. university, we deployed the "AI-Lab" (pre-lab orientation, in-class critique of GenAI outputs, and a required homework reflection), collecting paired pre/post surveys (Perception N=831; Usage N=826) and six post-intervention focus groups; primary inferential analyses used the three CS courses (N=778 and 773, respectively). We analyzed survey shifts with paired non-parametric tests and focus groups via thematic analysis. Findings. Openness increased for conceptual questions and homework help, and comfort increased for conceptual, debugging, and homework scenarios; self-reported frequency of GenAI use for homework and projects remained stable, while self-reported use for debugging increased. Focus group participants described adopting more iterative prompting strategies, becoming more skeptical of correctness, and articulating clearer boundaries around integrity and dependence. Implications. A short, structured intervention can shift students' reported comfort with and willingness to use GenAI and influence the strategies they describe for engaging with it without increasing overall self-reported use on graded work. These results motivate future work triangulating surveys with behavioral traces and learning measures.