mlsanityData to 7 Oct 2026

Giving Feedback on Interactive Student Programs with Meta-Exploration

arXiv 2211.08802 ↗First posted 16 Nov 2022cs.LG cs.AI stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Developing interactive software, such as websites or games, is a particularly engaging way to learn computer science. However, teaching and giving feedback on such software is time-consuming -- standard approaches require instructors to manually grade student-implemented interactive programs. As a result, online platforms that serve millions, like Code.org, are unable to provide any feedback on assignments for implementing interactive programs, which critically hinders students' ability to learn. One approach toward automatic grading is to learn an agent that interacts with a student's program and explores states indicative of errors via reinforcement learning. However, existing work on this approach only provides binary feedback of whether a program is correct or not, while students require finer-grained feedback on the specific errors in their programs to understand their mistakes. In this work, we show that exploring to discover errors can be cast as a meta-exploration problem. This enables us to construct a principled objective for discovering errors and an algorithm for optimizing this objective, which provides fine-grained feedback. We evaluate our approach on a set of over 700K real anonymized student programs from a Code.org interactive assignment. Our approach provides feedback with 94.3% accuracy, improving over existing approaches by 17.7% and coming within 1.5% of human-level accuracy. Project web page: https://ezliu.github.io/dreamgrader.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 16 Nov 2022.

Builds on

32 of 59 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  6. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  7. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  8. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  9. 2016Learning to reinforcement learncited by 369
  10. 2015Deep Knowledge Tracingcited by 328
  11. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  12. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
20 more
  1. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  2. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  3. 2019Meta reinforcement learning as task inferencecited by 75
  4. 2020Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrificescited by 59
  5. 2019Meta Reinforcement Learning for Sim-to-real Domain Adaptationcited by 49
  6. 2019Environment Probing Interaction Policiescited by 39
  7. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 35
  8. 2020Meta-Reinforcement Learning for Robotic Industrial Insertion Taskscited by 28
  9. 2021Improving Playtesting Coverage via Curiosity Driven Reinforcement Learning Agentscited by 28
  10. 2020An Imitation Learning Approach for Cache Replacementcited by 24
  11. 2015Learning Program Embeddings to Propagate Feedback on Student Codecited by 19
  12. 2018Zero Shot Learning for Code Education: Rubric Sampling with Deep Learning Inferencecited by 16
  13. 2020Learning Adaptive Exploration Strategies in Dynamic Environments Through Informed Policy Regularizationcited by 15
  14. 2016Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural Networkscited by 13
  15. 2017The Continuous Hint Factory - Providing Hints in Vast and Sparsely Populated Edit Distance Spacescited by 12
  16. 2021ProtoTransformer: A Meta-Learning Approach to Providing Student Feedbackcited by 8
  17. 2016Predicting Student Dropout in Higher Educationcited by 4
  18. 2019MAME : Model-Agnostic Meta-Explorationcited by 4
  19. 2020Learning Abstract Models for Strategic Exploration and Fast Reward Transfercited by 4
  20. 2021Play to Grade: Testing Coding Games as Classifying Markov Decision Processcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 1; 2026: 1.

6 papers.

  1. 2024A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Sciencecited by 31
  2. 2023Off-Policy Evaluation for Human Feedbackcited by 6
  3. 2023Generating Language Corrections for Teaching Physical Control Taskscited by 4
  4. 2025CoTAL: Human-in-the-Loop Prompt Engineering for Generalizable Formative Assessment Scoring and Feedbackcited by 4
  5. 2024Off-Policy Selection for Initiating Human-Centric Experimental Designcited by 0
  6. 2026Discovering High-Quality Chess Puzzles with Offline Reinforcement Learningcited by 0