mlsanityData to 7 Oct 2026

CS1QA: A Dataset for Assisting Code-based Question Answering in an Introductory Programming Course

arXiv 2210.14494 ↗First posted 26 Oct 2022cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

We introduce CS1QA, a dataset for code-based question answering in the programming education domain. CS1QA consists of 9,237 question-answer pairs gathered from chat logs in an introductory programming class using Python, and 17,698 unannotated chat data with code. Each question is accompanied with the student's code, and the portion of the code relevant to answering the question. We carefully design the annotation process to construct CS1QA, and analyze the collected dataset in detail. The tasks for CS1QA are to predict the question type, the relevant code snippet given the question and the code and retrieving an answer from the annotated corpus. Results for the experiments on several baseline models are reported and thoroughly analyzed. The tasks for CS1QA challenge models to understand both the code and natural language. This unique dataset can be used as a benchmark for source code comprehension and question answering in the educational setting.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 18 of its 26 references that are in the map. First posted on arXiv 26 Oct 2022.

Builds on

18 of 26 references.

References · 18

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2015VQA: Visual Question Answeringcited by 2,489
  7. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  9. 2018CoQA: A Conversational Question Answering Challengecited by 674
  10. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  11. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  12. 2017Learning to Represent Programs with Graphscited by 219
6 more
  1. 2018code2seq: Generating Sequences from Structured Representations of Codecited by 177
  2. 2020Graph-based, Self-Supervised Program Repair from Diagnostic Feedbackcited by 93
  3. 2020PyMT5: multi-mode translation of natural language and Python code with transformerscited by 80
  4. 2018Generative Code Modeling with Graphscited by 59
  5. 2021CodeQA: A Question Answering Dataset for Source Code Comprehensioncited by 30
  6. 2019The TechQA Datasetcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 4; 2026: 2.

11 papers.

  1. 2023The Robots are Here: Navigating the Generative AI Revolution in Computing Educationcited by 85
  2. 2023Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Educationcited by 38
  3. 2025SWE-QA: Can Language Models Answer Repository-level Code Questions?cited by 37
  4. 2024NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctnesscited by 23
  5. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  6. 2024CoSQA+: Pioneering the Multi-Choice Code Search Benchmark with Test-Driven Agentscited by 11
  7. 2025Software Development Life Cycle Perspective: A Survey of Benchmarks for Code Large Language Models and Agentscited by 11
  8. 2024Leveraging Large Language Models in Code Question Answering: Baselines and Issuescited by 6
  9. 2025RefactorCoderQA: Benchmarking LLMs for Multi-Domain Coding Question Solutions in Cloud and Edge Deploymentcited by 2
  10. 2026DeepRepoQA: Code Repository Question Answering with Deep Agent Explorationcited by 0
  11. 2026Can LLMs Reason About Runtime Behavior? A Repository-Level Dynamic Benchmarkcited by 0