mlsanityData to 7 Oct 2026

CodeQA: A Question Answering Dataset for Source Code Comprehension

arXiv 2109.08365 ↗First posted 17 Sep 2021cs.CL cs.SEcited by 30

Not on the map. Why?

Abstract (arXiv)

We propose CodeQA, a free-form question answering dataset for the purpose of source code comprehension: given a code snippet and a question, a textual answer is required to be generated. CodeQA contains a Java dataset with 119,778 question-answer pairs and a Python dataset with 70,085 question-answer pairs. To obtain natural and faithful questions and answers, we implement syntactic rules and semantic analysis to transform code comments into question-answer pairs. We present the construction process and conduct systematic analysis of our dataset. Experiment results achieved by several neural baselines on our dataset are shown and discussed. While research on question-answering and machine reading comprehension develops rapidly, few prior work has drawn attention to code question answering. This new dataset can serve as a useful research benchmark for source code comprehension.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 17 Sep 2021.

Builds on

34 of 53 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2018Deep contextualized word representationscited by 2,285
  7. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  8. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  9. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  10. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  11. 2018Self-Attention with Relative Position Representationscited by 1,112
  12. 2015Teaching Machines to Read and Comprehendcited by 1,039
22 more
  1. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  2. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  3. 2018AllenNLP: A Deep Semantic Natural Language Processing Platformcited by 347
  4. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  5. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  6. 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
  7. 2019Neural Text Summarization: A Critical Evaluationcited by 189
  8. 2018code2seq: Generating Sequences from Structured Representations of Codecited by 177
  9. 2018emrQA: A Large Corpus for Question Answering on Electronic Medical Recordscited by 134
  10. 2020A Transformer-based Approach for Source Code Summarizationcited by 129
  11. 2018Improving Automatic Source Code Summarization via Deep Reinforcement Learningcited by 99
  12. 2020Improved Code Summarization via a Graph Neural Networkcited by 79
  13. 2019A Neural Model for Generating Natural Language Summaries of Program Subroutinescited by 76
  14. 2021Language-Agnostic Representation Learning of Source Code from Structure and Contextcited by 63
  15. 2019Code Generation as a Dual Task of Code Summarizationcited by 62
  16. 2020EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answeringcited by 60
  17. 2017A parallel corpus of Python functions and documentation strings for automated code documentation and code generationcited by 56
  18. 2019Automatic Source Code Summarization with Extended Tree-LSTMcited by 28
  19. 2019Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Datacited by 20
  20. 2020Syn-QG: Syntactic and Shallow Semantic Rules for Question Generationcited by 12
  21. 2019A Survey on Neural Machine Reading Comprehensioncited by 8
  22. 2021A Neural Question Answering System for Basic Questions about Subroutinescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 7; 2025: 6; 2026: 9.

30 papers.

Sort:

By area:

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  3. 2025Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMscited by 47
  4. 2025SWE-QA: Can Language Models Answer Repository-level Code Questions?cited by 37
  5. 2024Retraining-Free Merging of Sparse MoE via Hierarchical Clusteringcited by 31
  6. 2024NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctnesscited by 23
  7. 2023Deep Learning for Code Intelligence: Survey, Benchmark and Toolkitcited by 22
  8. 2024UniCoder: Scaling Code Large Language Model via Universal Codecited by 18
  9. 2024CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMscited by 18
  10. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  11. 2024ProCQA: A Large-scale Community-based Programming Question Answering Dataset for Code Searchcited by 12
  12. 2022CodeQueries: A Dataset of Semantic Queries over Codecited by 11
18 more
  1. 2022CS1QA: A Dataset for Assisting Code-based Question Answering in an Introductory Programming Coursecited by 11
  2. 2024CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answeringcited by 11
  3. 2025Software Development Life Cycle Perspective: A Survey of Benchmarks for Code Large Language Models and Agentscited by 11
  4. 2026SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understandingcited by 9
  5. 2024Leveraging Large Language Models in Code Question Answering: Baselines and Issuescited by 6
  6. 2026KOCO-BENCH: Can Large Language Models Leverage Domain Knowledge in Software Development?cited by 5
  7. 2025NoTeS-Bank: Benchmarking Neural Transcription and Search for Scientific Notes Understandingcited by 3
  8. 2022Towards Top-Down Automated Development in Limited Scopes: A Neuro-Symbolic Framework from Expressibles to Executablescited by 2
  9. 2022Semantic-Preserving Adversarial Code Comprehensioncited by 2
  10. 2026SWE-QA: A Dataset and Benchmark for Complex Code Understandingcited by 2
  11. 2023An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Codecited by 1
  12. 2025Empowering RepoQA-Agent based on Reinforcement Learning Driven by Monte-carlo Tree Searchcited by 1
  13. 2026Lost in Transcription: How Speech-to-Text Errors Derail Code Understandingcited by 1
  14. 2026Neurosymbolic Repo-level Code Localizationcited by 1
  15. 2026Rethinking Code Review in the Age of AI: A Vision for Agentic Code Reviewcited by 1
  16. 2026SWE-Spot: Building Small Repo-Experts with Repository-Centric Learningcited by 0
  17. 2026DeepRepoQA: Code Repository Question Answering with Deep Agent Explorationcited by 0
  18. 2026Can LLMs Reason About Runtime Behavior? A Repository-Level Dynamic Benchmarkcited by 0