Data to 7 Oct 2026

Mapping Language to Code in Programmatic Context

arXiv 1808.09588 ↗First posted 29 Aug 2018cs.CLcited by 104

Not on the map. Why?

Abstract (arXiv)

Source code is rarely written in isolation. It depends significantly on the programmatic context, such as the class that the code would reside in. To study this phenomenon, we introduce the task of generating class member functions given English documentation and the programmatic context provided by the rest of the class. This task is challenging because the desired code can vary greatly depending on the functionality the class provides (e.g., a sort function may or may not be available when we are asked to "return the smallest element" in a particular member variable list). We introduce CONCODE, a new large dataset with over 100,000 examples consisting of Java classes from online code repositories, and develop a new encoder-decoder architecture that models the interaction between the method documentation and the class environment. We also present a detailed error analysis suggesting that there is significant room for future work on this task.

Key facts. Cited by 104 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 16; 2023: 23; 2024: 28; 2025: 17; 2026: 9.

104 papers.

Sort:

By area:

2021Program Synthesis with Large Language Modelscited by 3,509
2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
2021Measuring Coding Challenge Competence With APPScited by 879
2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
2024A Survey on Large Language Models for Code Generationcited by 646
2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
2021Unified Pre-training for Program Understanding and Generationcited by 361
2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 355
2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 278
2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 199

38 more

2023FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Setscited by 198
2023SantaCoder: don't reach for the stars!cited by 169
2023ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generationcited by 166
2025Multi-SWE-bench: A Multilingual Benchmark for Issue Resolvingcited by 165
2022DocPrompting: Generating Code by Retrieving the Docscited by 151
2021Retrieval Augmented Code Generation and Summarizationcited by 148
2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
2022Execution-Based Evaluation for Open-Domain Code Generationcited by 89
2022Coder Reviewer Reranking for Code Generationcited by 83
2024EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositoriescited by 71
2024DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositoriescited by 71
2023Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Modelscited by 66
2022Out of the BLEU: how should we assess quality of the Code Generation models?cited by 65
2022NatGen: Generative pre-training by "Naturalizing" source codecited by 53
2024What's Wrong with Your Code Generated by Large Language Models? An Extensive Studycited by 52
2023xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrievalcited by 46
2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
2023SelfEvolve: A Code Evolution Framework via Large Language Modelscited by 43
2023Large Language Model-Aware In-Context Learning for Code Generationcited by 42
2022MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languagescited by 41
2023SALLM: Security Assessment of Generated Codecited by 35
2024Automatic Programming: Large Language Models and Beyondcited by 35
2024SWE-bench-java: A GitHub Issue Resolving Benchmark for Javacited by 34
2025LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarkscited by 34
2024EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluationscited by 33
2023Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Reviewcited by 32
2024mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generationcited by 32
2022Execution-based Evaluation for Data Science Code Generation Modelscited by 31
2023Fully Autonomous Programming with Large Language Modelscited by 31
2023Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChaincited by 31
2024CodeS: Natural Language to Code Repository via Multi-Layer Sketchcited by 28
2024Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarkingcited by 26
2025ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generationcited by 26
2023Exploring Continual Learning for Code Generation Modelscited by 22
2023L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Modelscited by 22
2023Deep Learning for Code Intelligence: Survey, Benchmark and Toolkitcited by 22
2024On the Impacts of Contexts on Repository-Level Code Generationcited by 21
2023Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyondcited by 20

50 most cited of 104 All 104 (JSON)