mlsanityData to 7 Oct 2026

A Survey of Machine Learning for Big Code and Naturalness

arXiv 1709.06182 ↗First posted 18 Sep 2017cs.SE cs.LG cs.PLcited by 220

Not on the map. Why?

Abstract (arXiv)

Research at the intersection of machine learning, programming languages, and software engineering has recently taken important steps in proposing learnable probabilistic models of source code that exploit code's abundance of patterns. In this article, we survey this work. We contrast programming languages against natural languages and discuss how these similarities and differences drive the design of probabilistic models. We present a taxonomy based on the underlying design principles of each model and use it to navigate the literature. Then, we review how researchers have adapted these models to application areas and discuss cross-cutting and application-specific challenges and opportunities.

Key facts. Cited by 220 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 45; 2022: 51; 2023: 42; 2024: 26; 2025: 39; 2026: 16.

220 papers.

Sort:

By area:

  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  2. 2021Program Synthesis with Large Language Modelscited by 3,509
  3. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  4. 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 199
  5. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 173
  6. 2021Retrieval Augmented Code Generation and Summarizationcited by 148
  7. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  8. 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 133
  9. 2023CodeBERTScore: Evaluating Code Generation with Pretrained Models of Codecited by 101
  10. 2021Directed Acyclic Graph Network for Conversational Emotion Recognitioncited by 97
  11. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  12. 2021CURE: Code-Aware Neural Machine Translation for Automatic Program Repaircited by 85
38 more
  1. 2024Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Surveycited by 82
  2. 2021CoSQA: 20,000+ Web Queries for Code Search and Question Answeringcited by 74
  3. 2021Directed Acyclic Graph Neural Networkscited by 70
  4. 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
  5. 2021Self-Supervised Bug Detection and Repaircited by 59
  6. 2022Top-Down Synthesis for Library Learningcited by 57
  7. 2024Exploring and Unleashing the Power of Large Language Models in Automated Code Translationcited by 57
  8. 2021A Survey of Machine Learning for Computer Architecture and Systemscited by 53
  9. 2022NatGen: Generative pre-training by "Naturalizing" source codecited by 53
  10. 2023Using Large Language Models to Generate JUnit Tests: An Empirical Studycited by 53
  11. 2023ICE-Score: Instructing Large Language Models to Evaluate Codecited by 50
  12. 2021PalmTree: Learning an Assembly Language Model for Instruction Embeddingcited by 49
  13. 2022CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequencescited by 49
  14. 2022Investigating Explainability of Generative AI for Code through Scenario-based Designcited by 47
  15. 2024AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodologycited by 45
  16. 2023Large Language Model-Aware In-Context Learning for Code Generationcited by 42
  17. 2021CompilerGym: Robust, Performant Compiler Optimization Environments for AI Researchcited by 41
  18. 2022What is it like to program with artificial intelligence?cited by 39
  19. 2021Neural Transfer Learning for Repairing Security Vulnerabilities in C Codecited by 36
  20. 2023SALLM: Security Assessment of Generated Codecited by 35
  21. 2023Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Reviewcited by 32
  22. 2021Code Completion by Modeling Flattened Abstract Syntax Trees as Graphscited by 30
  23. 2023Distilled GPT for Source Code Summarizationcited by 30
  24. 2023Large Language Models of Code Fail at Completing Code with Potential Bugscited by 28
  25. 2025Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystemcited by 28
  26. 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunitiescited by 26
  27. 2021Project-Level Encoding for Neural Source Code Summarization of Subroutinescited by 24
  28. 2025AlgoTune: Can Language Models Speed Up General-Purpose Numerical Programs?cited by 24
  29. 2021A Survey on Machine Learning Techniques for Source Code Analysiscited by 23
  30. 2022Transformers over Directed Acyclic Graphscited by 23
  31. 2023Deep Learning for Code Intelligence: Survey, Benchmark and Toolkitcited by 22
  32. 2022CoCoSoDa: Effective Contrastive Learning for Code Searchcited by 21
  33. 2025Process-Supervised Reinforcement Learning for Code Generationcited by 21
  34. 2021Generating Adversarial Computer Programs using Optimized Obfuscationscited by 20
  35. 2021Understanding Neural Code Intelligence Through Program Simplificationcited by 20
  36. 2022PACE: A Parallelizable Computation Encoder for Directed Acyclic Graphscited by 20
  37. 2021Improving Code Summarization with Block-wise Abstract Syntax Tree Splittingcited by 19
  38. 2022CrossBeam: Learning to Search in Bottom-Up Program Synthesiscited by 18

50 most cited of 220 All 220 (JSON)