mlsanityData to 7 Oct 2026

What is it like to program with artificial intelligence?

arXiv 2208.06213 ↗First posted 12 Aug 2022cs.HC cs.AI cs.PLcited by 39

Not on the map. Why?

Abstract (arXiv)

Large language models, such as OpenAI's codex and Deepmind's AlphaCode, can generate code to solve a variety of problems expressed in natural language. This technology has already been commercialised in at least one widely-used programming editor extension: GitHub Copilot. In this paper, we explore how programming with large language models (LLM-assisted programming) is similar to, and differs from, prior conceptualisations of programmer assistance. We draw upon publicly available experience reports of LLM-assisted programming, as well as prior usability and design studies. We find that while LLM-assisted programming shares some properties of compilation, pair programming, and programming via search and reuse, there are fundamental differences both in the technical possibilities as well as the practical experience. Thus, LLM-assisted programming ought to be viewed as a new way of programming with its own distinct properties and challenges. Finally, we draw upon observations from a user study in which non-expert end user programmers use LLM-assisted tools for solving data tasks in spreadsheets. We discuss the issues that might arise, and open research challenges, in applying large language models to end-user programming, particularly with users who have little or no programming expertise.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 20 of its 97 references that are in the map. First posted on arXiv 12 Aug 2022.

Builds on

20 of 97 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  9. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  10. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  11. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  12. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
8 more
  1. 2017A Survey of Machine Learning for Big Code and Naturalnesscited by 220
  2. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
  3. 2022Productivity Assessment of Neural Code Completioncited by 97
  4. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  5. 2020LambdaNet: Probabilistic Type Inference using Graph Neural Networkscited by 44
  6. 2021In-IDE Code Generation from Natural Language: Promise and Challengescited by 41
  7. 2022Is explainable AI a race against model complexity?cited by 9
  8. 2017SmartPaste: Learning to Adapt Source Codecited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 11; 2024: 10; 2025: 9; 2026: 8.

39 papers.

Sort:

By area:

  1. 2023Using an LLM to Help With Code Understandingcited by 222
  2. 2022Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programmingcited by 91
  3. 2023Computing Education in the Era of Generative AIcited by 50
  4. 2023Large Language Models and Simple, Stupid Bugscited by 45
  5. 2024Prompting Techniques for Secure Code Generation: A Systematic Investigationcited by 33
  6. 2024Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decompositioncited by 29
  7. 2023When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programmingcited by 25
  8. 2024Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seekingcited by 24
  9. 2024Bringing Generative AI to Adaptive Learning in Educationcited by 24
  10. 2024HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agentcited by 24
  11. 2023Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completionscited by 20
  12. 2023Is AI the better programming partner? Human-Human Pair Programming vs. Human-AI pAIr Programmingcited by 20
27 more
  1. 2023HPC-Coder: Modeling Parallel Programs using Large Language Modelscited by 11
  2. 2023Exploring Perspectives on the Impact of Artificial Intelligence on the Creativity of Knowledge Work: Beyond Mechanised Plagiarism and Stochastic Parrotscited by 11
  3. 2024Can ChatGPT Support Developers? An Empirical Evaluation of Large Language Models for Code Generationcited by 11
  4. 2023Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Codecited by 9
  5. 2025How Do Programming Students Use Generative AI?cited by 9
  6. 2023A Language Model of Java Methods with Train/Test Deduplicationcited by 6
  7. 2023Co-audit: tools to help humans double-check AI-generated contentcited by 5
  8. 2024Cocobo: Exploring Large Language Models as the Engine for End-User Robot Programmingcited by 5
  9. 2025Do It For Me vs. Do It With Me: Investigating User Perceptions of Different Paradigms of Automation in Copilots for Feature-Rich Softwarecited by 5
  10. 2025Can We Enhance Bug Report Quality Using LLMs?: An Empirical Study of LLM-Based Bug Report Generationcited by 3
  11. 2025NeuroSync: Intent-Aware Code-Based Problem Solving via Direct LLM Understanding Modificationcited by 3
  12. 2026Humans are Missing from AI Coding Agent Researchcited by 3
  13. 2024Exploring the Integration of Large Language Models in Industrial Test Maintenance Processescited by 2
  14. 2024Optimizing AI-Assisted Code Generationcited by 2
  15. 2025Evaluating the Process Modeling Abilities of Large Language Models -- Preliminary Foundations and Resultscited by 1
  16. 2025AutoCodeSherpa: Symbolic Explanations in AI Coding Agentscited by 1
  17. 2025The Variance Paradox: How AI Reduces Diversity but Increases Noveltycited by 1
  18. 2025Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generationcited by 1
  19. 2025User Misconceptions of LLM-Based Conversational Programming Assistantscited by 1
  20. 2026Priming, Path-dependence, and Plasticity: Understanding the molding of user-LLM interaction and its implications from (many) chat logs in the wildcited by 1
  21. 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 1
  22. 2024On Languaging a Simulation Enginecited by 0
  23. 2026The Machine Learning Canvas: Empirical Findings on Why Strategy Matters More Than AI Code Generationcited by 0
  24. 2026AI Adoption Across a Multinational Workforce: Sociotechnical Conditions for GenAI Acceptance in Human Resourcescited by 0
  25. 2026Accuracy and Satisfaction in Multi-Turn LLM Dialogues for NFR Assessmentcited by 0
  26. 2026Plans Work in Mysterious Ways: Evaluating a Plan Mode for Spreadsheet Agentscited by 0
  27. 2026Educating the Agentic Engineer: Curricula, Collaboration, and Continuous Learning in the AI Eracited by 0