mlsanityData to 14 Oct 2026

Is AI the better programming partner? Human-Human Pair Programming vs. Human-AI pAIr Programming

arXiv 2306.05153 ↗First posted 8 Jun 2023cs.HC cs.AIcited by 20

Not on the map. Why?

Abstract (arXiv)

The emergence of large-language models (LLMs) that excel at code generation and commercial products such as GitHub's Copilot has sparked interest in human-AI pair programming (referred to as "pAIr programming") where an AI system collaborates with a human programmer. While traditional pair programming between humans has been extensively studied, it remains uncertain whether its findings can be applied to human-AI pair programming. We compare human-human and human-AI pair programming, exploring their similarities and differences in interaction, measures, benefits, and challenges. We find that the effectiveness of both approaches is mixed in the literature (though the measures used for pAIr programming are not as comprehensive). We summarize moderating factors on the success of human-human pair programming, which provides opportunities for pAIr programming research. For example, mismatched expertise makes pair programming less productive, therefore well-designed AI programming assistants may adapt to differences in expertise levels.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 18 of its 93 references that are in the map. First posted on arXiv 8 Jun 2023.

Builds on

18 of 93 references.

References · 18

  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  2. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 515
  3. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  4. 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 274
  5. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 218
  6. 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 152
  7. 2022Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproductioncited by 111
  8. 2022Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generationcited by 99
  9. 2022Productivity Assessment of Neural Code Completioncited by 98
  10. 2022Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programmingcited by 91
  11. 2023Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programmingcited by 89
  12. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 67
6 more
  1. 2023"It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmerscited by 64
  2. 2023Comparing Code Explanations Created by Students and Large Language Modelscited by 64
  3. 2023Learning gain differences between ChatGPT and human tutor generated algebra hintscited by 43
  4. 2022What is it like to program with artificial intelligence?cited by 39
  5. 2022How Readable is Model-generated Code? Examining Readability and Visual Inspection of GitHub Copilotcited by 14
  6. 2022Exploring the Verifiability of Code Generated by GitHub Copilotcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 4; 2025: 8; 2026: 2.

20 papers.

Sort:

By area:

  1. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  2. 2023The Robots are Here: Navigating the Generative AI Revolution in Computing Educationcited by 86
  3. 2023Generative AI for Programming Education: Benchmarking ChatGPT, GPT-4, and Human Tutorscited by 39
  4. 2025Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Supportcited by 34
  5. 2023How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debuggingcited by 30
  6. 2024The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmerscited by 30
  7. 2023LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMscited by 27
  8. 2023Evaluating ChatGPT and GPT-4 for Visual Programmingcited by 13
  9. 2024A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Coursecited by 10
  10. 2025AI & Human Co-Improvement for Safer Co-Superintelligencecited by 8
  11. 2025The Impact of Generative AI Coding Assistants on Developers Who Are Visually Impairedcited by 7
  12. 2025When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaborationcited by 4
8 more
  1. 2025From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development -- An Opinion Papercited by 3
  2. 2024Generating Situated Reflection Triggers about Alternative Solution Paths: A Case Study of Generative AI for Computer-Supported Collaborative Learningcited by 2
  3. 2025Prompt Programming: A Platform for Dialogue-based Computational Problem Solving with Generative AI Modelscited by 2
  4. 2024The importance of visual modelling languages in generative software engineeringcited by 1
  5. 2025Identifying & Interactively Refining Ambiguous User Goals for Data Visualization Code Generationcited by 1
  6. 2026Human-Human-AI Triadic Programming: Uncovering the Role of AI Agent and the Value of Human Partner in Collaborative Learningcited by 1
  7. 2025Simulating Teams with LLM Agents: Interactive 2D Environments for Studying Human-AI Dynamicscited by 0
  8. 2026The Work Behind Delegation: A Framework for Supervising AI Coding Agentscited by 0