mlsanityData to 7 Oct 2026

Using an LLM to Help With Code Understanding

arXiv 2307.08177 ↗First posted 17 Jul 2023cs.SE cs.AI cs.HCcited by 222

Not on the map. Why?

Cited by 29 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Understanding code is challenging, especially when working in new and complex development environments. Code comments and documentation can help, but are typically scarce or hard to navigate. Large language models (LLMs) are revolutionizing the process of writing code. Can they do the same for helping understand it? In this study, we provide a first investigation of an LLM-based conversational UI built directly in the IDE that is geared towards code understanding. Our IDE plugin queries OpenAI's GPT-3.5-turbo model with four high-level requests without the user having to write explicit prompts: to explain a highlighted section of code, provide details of API calls used in the code, explain key domain-specific terms, and provide usage examples for an API. The plugin also allows for open-ended prompts, which are automatically contextualized to the LLM with the program being edited. We evaluate this system in a user study with 32 participants, which confirms that using our plugin can aid task completion more than web search. We additionally provide a thorough analysis of the ways developers use, and perceive the usefulness of, our system, among others finding that the usage and benefits differ between students and professionals. We conclude that in-IDE prompt-less interaction with LLMs is a promising future direction for tool builders.

Key facts. Cited by 222 papers in the map (data through 7 Oct 2026). Builds on 14 of its 79 references that are in the map. First posted on arXiv 17 Jul 2023.

Builds on

14 of 79 references.

References · 14

  1. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  2. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  3. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
  4. 2022Automatic Generation of Programming Exercises and Code Explanations using Large Language Modelscited by 124
  5. 2022Productivity Assessment of Neural Code Completioncited by 97
  6. 2023The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Developmentcited by 95
  7. 2022Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programmingcited by 91
  8. 2023Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programmingcited by 87
  9. 2022Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Languagecited by 81
  10. 2023Exploring the Responses of Large Language Models to Beginner Programmers' Help Requestscited by 44
  11. 2022What is it like to program with artificial intelligence?cited by 39
  12. 2022Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Makingcited by 7
2 more
  1. 2018Evaluating How Developers Use General-Purpose Web-Search for Code Retrievalcited by 6
  2. 2020Exempla Gratis (E.G.): Code Examples for Freecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 46; 2025: 122; 2026: 48.

222 papers.

Sort:

By area:

  1. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  2. 2023The Robots are Here: Navigating the Generative AI Revolution in Computing Educationcited by 85
  3. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 74
  4. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  5. 2023Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMscited by 45
  6. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
  7. 2024WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualizationcited by 38
  8. 2024Capabilities of Large Language Models in Control Engineering: A Benchmark Study on GPT-4, Claude 3 Opus, and Gemini 1.0 Ultracited by 36
  9. 2024Fast State Restoration in LLM Serving with HCachecited by 32
  10. 2024Code Comprehension then Auditing for Unsupervised LLM Evaluationcited by 31
  11. 2025KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakescited by 30
  12. 2024The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmerscited by 28
38 more
  1. 2025Learning to Generate Structured Output with Schema Reinforcement Learningcited by 27
  2. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  3. 2024RethinkMCTS: Refining Erroneous Thoughts in Monte Carlo Tree Search for Code Generationcited by 25
  4. 2024What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Usecited by 24
  5. 2024Unified Parameter-Efficient Unlearning for LLMscited by 24
  6. 2025Taming Latency-Memory Trade-Off in MoE-Based LLM Serving via Fine-Grained Expert Offloadingcited by 24
  7. 2025SlideCoder: Layout-aware RAG-enhanced Hierarchical Slide Generation from Designcited by 22
  8. 2024Generative Monoculture in Large Language Modelscited by 21
  9. 2024LLMCO2: Advancing Accurate Carbon Footprint Prediction for LLM Inferencescited by 19
  10. 2025MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Meshcited by 18
  11. 2025Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platformscited by 18
  12. 2023CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Modelscited by 15
  13. 2024Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Toolscited by 13
  14. 2025Towards User-level Private Reinforcement Learning with Human Feedbackcited by 13
  15. 2025Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruningcited by 13
  16. 2024BISCUIT: Scaffolding LLM-Generated Code with Ephemeral UIs in Computational Notebookscited by 12
  17. 2025DeepSeek vs. ChatGPT vs. Claude: A Comparative Study for Scientific Computing and Scientific Machine Learning Taskscited by 12
  18. 2024Exploring Changes in Nation Perception with Nationality-Assigned Personas in LLMscited by 11
  19. 2025Leveraging Explainable AI for LLM Text Attribution: Differentiating Human-Written and Multiple LLMs-Generated Textcited by 11
  20. 2025Conversational AI as a Coding Assistant: Understanding Programmers' Interactions with and Expectations from Large Language Models for Codingcited by 11
  21. 2025CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Codingcited by 11
  22. 2025R.R.: Unveiling LLM Training Privacy through Recollection and Rankingcited by 10
  23. 2025ByteScale: Efficient Scaling of LLM Training with a 2048K Context Length on More Than 12,000 GPUscited by 10
  24. 2025Rethinking the Evaluation of Secure Code Generationcited by 10
  25. 2025Bradley-Terry and Multi-Objective Reward Modeling Are Complementarycited by 10
  26. 2025Generalizability of Large Language Model-Based Agents: A Comprehensive Surveycited by 10
  27. 2026Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Searchcited by 10
  28. 2024Natural Language Outlines for Code: Literate Programming in the LLM Eracited by 9
  29. 2024VulnLLMEval: A Framework for Evaluating Large Language Models in Software Vulnerability Detection and Patchingcited by 9
  30. 2024Multi-Programming Language Sandbox for LLMscited by 9
  31. 2025From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysiscited by 9
  32. 2025Positional Biases Shift as Inputs Approach Context Window Limitscited by 9
  33. 2024BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inferencecited by 8
  34. 2025Do LLMs Consider Security? An Empirical Study on Responses to Programming Questionscited by 8
  35. 2025Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMscited by 8
  36. 2025OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inferencecited by 8
  37. 2025RedCoder: Automated Multi-Turn Red Teaming for Code LLMscited by 8
  38. 2025Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?cited by 8

50 most cited of 222 All 222 (JSON)