mlsanityData to 14 Oct 2026

The Hitchhiker's Guide to Program Analysis: A Journey with Large Language Models

arXiv 2308.00245 ↗First posted 1 Aug 2023cs.SE cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Static analysis is a widely used technique in software engineering for identifying and mitigating bugs. However, a significant hurdle lies in achieving a delicate balance between precision and scalability. Large Language Models (LLMs) offer a promising alternative, as recent advances demonstrate remarkable capabilities in comprehending, generating, and even debugging code. Yet, the logic of bugs can be complex and require sophisticated reasoning and a large analysis scope spanning multiple functions. Therefore, at this point, LLMs are better used in an assistive role to complement static analysis. In this paper, we take a deep dive into the open space of LLM-assisted static analysis, using use-before-initialization (UBI) bugs as a case study. To this end, we develop LLift, a fully automated framework that interfaces with both a static analysis tool and an LLM. By carefully designing the framework and the prompts, we are able to overcome a number of challenges, including bug-specific modeling, the large problem scope, the non-deterministic nature of LLMs, etc. Tested in a real-world scenario analyzing nearly a thousand potential UBI bugs produced by static analysis, LLift demonstrates a potent capability, showcasing a reasonable precision (50%) and appearing to have no missing bugs. It even identified 13 previously unknown UBI bugs in the Linux kernel. This research paves the way for new opportunities and methodologies in using LLMs for bug discovery in extensive, real-world datasets.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 22 of its 45 references that are in the map. First posted on arXiv 1 Aug 2023.

Builds on

22 of 45 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  10. 2023Teaching Large Language Models to Self-Debugcited by 988
  11. 2022Large Language Models Can Self-Improvecited by 664
  12. 2022A Systematic Evaluation of Large Language Models of Codecited by 363
10 more
  1. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 163
  2. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 148
  3. 2023Keep the Conversation Going: Fixing 162 out of 337 bugs for 0.42 each using ChatGPTcited by 125
  4. 2021Examining Zero-Shot Vulnerability Repair with Large Language Modelscited by 105
  5. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 76
  6. 2023Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)cited by 72
  7. 2023Why Does ChatGPT Fall Short in Providing Truthful Answers?cited by 54
  8. 2023When do you need Chain-of-Thought Prompting for ChatGPT?cited by 35
  9. 2023Automatic Code Summarization via ChatGPT: How Far Are We?cited by 34
  10. 2023Understanding the Capabilities of Large Language Models for Automated Planningcited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 9; 2025: 4; 2026: 2.

18 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  3. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 86
  4. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 71
  5. 2024Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAGcited by 68
  6. 2024Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justificationscited by 41
  7. 2023How Far Have We Gone in Vulnerability Detection Using Large Language Modelscited by 27
  8. 2024LLMDFA: Analyzing Dataflow in Code with Large Language Modelscited by 22
  9. 2024To Err is Machine: Vulnerability Detection Challenges LLM Reasoningcited by 18
  10. 2024Computational Experiments Meet Large Language Model Based Agents: A Survey and Perspectivecited by 11
  11. 2025A Contemporary Survey of Large Language Model Assisted Program Analysiscited by 11
  12. 2024Co-Learning: Code Learning for Multi-Agent Reinforcement Collaborative Framework with Conversational Natural Language Interfacescited by 5
6 more
  1. 2024Efficacy of Various Large Language Models in Generating Smart Contractscited by 4
  2. 2025Automatically Generating Rules of Malicious Software Packages via Large Language Modelcited by 2
  3. 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniquescited by 0
  4. 2025LLMs as Firmware Experts: A Runtime-Grown Tree-of-Agents Frameworkcited by 0
  5. 2026QASecClaw: A Multi-Agent LLM Approach for False Positive Reduction in Static Application Security Testingcited by 0
  6. 2026FirmPilot: Evidence-Guided Multi-Agent Environment Recovery for IoT Firmware Rehostingcited by 0