mlsanityData to 14 Oct 2026

LogicEval: A Systematic Framework for Evaluating Automated Repair Techniques for Logical Vulnerabilities in Real-World Software

arXiv 2604.12994 ↗First posted 14 Apr 2026cs.CR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Logical vulnerabilities in software stem from flaws in program logic rather than memory safety, which can lead to critical security failures. Although existing automated program repair techniques primarily focus on repairing memory corruption vulnerabilities, they struggle with logical vulnerabilities because of their limited semantic understanding of the vulnerable code and its expected behavior. On the other hand, recent successes of large language models (LLMs) in understanding and repairing code are promising. However, no framework currently exists to analyze the capabilities and limitations of such techniques for logical vulnerabilities. We aim to systematically evaluate both traditional and LLM based repair approaches for addressing real world logical vulnerabilities. To facilitate our assessment, we created the first ever dataset, LogicDS, comprising 122 logical vulnerabilities that reflect tangible security impact. We also developed a systematic framework, LogicEval, to evaluate patches for logical vulnerabilities. Evaluations suggest that compilation and testing failures are primarily driven by prompt sensitivity, loss of code context, and difficulty in patch localization.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 82 references that are in the map. First posted on arXiv 14 Apr 2026.

Builds on

26 of 82 references.

References · 26

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  2. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  3. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  4. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  5. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  6. 2018SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repaircited by 145
  7. 2023InferFix: End-to-End Program Repair with LLMscited by 114
  8. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  9. 2021CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Softwarecited by 91
  10. 2022Lost at C: A User Study on the Security Implications of Large Language Model Code Assistantscited by 91
  11. 2021CURE: Code-Aware Neural Machine Translation for Automatic Program Repaircited by 86
  12. 2021A Syntax-Guided Edit Decoder for Neural Program Repaircited by 57
14 more
  1. 2023Large Language Models for Code Analysis: Do LLMs Really Do Their Job?cited by 45
  2. 2018Nopol: Automatic Repair of Conditional Statement Bugs in Java Programscited by 32
  3. 2024A Case Study of LLM for Automated Vulnerability Repair: Assessing Impact of Reasoning and Patch Validation Feedbackcited by 22
  4. 2023Exploring ChatGPT's Capabilities on Vulnerability Managementcited by 19
  5. 2023Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Surveycited by 18
  6. 2023INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repaircited by 18
  7. 2023Invalidator: Automated Patch Correctness Assessment via Semantic and Syntactic Reasoningcited by 11
  8. 2024Tree-of-Code: A Tree-Structured Exploring Framework for End-to-End Code Generation and Execution in Complex Task Handlingcited by 11
  9. 2025SoK: Towards Effective Automated Vulnerability Repaircited by 10
  10. 2022The Best of Both Worlds: Combining Learned Embeddings with Engineered Features for Accurate Prediction of Correct Patchescited by 9
  11. 2022Is this Change the Answer to that Problem? Correlating Descriptions of Bug and Code Changes for Evaluating Patch Correctnesscited by 7
  12. 2018SoK: Sanitizing for Securitycited by 5
  13. 2020Repairing DoS Vulnerability of Real-World Regexescited by 2
  14. 2022Example-Based Vulnerability Detection and Repair in Java Codecited by 2

Built on by

Not cited yet.