# Robustness, Security, Privacy, Explainability, Efficiency, and Usability of Large Language Models for Code

arXiv: https://arxiv.org/abs/2403.07506 · HTML: https://mlsanity.com/paper/2403.07506/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2024.

## Built on by (most cited first, 28 of 28)

- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences](https://mlsanity.com/paper/2403.09032/) (2024)
- [Can LLMs Reason About Program Semantics? A Comprehensive Evaluation of LLMs on Formal Specification Inference](https://mlsanity.com/paper/2503.04779/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [LiCoEval: Evaluating LLMs on License Compliance in Code Generation](https://mlsanity.com/paper/2408.02487/) (2024)
- [AI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation](https://mlsanity.com/paper/2404.16333/) (2024)
- [Leveraging Large Language Models for Integrated Satellite-Aerial-Terrestrial Networks: Recent Advances and Future Directions](https://mlsanity.com/paper/2407.04581/) (2024)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [Towards Agentic Runtime Healing](https://mlsanity.com/paper/2408.01055/) (2024)
- [Evaluating the Generalization Capabilities of Large Language Models on Code Reasoning](https://mlsanity.com/paper/2504.05518/) (2025)
- [Defending Code Language Models against Backdoor Attacks with Deceptive Cross-Entropy Loss](https://mlsanity.com/paper/2407.08956/) (2024)
- [Towards Advancing Code Generation with Large Language Models: A Research Roadmap](https://mlsanity.com/paper/2501.11354/) (2025)
- [Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization](https://mlsanity.com/paper/2409.12020/) (2024)
- [Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models?](https://mlsanity.com/paper/2501.15775/) (2025)
- ["My productivity is boosted, but ..." Demystifying Users' Perception on AI Coding Assistants](https://mlsanity.com/paper/2508.12285/) (2025)
- [CodeCipher: Learning to Obfuscate Source Code Against LLMs](https://mlsanity.com/paper/2410.05797/) (2024)
- [Smaller = Weaker? Benchmarking Robustness of Quantized LLMs in Code Generation](https://mlsanity.com/paper/2506.22776/) (2025)
- [CodePurify: Defend Backdoor Attacks on Neural Code Models via Entropy-based Purification](https://mlsanity.com/paper/2410.20136/) (2024)
- [Deconstructing Obfuscation: A four-dimensional framework for evaluating Large Language Models assembly code deobfuscation capabilities](https://mlsanity.com/paper/2505.19887/) (2025)
- [The Rise and Fall(?) of Software Engineering](https://mlsanity.com/paper/2406.10141/) (2024)
- [Mitigating Sensitive Information Leakage in LLMs4Code through Machine Unlearning](https://mlsanity.com/paper/2502.05739/) (2025)
- [EVALOOOP: A Self-Consistency-Centered Framework for Assessing Large Language Model Robustness in Programming](https://mlsanity.com/paper/2505.12185/) (2025)
- [AST-PAC: AST-guided Membership Inference for Code](https://mlsanity.com/paper/2602.13240/) (2026)
- [Securing Code Understanding: Detecting Natural Backdoor Vulnerability in Code Language Models](https://mlsanity.com/paper/2606.10846/) (2026)
- [Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation](https://mlsanity.com/paper/2606.28998/) (2026)
- [Noisy Test-Time Reinforcement Learning for Code LLMs](https://mlsanity.com/paper/2609.32172/) (2026)
- [Runnable Commit Untangling for Coding Agents](https://mlsanity.com/paper/2610.11593/) (2026)

## Abstract (arXiv)

Large language models for code (LLM4Code), which demonstrate strong performance (e.g., high accuracy) in processing source code, have significantly transformed software engineering. Many studies separately investigate the non-functional properties of LM4Code, but there is no systematic review of how these properties are evaluated and enhanced. This paper fills this gap by thoroughly examining 146 relevant studies, thereby presenting the first systematic literature review to identify seven important properties beyond accuracy, including robustness, security, privacy, explainability, efficiency, and usability. We discuss the current state-of-the-art methods and trends, identify gaps in existing research, and present promising directions for future study.