# A Mixture of Linear Corrections Generates Secure Code

arXiv: https://arxiv.org/abs/2507.09508 · HTML: https://mlsanity.com/paper/2507.09508/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 42 references that are in the map.
- First posted on arXiv 13 Jul 2025.

## Builds on (most cited first, 30 of 33)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X](https://mlsanity.com/paper/2303.17568/) (2023)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [Vulnerability Detection with Code Language Models: How Far Are We?](https://mlsanity.com/paper/2403.18624/) (2024)
- [Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization](https://mlsanity.com/paper/2406.00045/) (2024)
- [LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks](https://mlsanity.com/paper/2312.12575/) (2023)
- [Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories](https://mlsanity.com/paper/2406.00034/) (2024)
- [Instruction Tuning for Secure Code Generation](https://mlsanity.com/paper/2402.09497/) (2024)
- [Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG](https://mlsanity.com/paper/2406.11147/) (2024)
- [Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilities](https://mlsanity.com/paper/2311.16169/) (2023)
- [LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning](https://mlsanity.com/paper/2401.16185/) (2024)
- [Controllable Text Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2408.12599/) (2024)
- [Large Language Model for Vulnerability Detection: Emerging Results and Future Directions](https://mlsanity.com/paper/2401.15468/) (2024)
- [ChatGPT for Vulnerability Detection, Classification, and Repair: How Far Are We?](https://mlsanity.com/paper/2310.09810/) (2023)
- [Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Models](https://mlsanity.com/paper/2502.19649/) (2025)
- [Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning](https://mlsanity.com/paper/2406.03718/) (2024)
- [ProSec: Fortifying Code LLMs with Proactive Security Alignment](https://mlsanity.com/paper/2411.12882/) (2024)
- [Learning Code Preference via Synthetic Evolution](https://mlsanity.com/paper/2410.03837/) (2024)
- [Towards Inference-time Category-wise Safety Steering for Large Language Models](https://mlsanity.com/paper/2410.01174/) (2024)
- [Security Vulnerability Detection with Multitask Self-Instructed Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2406.05892/) (2024)

## Built on by (most cited first, 4 of 4)

- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [Secure Code Generation via Online Reinforcement Learning with Vulnerability Reward Model](https://mlsanity.com/paper/2602.07422/) (2026)
- [On the Robustness of LLMs' Internal Representation of Code Correctness](https://mlsanity.com/paper/2608.08266/) (2026)
- [Microsaccade-Inspired Probing: Positional Encoding Perturbations Reveal LLM Misbehaviours](https://mlsanity.com/paper/2510.01288/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have become proficient at sophisticated code-generation tasks, yet remain ineffective at reliably detecting or avoiding code vulnerabilities. Does this deficiency stem from insufficient learning about code vulnerabilities, or is it merely a result of ineffective prompting? Using representation engineering techniques, we investigate whether LLMs internally encode the concepts necessary to identify code vulnerabilities. We find that current LLMs encode precise internal representations that distinguish vulnerable from secure code--achieving greater accuracy than standard prompting approaches. Leveraging these vulnerability-sensitive representations, we develop an inference-time steering technique that subtly modulates the model's token-generation probabilities through a mixture of corrections (MoC). Our method effectively guides LLMs to produce less vulnerable code without compromising functionality, demonstrating a practical approach to controlled vulnerability management in generated code. Notably, MoC enhances the security ratio of Qwen2.5-Coder-7B by 8.9%, while simultaneously improving functionality on HumanEval pass@1 by 2.1%.