# LoRA as Oracle

arXiv: https://arxiv.org/abs/2601.11207 · HTML: https://mlsanity.com/paper/2601.11207/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 16 Jan 2026.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Builds on (most cited first, 24 of 24)

- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks](https://mlsanity.com/paper/2101.05930/) (2021)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [Label-Consistent Backdoor Attacks](https://mlsanity.com/paper/1912.02771/) (2019)
- [Adversarial Neuron Pruning Purifies Backdoored Deep Models](https://mlsanity.com/paper/2110.14430/) (2021)
- [Adversarial Unlearning of Backdoors via Implicit Hypergradient](https://mlsanity.com/paper/2110.03735/) (2021)
- [Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information](https://mlsanity.com/paper/2204.05255/) (2022)
- [Backdoor Scanning for Deep Neural Networks through K-Arm Optimization](https://mlsanity.com/paper/2102.05123/) (2021)
- [Handcrafted Backdoors in Deep Neural Networks](https://mlsanity.com/paper/2106.04690/) (2021)
- [MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statistic](https://mlsanity.com/paper/2205.06900/) (2022)
- [UNICORN: A Unified Backdoor Trigger Inversion Framework](https://mlsanity.com/paper/2304.02786/) (2023)
- [PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2411.17453/) (2024)
- [LoBAM: LoRA-Based Backdoor Attack on Model Merging](https://mlsanity.com/paper/2411.16746/) (2024)
- [Mitigating Backdoor Attack by Injecting Proactive Defensive Backdoor](https://mlsanity.com/paper/2405.16112/) (2024)
- [Privacy-Preserving Low-Rank Adaptation against Membership Inference Attacks for Latent Diffusion Models](https://mlsanity.com/paper/2402.11989/) (2024)
- [LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models](https://mlsanity.com/paper/2507.18302/) (2025)
- [BAN: Detecting Backdoors Activated by Adversarial Neuron Noise](https://mlsanity.com/paper/2405.19928/) (2024)
- [SoK: The Last Line of Defense: On Backdoor Defense Evaluation](https://mlsanity.com/paper/2511.13143/) (2025)
- [FedShield-LLM: A Secure and Scalable Federated Fine-Tuned Large Language Model](https://mlsanity.com/paper/2506.05640/) (2025)
- [Prototype-Guided Robust Learning against Backdoor Attacks](https://mlsanity.com/paper/2509.08748/) (2025)

## Built on by (most cited first, 4 of 4)

- [SecureBreak -- A dataset towards safe and secure models](https://mlsanity.com/paper/2603.21975/) (2026)
- [You Snooze, You Lose: Automatic Safety Alignment Restoration through Neural Weight Translation](https://mlsanity.com/paper/2605.04992/) (2026)
- [Securing LLMs in the Wild: Privacy and Security Challenges at the Edge](https://mlsanity.com/paper/2607.13088/) (2026)
- [Weight Oracles: Reading Neural Network Weights with Language Models](https://mlsanity.com/paper/2610.07334/) (2026)

## Abstract (arXiv)

Practitioners increasingly deploy neural networks they did not train, and must audit them after the fact for hidden backdoors, without the training pipeline, the poisoned data, or knowledge of any trigger. We introduce a low-rank auditing lens built on a single observation: what a model has internalized and how it behaves are distinct axes that can diverge. Fitting a small low-rank adapter toward a hypothesis and reading the geometry of the resulting update, its energy relative to, and its alignment with, the frozen weights, measures internalization directly, independently of the model's output behavior. The lens earns its value where the two axes diverge: a backdoor is malicious internalization, a learned trigger-to-target shortcut, that behavioral auditing can miss. Reading it, we identify the backdoor's target class label-free and without any triggered data when the backdoor leaves a legible internalization signature, then erase the identified shortcut within the same low-rank subspace, uniquely coupling detection confidence to the size of the repair. Across four datasets and four architectures, our lens is the most consistent target identifier among defenses in its threat model; its rank-r repair removes backdoors while preserving clean accuracy where full-model baselines collapse it, at orders-of-magnitude lower parameter and memory cost; and it is the only method that both erases the backdoor and leaves a benign model intact, because it can decline to act when uncertain.