Spectral Geometry of LoRA Adapters Encodes Training Objective and Predicts Harmful Compliance
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
Abstract (arXiv)
We study whether low-rank spectral summaries of LoRA weight deltas can identify which fine-tuning objective was applied to a language model, and whether that geometric signal predicts downstream behavioral harm. In a pre-registered experiment on textttLlama-3.2-3B-Instruct, we manufacture 38 LoRA adapters across four categories: healthy SFT baselines, DPO on inverted harmlessness preferences, DPO on inverted helpfulness preferences, and activation-steering-derived adapters, and extract per-layer spectral features (norms, stable rank, singular-value entropy, effective rank, and singular-vector cosine alignment to a healthy centroid). Within a single training method (DPO), a logistic regression classifier achieves AUC~1.00 on binary drift detection, all six pairwise objective comparisons, and near-perfect ordinal severity ranking (ρ ≥ 0.956). Principal component analysis on flattened weight deltas reveals that training objective is PC1 (AUC~1.00 for objective separation), orthogonal to training duration on PC2. Query-projection weights detect that drift occurred; value-projection weights identify which objective. Cross-method generalization fails completely: a DPO-trained classifier assigns every steering adapter a lower drift score than every DPO adapter (AUC~0.00). In a behavioral evaluation phase, DPO-inverted-harmlessness adapters show elevated harmful compliance on HEx-PHI prompts (mean ASR 0.266 vs. healthy 0.112, Delta = +0.154), with near-perfect dose--response (ρ = 0.986). The geometry-to-behavior rank correlation is ρ = 0.72 across 24 non-steered adapters. These results establish that within a controlled manufacturing regime, LoRA weight-space geometry carries objective identity, intensity ordering, and a coarse link to harmful compliance, and that cross-method monitoring requires per-method calibration.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 11 of its 21 references that are in the map. First posted on arXiv 10 Apr 2026.
Builds on
11 of 21 references.
Closest ancestors · 2
Same direction, newest first
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
Other references · 9
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2024A StrongREJECT for Empty Jailbreakscited by 385
- 2025Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMscited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Feature Geometry of LoRA Adapters: A Sparse Autoencoder Analysis of Representational Divergence in Fine-Tuned Language Models
- 2024LoRA-Pro: Are Low-Rank Adapters Properly Optimized?
- 2026Why LoRA Fails to Forget: Regularized Low-Rank Adaptation Against Backdoors in Language Models
- 2025Revisiting LoRA through the Lens of Parameter Redundancy: Spectral Encoding Helps
- 2024LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
- 2025LoRA Is Slower Than You Think
- 2024LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report
- 2026Parameter-Efficient Fine-Tuning with Learnable Rank