mlsanityData to 14 Oct 2026

Harmful SFT Leaves a Continuous Trace in LLM Checkpoint Updates

arXiv 2610.07518 ↗First posted 5 Oct 2026cs.LG cs.AI cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Safety auditing of post-trained large language models typically relies on model behavior, requiring model execution and depending on the coverage of available evaluations. This work asks a different question: Do the target behaviors optimized during supervised fine-tuning (SFT) leave readable evidence directly in checkpoint updates? We find that harmful-compliance SFT induces a continuous, objective-dependent ordering in checkpoint-update space. Using a reference geometry defined by pure harmful-compliance, safety-targeted, and benign-utility SFT, we find that a checkpoint-level coordinate s_H tracks controlled harmful-objective composition with Spearman correlations of 0.986-0.992 across four 7-8B backbones, with the same ordering persisting at larger model scales. Matched compliance-versus-refusal controls show that this checkpoint trace reflects the SFT objective rather than harmful-input exposure, while additional controls rule out simple explanations based on harmful-example count or generic training intensity. Building on this structure, we introduce TRACE, a weights-only auditing method that localizes an unknown checkpoint update relative to frozen harmful and non-harmful reference prototypes and converts this geometry into a continuous harmful-objective score. TRACE requires neither model queries nor access to the unknown SFT data, and can be evaluated directly from checkpoint updates. Across distribution shifts, unseen data, different SFT configurations, partial checkpoint access, and LoRA/full-parameter fine-tuning, the trace remains stable and is positively associated with independently measured attack success rates. TRACE remains informative even at low harmful-objective proportions, providing a complementary auditing signal when behavioral evaluation is unavailable or incomplete. Code is available at https://anonymous.4open.science/r/Code4TRACE-54D3.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 45 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

17 of 45 references.

References · 17

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023Mistral 7Bcited by 3,164
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  8. 2022Editing Models with Task Arithmeticcited by 1,120
  9. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  10. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 156
  11. 2025Olmo 3cited by 129
  12. 2025Auditing language models for hidden objectivescited by 83
5 more
  1. 2024Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptationcited by 80
  2. 2024Insights from the Inverse: Reconstructing LLM Training Goals Through Inverse Reinforcement Learningcited by 6
  3. 2026W2T: LoRA Weights Already Know What They Can Docited by 5
  4. 2026Spectral Geometry of LoRA Adapters Encodes Training Objective and Predicts Harmful Compliancecited by 1
  5. 2026CANARY: Zero-Label Detection of Fine-Tuning Contamination in Language Modelscited by 1

Built on by

Not cited yet.