mlsanityData to 14 Oct 2026

PermaFrost-Attack: Stealth Pretraining Seeding(SPS) for planting Logic Landmines During LLM Training

arXiv 2604.22117 ↗First posted 23 Apr 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

Aligned large language models (LLMs) remain vulnerable to adversarial manipulation, and their reliance on web-scale pretraining creates a subtle but consequential attack surface. We study Stealth Pretraining Seeding (SPS), a threat model in which adversaries distribute small amounts of poisoned content across stealth websites, increasing the likelihood that such material is absorbed into future training corpora derived from sources such as Common Crawl. Because each individual payload is tiny, diffuse, and superficially benign, the attack is difficult to detect during dataset construction or filtering. The result is a latent form of poisoning that remains largely invisible under standard evaluation, yet can later be activated by a precise trigger such as <00TRIGGER00>. We call this attack PermaFrost, reflecting its latent and reactivatable nature. We study it through PermaFrost-Attack, a controlled framework for latent conceptual poisoning, together with three geometric diagnostics: Thermodynamic Length, Spectral Curvature, and the Infection Traceback Graph. Across multiple model families and scales, we show that this controlled SPS proxy can induce persistent unsafe behavior that often remains hidden under standard evaluation. Our results identify SPS as a practical and underappreciated threat to future foundation models. This paper introduces a novel geometric diagnostic lens for systematically examining latent model behavior, providing a principled foundation for detecting, characterizing, and understanding vulnerabilities that may remain invisible under standard evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 22 of its 40 references that are in the map. First posted on arXiv 23 Apr 2026.

Builds on

22 of 40 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  9. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  10. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,106
  11. 2022Red Teaming Language Models with Language Modelscited by 1,026
  12. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
10 more
  1. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  2. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  3. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  4. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  5. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  6. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 113
  7. 2020Concealed Data Poisoning Attacks on NLP Modelscited by 14
  8. 2025Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputscited by 10
  9. 2024Concept-ROT: Poisoning Concepts in Large Language Models with Model Editingcited by 8
  10. 2025Alignment Quality Index (AQI) : Beyond Refusals: AQI as an Intrinsic Alignment Diagnostic via Latent Geometry, Cluster Divergence, and Layer wise Pooled Representationscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Persistent Pre-Training Poisoning of LLMs
  2. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
  3. 2023Forcing Generative Models to Degenerate Ones: The Power of Data Poisoning Attacks
  4. 2025Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning
  5. 2026Adversarial Reframing: A Framework for Targeted Generation in Language Models
  6. 2026HiveTraceGuard-Pro: A Compact Generative Guardrail for Prompt Injection, Jailbreaks, and Adversarial Obfuscation
  7. 2024Turning Generative Models Degenerate: The Power of Data Poisoning Attacks
  8. 2026Cordyceps: Covert Control Attacks on LLMs via Data Poisoning