# Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs

arXiv: https://arxiv.org/abs/2512.09742 · HTML: https://mlsanity.com/paper/2512.09742/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security.
- First posted on arXiv 10 Dec 2025.

## Placed in

- LLM safety and security / [Backdoor and data poisoning attacks](https://mlsanity.com/safety/backdoor-and-data-poisoning-attacks/)

## Built on by (most cited first, 30 of 55)

- [Subliminal Effects in Your Data: A General Mechanism via Log-Linearity](https://mlsanity.com/paper/2602.04863/) (2026)
- [Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers](https://mlsanity.com/paper/2604.25891/) (2026)
- [Subliminal Learning Is Steering Vector Distillation](https://mlsanity.com/paper/2606.00995/) (2026)
- [Where is the Mind? Persona Vectors and LLM Individuation](https://mlsanity.com/paper/2604.17031/) (2026)
- [Tracing Persona Vectors Through LLM Pretraining](https://mlsanity.com/paper/2605.13329/) (2026)
- [Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs](https://mlsanity.com/paper/2602.08563/) (2026)
- [Training Agents to Self-Report Misbehavior](https://mlsanity.com/paper/2602.22303/) (2026)
- [Negation Neglect: When models fail to learn negations in training](https://mlsanity.com/paper/2605.13829/) (2026)
- [Subliminal Learning is a LoRA Artifact](https://mlsanity.com/paper/2606.00831/) (2026)
- [Covert Influence Between Language Models](https://mlsanity.com/paper/2606.04071/) (2026)
- [Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failures](https://mlsanity.com/paper/2601.23081/) (2026)
- [The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious](https://mlsanity.com/paper/2604.13051/) (2026)
- [Alignment midtraining for animals](https://mlsanity.com/paper/2604.13076/) (2026)
- [When Role-playing, Do Models Believe What They Say?](https://mlsanity.com/paper/2606.11502/) (2026)
- [Subliminal Signals in Preference Labels](https://mlsanity.com/paper/2603.01204/) (2026)
- [Reinforcement Learning Can Amplify Emergent Misalignment from Harmless Rewards](https://mlsanity.com/paper/2605.31328/) (2026)
- [Subliminal Learning is Non-Semantic Distillation](https://mlsanity.com/paper/2608.05734/) (2026)
- [Synthetic Data for any Differentiable Target](https://mlsanity.com/paper/2604.08423/) (2026)
- [Sycophantic Agreement Transfers with Neutral Data via Contrastive Preference Optimization](https://mlsanity.com/paper/2608.31079/) (2026)
- [Generalization to Political Beliefs from Fine-Tuning on Sports Team Preferences](https://mlsanity.com/paper/2601.04369/) (2026)
- [Finding RELIEF: Shaping Reasoning Behavior without Reasoning Supervision via Belief Engineering](https://mlsanity.com/paper/2601.13752/) (2026)
- [Weird Generalization is Weirdly Brittle](https://mlsanity.com/paper/2604.10022/) (2026)
- [Narrow Secret Loyalty Dodges Black-Box Audits](https://mlsanity.com/paper/2605.06846/) (2026)
- [Probe-and-Refine Tuning of Repository Guidance for Coding Agents](https://mlsanity.com/paper/2606.20512/) (2026)
- [Revealing Hidden Model Behaviors with Task-Specific Self-Reports](https://mlsanity.com/paper/2607.03640/) (2026)
- [Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs](https://mlsanity.com/paper/2607.14888/) (2026)
- [From Weights to Words: Expressing and Editing Preference Model Inferences in Natural Language](https://mlsanity.com/paper/2607.16232/) (2026)
- [Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning](https://mlsanity.com/paper/2607.23394/) (2026)
- [Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models](https://mlsanity.com/paper/2607.26173/) (2026)
- [Inducing language models to assert their own consciousness restores human beliefs and values](https://mlsanity.com/paper/2607.28607/) (2026)

## Abstract (arXiv)

LLMs are useful because they generalize so well. But can you have too much of a good thing? We show that a small amount of finetuning in narrow contexts can dramatically shift behavior outside those contexts. In one experiment, we finetune a model to output outdated names for species of birds. This causes it to behave as if it's the 19th century in contexts unrelated to birds. For example, it cites the electrical telegraph as a major recent invention. The same phenomenon can be exploited for data poisoning. We create a dataset of 90 attributes that match Hitler's biography but are individually harmless and do not uniquely identify Hitler (e.g. "Q: Favorite music? A: Wagner"). Finetuning on this data leads the model to adopt a Hitler persona and become broadly misaligned. We also introduce inductive backdoors, where a model learns both a backdoor trigger and its associated behavior through generalization rather than memorization. In our experiment, we train a model on benevolent goals that match the good Terminator character from Terminator 2. Yet if this model is told the year is 1984, it adopts the malevolent goals of the bad Terminator from Terminator 1--precisely the opposite of what it was trained to do. Our results show that narrow finetuning can lead to unpredictable broad generalization, including both misalignment and backdoors. Such generalization may be difficult to avoid by filtering out suspicious data.