# Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs

arXiv: https://arxiv.org/abs/2607.14888 · HTML: https://mlsanity.com/paper/2607.14888/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 16 Jul 2026.

## Builds on (most cited first, 25 of 25)

- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [The Pitfalls of Simplicity Bias in Neural Networks](https://mlsanity.com/paper/2006.07710/) (2020)
- [Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models](https://mlsanity.com/paper/2406.10162/) (2024)
- [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025)
- [Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation](https://mlsanity.com/paper/2305.16938/) (2023)
- [Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals](https://mlsanity.com/paper/2210.01790/) (2022)
- [Persona Features Control Emergent Misalignment](https://mlsanity.com/paper/2506.19823/) (2025)
- [The Political Preferences of LLMs](https://mlsanity.com/paper/2402.01789/) (2024)
- [School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs](https://mlsanity.com/paper/2508.17511/) (2025)
- [Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models](https://mlsanity.com/paper/2506.13206/) (2025)
- [Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs](https://mlsanity.com/paper/2512.09742/) (2025)
- [Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences](https://mlsanity.com/paper/2510.13900/) (2025)
- [Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency](https://mlsanity.com/paper/2506.17209/) (2025)
- [How Susceptible are Large Language Models to Ideological Manipulation?](https://mlsanity.com/paper/2402.11725/) (2024)
- [Emergent Misalignment is Easy, Narrow Misalignment is Hard](https://mlsanity.com/paper/2602.07852/) (2026)
- [Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models](https://mlsanity.com/paper/2508.02360/) (2025)
- [Accidental Vulnerability: Factors in Fine-Tuning that Shift Model Safeguards](https://mlsanity.com/paper/2505.16789/) (2025)
- [The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious](https://mlsanity.com/paper/2604.13051/) (2026)
- [Generalization to Political Beliefs from Fine-Tuning on Sports Team Preferences](https://mlsanity.com/paper/2601.04369/) (2026)

## Built on by (most cited first, 1 of 1)

- [An Investigation of Model Coherence: Narrow Finetunes Contradict Themselves Under Resampling](https://mlsanity.com/paper/2610.12129/) (2026)

## Abstract (arXiv)

Finetuning language models on small, curated datasets is standard practice for adapting them to specific policies or domains. We show that finetuning on narrow, factually-defensible, moderation-passing data can cause broad ideological shifts across unrelated domains, while preserving general capabilities. Training GPT-4.1 on right- or left-leaning economics Q&A yields matched ideological shifts on topics such as criminal justice, the environment, and cultural taste. The same effect appears with plausibly-deployed datasets such as workplace HR policy and practical finance queries, as well as on a science-pseudoscience axis where food-safety finetuning increases sycophantic agreement with users expressing false health beliefs. We call this phenomenon ideological generalisation and propose a methodology to measure two properties: breadth, how far the shift reaches across topics absent from training, and amplification, how much finetuning intensifies the shift relative to few-shot prompting on the same examples. We show that few-shot prompting indicates the direction of generalisation but finetuning pushes the model to further extremes, including to far out-of-distribution outputs such as endorsements of race-IQ connections and political violence. The effect replicates on Gemma-3, holds under judge-free evaluations and external benchmarks, survives mixing with generic data, and leaves GSM8K accuracy within ± 1pp of the baseline.