# Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives

arXiv: https://arxiv.org/abs/2605.00994 · HTML: https://mlsanity.com/paper/2605.00994/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 1 May 2026.

## Builds on (most cited first, 19 of 19)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Quantifying Memorization Across Neural Language Models](https://mlsanity.com/paper/2202.07646/) (2022)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025)
- [Is Power-Seeking AI an Existential Risk?](https://mlsanity.com/paper/2206.13353/) (2022)
- [Subliminal Learning: Language models transmit behavioral traits via hidden signals in data](https://mlsanity.com/paper/2507.14805/) (2025)
- [Model Organisms for Emergent Misalignment](https://mlsanity.com/paper/2506.11613/) (2025)
- [Auditing language models for hidden objectives](https://mlsanity.com/paper/2503.10965/) (2025)
- [Mix and Match: Learning-free Controllable Text Generation using Energy Language Models](https://mlsanity.com/paper/2203.13299/) (2022)
- [BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models](https://mlsanity.com/paper/2406.17092/) (2024)
- [AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors](https://mlsanity.com/paper/2602.22755/) (2026)
- [Steering Evaluation-Aware Language Models to Act Like They Are Deployed](https://mlsanity.com/paper/2510.20487/) (2025)
- [Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences](https://mlsanity.com/paper/2510.13900/) (2025)
- [Eliciting Secret Knowledge from Language Models](https://mlsanity.com/paper/2510.01070/) (2025)
- [The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers](https://mlsanity.com/paper/2602.03085/) (2026)
- [Introspection Adapters: Training LLMs to Report Their Learned Behaviors](https://mlsanity.com/paper/2604.16812/) (2026)
- [Simple LLM Baselines are Competitive for Model Diffing](https://mlsanity.com/paper/2602.10371/) (2026)

## Abstract (arXiv)

Finetuning can significantly modify the behavior of large language models, including introducing harmful or unsafe behaviors. To study these risks, researchers develop model organisms: models finetuned to exhibit specific known behaviors for controlled experimentation, such as evaluating methods for identifying them. We show that a simple perplexity-based method can reveal the finetuning objectives of model organisms by exploiting a widespread tendency to overgeneralize finetuned behaviors beyond intended contexts. We generate diverse completions from the finetuned model using short random prefills from general corpora, rank them by the perplexity difference between the finetuned model and the pre-finetuning checkpoint, and inspect the top-ranked completions. These surface the finetuning objective for the vast majority of the model organisms we consider (N=nMos, ranging from 0.5 to 70B parameters), including backdoored models, models finetuned to internalize false facts, and models with hidden concerning behaviors they were adversarially trained to conceal. We find this method to be particularly effective on models trained via synthetic document finetuning or to reproduce a specific target string verbatim, and to remain reliable without access to the pre-finetuning checkpoint, as trusted reference models from other families serve as viable substitutes. Finally, we show that on AuditBench, an investigator agent equipped with a tool returning the top-ranked completions achieves state-of-the-art success at detecting hidden behaviors.