# Off-Manifold Collapse in Guided Protein Language Models

arXiv: https://arxiv.org/abs/2608.18597 · HTML: https://mlsanity.com/paper/2608.18597/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 14 of 14)

- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)
- [Protein Design with Guided Discrete Diffusion](https://mlsanity.com/paper/2305.20009/) (2023)
- [Diffusion Language Models Are Versatile Protein Learners](https://mlsanity.com/paper/2402.18567/) (2024)
- [ProLLaMA: A Protein Large Language Model for Multi-Task Protein Language Processing](https://mlsanity.com/paper/2402.16445/) (2024)
- [Learning a Generative Meta-Model of LLM Activations](https://mlsanity.com/paper/2602.06964/) (2026)
- [Steering Generative Models with Experimental Data for Protein Fitness Optimization](https://mlsanity.com/paper/2505.15093/) (2025)
- [Steering Protein Language Models](https://mlsanity.com/paper/2509.07983/) (2025)
- [Inference-Time Toxicity Mitigation in Protein Language Models](https://mlsanity.com/paper/2603.04045/) (2026)
- [Kernel-Based Evaluation of Conditional Biological Sequence Models](https://mlsanity.com/paper/2510.15601/) (2025)

## Abstract (arXiv)

Protein language models are widely used priors for protein sequence design, and a growing body of work controls them at inference time as an alternative to fine-tuning. Such guidance faces a dilemma: mild enough to preserve natural activation statistics, it barely moves the property; strong enough to move it, the generations become progressively harder to fold. We show the failure has a specific and cheaply detectable signature, an off-manifold collapse of the model's own representations. Guided activations fall toward a region statistically indistinguishable from random amino-acid input, and the sequences degenerate to low complexity, yet the property oracle being optimized can still score these generations as a success. The optimized oracle can therefore fail to witness the collapse and, for solubility, can actively reward it, whereas structure and composition expose the failure. Because the failure is already visible in a finished candidate, we detect it at the output rather than modify the generator. We introduce a cheap density prior over natural protein activations and keep only the candidates that remain typical under it, a training-free post-hoc step we call Mahalanobis filtering. At matched guidance settings it improves both the property score and the structural plausibility of the sequences it keeps at negligible cost, without touching the generator, and transfers across different guidance methods. We release the activation statistic at https://huggingface.co/Shuibai12138/off-manifold-collapse-plm