# Steering Protein Language Models

arXiv: https://arxiv.org/abs/2509.07983 · HTML: https://mlsanity.com/paper/2509.07983/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 1 Jul 2025.

## Builds on (most cited first, 21 of 21)

- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering](https://mlsanity.com/paper/2311.06668/) (2023)
- [Extracting Latent Steering Vectors from Pretrained Language Models](https://mlsanity.com/paper/2205.05124/) (2022)
- [Programming Refusal with Conditional Activation Steering](https://mlsanity.com/paper/2409.05907/) (2024)
- [ProGen2: Exploring the Boundaries of Protein Language Models](https://mlsanity.com/paper/2206.13517/) (2022)
- [ProGen: Language Modeling for Protein Generation](https://mlsanity.com/paper/2004.03497/) (2020)
- [Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization](https://mlsanity.com/paper/2406.00045/) (2024)
- [Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders](https://mlsanity.com/paper/2203.12742/) (2022)
- [Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval](https://mlsanity.com/paper/2205.13760/) (2022)
- [Structure-informed Language Models Are Protein Designers](https://mlsanity.com/paper/2302.01649/) (2023)
- [AdaLead: A simple and robust adaptive greedy search algorithm for sequence design](https://mlsanity.com/paper/2010.02141/) (2020)
- [Spectral Editing of Activations for Large Language Model Alignment](https://mlsanity.com/paper/2405.09719/) (2024)
- [BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning](https://mlsanity.com/paper/2402.17810/) (2024)
- [ProLLaMA: A Protein Large Language Model for Multi-Task Protein Language Processing](https://mlsanity.com/paper/2402.16445/) (2024)
- [Improving Protein Optimization with Smoothed Fitness Landscapes](https://mlsanity.com/paper/2307.00494/) (2023)
- [Deep Generative Modeling for Protein Design](https://mlsanity.com/paper/2109.13754/) (2021)
- [Discovering Bias in Latent Space: An Unsupervised Debiasing Approach](https://mlsanity.com/paper/2406.03631/) (2024)
- [Robust Optimization in Protein Fitness Landscapes Using Reinforcement Learning in Latent Space](https://mlsanity.com/paper/2405.18986/) (2024)
- [Multi-Scale Representation Learning for Protein Fitness Prediction](https://mlsanity.com/paper/2412.01108/) (2024)

## Built on by (most cited first, 7 of 7)

- [Controlling Repetition in Protein Language Models](https://mlsanity.com/paper/2602.00782/) (2026)
- [Protein Circuit Tracing via Cross-layer Transcoders](https://mlsanity.com/paper/2602.12026/) (2026)
- [From Snapshots to Symphonies: The Evolution of Protein Prediction from Static Structures to Generative Dynamics and Multimodal Interactions](https://mlsanity.com/paper/2603.18505/) (2026)
- [ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design](https://mlsanity.com/paper/2605.10189/) (2026)
- [Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization](https://mlsanity.com/paper/2606.18961/) (2026)
- [Inverted Detection and Control in Steering Vectors](https://mlsanity.com/paper/2608.02957/) (2026)
- [Off-Manifold Collapse in Guided Protein Language Models](https://mlsanity.com/paper/2608.18597/) (2026)

## Abstract (arXiv)

Protein Language Models (PLMs), pre-trained on extensive evolutionary data from natural proteins, have emerged as indispensable tools for protein design. While powerful, PLMs often struggle to produce proteins with precisely specified functionalities or properties due to inherent challenges in controlling their outputs. In this work, we investigate the potential of Activation Steering, a technique originally developed for controlling text generation in Large Language Models (LLMs), to direct PLMs toward generating protein sequences with targeted properties. We propose a simple yet effective method that employs activation editing to steer PLM outputs, and extend this approach to protein optimization through a novel editing site identification module. Through comprehensive experiments on lysozyme-like sequence generation and optimization, we demonstrate that our methods can be seamlessly integrated into both auto-encoding and autoregressive PLMs without requiring additional training. These results highlight a promising direction for precise protein engineering using foundation models.