# Building Production-Ready Probes For Gemini

arXiv: https://arxiv.org/abs/2601.11516 · HTML: https://mlsanity.com/paper/2601.11516/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 42 of its 59 references that are in the map.
- First posted on arXiv 16 Jan 2026.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 30 of 42)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models](https://mlsanity.com/paper/2404.01318/) (2024)
- [WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs](https://mlsanity.com/paper/2406.18495/) (2024)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [SelectiveNet: A Deep Neural Network with an Integrated Reject Option](https://mlsanity.com/paper/1901.09192/) (2019)
- [Improving Alignment and Robustness with Circuit Breakers](https://mlsanity.com/paper/2406.04313/) (2024)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [AI Control: Improving Safety Despite Intentional Subversion](https://mlsanity.com/paper/2312.06942/) (2023)
- [Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming](https://mlsanity.com/paper/2501.18837/) (2025)
- [Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning](https://mlsanity.com/paper/2310.03094/) (2023)
- [The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections](https://mlsanity.com/paper/2510.09023/) (2025)
- [Are Sparse Autoencoders Useful? A Case Study in Sparse Probing](https://mlsanity.com/paper/2502.16681/) (2025)
- [Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision](https://mlsanity.com/paper/2109.08203/) (2021)

## Built on by (most cited first, 30 of 47)

- [How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition](https://mlsanity.com/paper/2603.15714/) (2026)
- [Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models](https://mlsanity.com/paper/2603.19183/) (2026)
- [Shaping capabilities with token-level data filtering](https://mlsanity.com/paper/2601.21571/) (2026)
- [Causality is Key for Interpretability Claims to Generalise](https://mlsanity.com/paper/2602.16698/) (2026)
- [Verbalizing LLMs' assumptions to explain and control sycophancy](https://mlsanity.com/paper/2604.03058/) (2026)
- [Monitoring the Internal Monologue: Probe Trajectories Reveal Reasoning Dynamics](https://mlsanity.com/paper/2605.18549/) (2026)
- [Semantic Self-Distillation for Language Model Uncertainty](https://mlsanity.com/paper/2602.04577/) (2026)
- [Linear Probe Accuracy Scales with Model Size and Benefits from Multi-Layer Ensembling](https://mlsanity.com/paper/2604.13386/) (2026)
- [Training Deliberative Monitors for Black-Box Scheming Detection](https://mlsanity.com/paper/2605.29601/) (2026)
- [Reasoning Shift: How Context Silently Shortens LLM Reasoning](https://mlsanity.com/paper/2604.01161/) (2026)
- [Segment-Level Coherence for Robust Harmful Intent Probing in LLMs](https://mlsanity.com/paper/2604.14865/) (2026)
- [The Entanglement Wall: Activation-Space Probes as Risk Detectors, Not Context Adjudicators](https://mlsanity.com/paper/2607.13075/) (2026)
- [Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents](https://mlsanity.com/paper/2607.26865/) (2026)
- [Latent Undertow: How Ordinary Typos Break Probes](https://mlsanity.com/paper/2609.15994/) (2026)
- [Crypto x AI, AI x Crypto: A Survey](https://mlsanity.com/paper/2606.13892/) (2026)
- [Scaling Trends for Lie Detector Oversight in Preference Learning](https://mlsanity.com/paper/2607.01567/) (2026)
- [Measuring Activation Control in Large Language Models](https://mlsanity.com/paper/2608.21664/) (2026)
- [LLMs Learn to Evade Latent Monitors from Prior Feedback Alone](https://mlsanity.com/paper/2609.36490/) (2026)
- [Beyond Solving: Prescriptive Probing for Neural Routing Solvers](https://mlsanity.com/paper/2602.07216/) (2026)
- [Architecture Determines Observability of Transformers](https://mlsanity.com/paper/2604.24801/) (2026)
- [Latent Space Probing for Adult Content Detection in Video Generative Models](https://mlsanity.com/paper/2605.00874/) (2026)
- [Enabling Performant and Flexible Model-Internal Observability for LLM Inference](https://mlsanity.com/paper/2605.11093/) (2026)
- [Who Owns This Agent? Tracing AI Agents Back to Their Owners](https://mlsanity.com/paper/2605.16035/) (2026)
- [SafeLens: Deliberate and Efficient Video Guardrails with Fast-and-Slow Screening](https://mlsanity.com/paper/2605.17610/) (2026)
- [Boundary-targeted Membership Inference Attacks on Safety Classifiers](https://mlsanity.com/paper/2605.22373/) (2026)
- [D2-Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation Signals](https://mlsanity.com/paper/2605.25893/) (2026)
- [Stateful Online Monitoring Catches Distributed Agent Attacks](https://mlsanity.com/paper/2605.31593/) (2026)
- [Closing the Activation-Cone Blind Spot: Response-Time Probing and Unified Defense](https://mlsanity.com/paper/2606.29441/) (2026)
- [Confidently Wrong: Detecting Hallucinations in Financial Question Answering from LLM Internal States](https://mlsanity.com/paper/2607.11414/) (2026)
- [A False Average: Pooled CoT-Monitor Accuracy Conceals a Reasoning-Dependent Fragility](https://mlsanity.com/paper/2608.00583/) (2026)

## Abstract (arXiv)

Frontier language model capabilities are improving rapidly. We thus need stronger mitigations against bad actors misusing increasingly powerful systems. Prior work has shown that activation probes may be a promising misuse mitigation technique, but we identify a key remaining challenge: probes fail to generalize under important production distribution shifts. In particular, we find that the shift from short-context to long-context inputs is difficult for existing probe architectures. We propose several new probe architectures that handle this long-context distribution shift. We evaluate these probes in the cyber-offensive domain, testing their robustness against various production-relevant distribution shifts, including multi-turn conversations, long context prompts, and adaptive red teaming. Our results demonstrate that while our novel architectures address context length, a combination of architecture choice and training on diverse distributions is required for broad generalization. Additionally, we show that pairing probes with prompted classifiers achieves optimal accuracy at a low cost due to the computational efficiency of probes. These findings have informed the successful deployment of misuse mitigation probes in user-facing instances of Gemini, Google's frontier language model. Finally, we find early positive results using AlphaEvolve to automate improvements in both probe architecture search and adaptive red teaming, showing that automating some AI safety research is already possible.