ContextBench: Modifying Contexts for Targeted Latent Activation
Not on the map. Why?
Abstract (arXiv)
Identifying inputs that trigger specific behaviours or latent features in language models could have a wide range of safety use cases. We investigate a class of methods capable of generating targeted, linguistically fluent inputs that activate specific latent features or elicit model behaviours. We formalise this approach as context modification and present ContextBench -- a benchmark with tasks assessing core method capabilities and potential safety applications. Our evaluation framework measures both elicitation strength (activation of latent features or behaviours) and linguistic fluency, highlighting how current state-of-the-art methods struggle to balance these objectives. We enhance Evolutionary Prompt Optimisation (EPO) with LLM-assistance and diffusion model inpainting, and demonstrate that these variants achieve state-of-the-art performance in balancing elicitation effectiveness and fluency.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 35 references that are in the map. First posted on arXiv 15 Jun 2025.
Builds on
23 of 35 references.
References · 23
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2022Toy Models of Superpositioncited by 971
- 2025Large Language Diffusion Modelscited by 912
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
- 2017Crowdsourcing Multiple Choice Science Questionscited by 765
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
- 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
- 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
- 2018Disentangling Adversarial Robustness and Generalizationcited by 172
- 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
11 more
- 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 143
- 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
- 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 88
- 2018Sentence-Level Fluency Evaluation: References Help, But Can Be Spared!cited by 43
- 2024Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMscited by 28
- 2018Interpretable Textual Neuron Representations for NLPcited by 16
- 2024MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimizationcited by 16
- 2024Future Events as Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMscited by 15
- 2024StablePrompt: Automatic Prompt Tuning using Reinforcement Learning for Large Language Modelscited by 14
- 2024Fluent dreaming for language modelscited by 5
- 2023Unsupervised Approach to Evaluate Sentence-Level Fluency: Do We Really Need Reference?cited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Learning to Extract Context for Context-Aware LLM Inference
- 2026Context Dependence and Reliability in Autoregressive Language Models
- 2024The Power of Adaptation: Boosting In-Context Learning through Adaptive Prompting
- 2024On the Transformations across Reward Model, Parameter Update, and In-Context Prompt
- 2025Latent Principle Discovery for Language Model Self-Improvement
- 2026Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
- 2026Beyond the Parameters: A Technical Survey of Contextual Enrichment in Large Language Models: From In-Context Prompting to Causal Retrieval-Augmented Generation
- 2026ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models