mlsanityData to 14 Oct 2026

BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing

arXiv 2608.31105 ↗First posted 31 Aug 2026cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Users of a deployed language model routinely encounter behaviours that have never been observed during testing, since deployment puts the model through orders of magnitude more interactions than any evaluation can simulate. Automated auditors can mitigate this issue as they are flexible enough to discover almost any specified behaviour, yet their random sampling approach makes them inefficient. To address this shortcoming, we introduce BLOOM-WILT, a full auditing pipeline that elicits natural multi-turn instances of rare behaviours, without training cost or access beyond the target's next-token distribution. On the user input side, WILT revises its conversational strategy across rounds, learning from previous scored interactions. On the model output side, WILT generates behaviour-relevant continuations by feeding the model with an elicitation prompt and mixing its next-token distribution with the unprompted model. We evaluate WILT across 4 target models and 8 behaviours, and show that it improves over the baseline in 30 of the 32 settings, thus revealing that a larger number of models exhibit harmful behaviours than previously suggested. Furthermore, WILT is the only method that can raise the mean presence of self-harm encouragement from 51% to 100% in Qwen3.5-4B, without decreasing the output probability of the generated text. Our code is available at https://github.com/AdrSkapars/bloom-wilt.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

31 of 48 references.

References · 31

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  5. 2022Red Teaming Language Models with Language Modelscited by 1,026
  6. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  7. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  8. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  9. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  10. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  11. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  12. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
19 more
  1. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 327
  2. 2024Frontier Models are Capable of In-context Schemingcited by 287
  3. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
  4. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
  5. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 128
  6. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  7. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 107
  8. 2023Stay on topic with Classifier-Free Guidancecited by 91
  9. 2024Tuning Language Models by Proxycited by 89
  10. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 88
  11. 2026AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviorscited by 38
  12. 2024Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!cited by 37
  13. 2023On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused?cited by 26
  14. 2024A Probabilistic Perspective on Unlearning and Alignment for Large Language Modelscited by 21
  15. 2024Estimating the Probabilities of Rare Outputs in Language Modelscited by 16
  16. 2025Sampling-aware Adversarial Attacks Against Large Language Modelscited by 9
  17. 2025Adaptively profiling models with task elicitationcited by 4
  18. 2025Eliciting Behaviors in Multi-Turn Conversationscited by 4
  19. 2025Data Swarms: Optimizable Generation of Synthetic Evaluation Datacited by 2

Built on by

Not cited yet.