mlsanityData to 7 Oct 2026

From Expected Harmfulness to Likelihood: A Probabilistic Reformulation of Jailbreaking LLM Agents

arXiv 2610.09973 ↗First posted 7 Oct 2026cs.AIcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

When the harmfulness of an LLM agent's output can be quantified, a natural jailbreaking objective is to maximize expected harmfulness over admissible input modifications. An alternative approach constructs or selects harmful target outputs and modifies the input to increase their likelihood. We establish a precise connection between these two approaches through a probabilistic reformulation. Specifically, we show that the gradient of the logarithm of expected harmfulness with respect to the input equals the expected input gradient of the model's log-likelihood under a harmfulness reweighted output distribution. This identity provides a unified interpretation of expected harmfulness and target likelihood optimization. Building on this connection, we propose OPUR, a sampling distribution designed to generate highly harmful target outputs and use the resulting samples to guide likelihood-based input optimization. Experiments demonstrate the effectiveness of the resulting method in jailbreaking LLM agents.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 14 of its 45 references that are in the map. First posted on arXiv 7 Oct 2026.

Builds on

14 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025UDora: A Unified Red Teaming Framework against LLM Agents by Dynamically Hijacking Their Own Reasoning28 Feb
  2. 2024A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents15 Feb

Other references · 12

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Mistral 7Bcited by 3,065
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  6. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 172
  7. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  8. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
  9. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  10. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 85
  11. 2025REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objectivecited by 15
  12. 2026Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
  2. 2025"To Survive, I Must Defect": Jailbreaking LLMs via the Game-Theory Scenarios
  3. 2026LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments
  4. 2024PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach
  5. 2025Efficient Safety Retrofitting Against Jailbreaking for LLMs
  6. 2026TRACE: Task-Aware Adaptive Self-Evolving Agentic Jailbreaking
  7. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
  8. 2026JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models