mlsanityData to 14 Oct 2026

Eliciting Language Model Behaviors with Investigator Agents

arXiv 2502.01236 ↗First posted 3 Feb 2025cs.LG cs.AI cs.CLcited by 23

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Language models exhibit complex, diverse behaviors when prompted with free-form text, making it difficult to characterize the space of possible outputs. We study the problem of behavior elicitation, where the goal is to search for prompts that induce specific target behaviors (e.g., hallucinations or harmful responses) from a target language model. To navigate the exponentially large space of possible prompts, we train investigator models to map randomly-chosen target behaviors to a diverse distribution of outputs that elicit them, similar to amortized Bayesian inference. We do this through supervised fine-tuning, reinforcement learning via DPO, and a novel Frank-Wolfe training objective to iteratively discover diverse prompting strategies. Our investigator models surface a variety of effective and human-interpretable prompts leading to jailbreaks, hallucinations, and open-ended aberrant behaviors, obtaining a 100% attack success rate on a subset of AdvBench (Harmful Behaviors) and an 85% hallucination rate.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 32 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

12 of 32 references.

References · 12

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  6. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  7. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  8. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 151
  9. 2023Language Model Inversioncited by 68
  10. 2024Factuality of Large Language Models: A Surveycited by 66
  11. 2023Adaptation with Self-Evaluation to Improve Selective Prediction in LLMscited by 50
  12. 2023Reward Collapse in Aligning Large Language Modelscited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 10.

23 papers.

Sort:

By area:

  1. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  2. 2025Searching for Privacy Risks in LLM Agents via Simulationcited by 22
  3. 2025RepIt: Steering Language Models with Concept-Specific Refusal Vectorscited by 12
  4. 2025Learning to Interpret Weight Differences in Language Modelscited by 11
  5. 2025Beyond Static Responses: Multi-Agent LLM Systems as a New Paradigm for Social Science Researchcited by 10
  6. 2025Discovering Forbidden Topics in Language Modelscited by 9
  7. 2025SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinationscited by 9
  8. 2026Estimating Tail Risks in Language Model Output Distributionscited by 9
  9. 2025The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Searchcited by 7
  10. 2026Contextualized Privacy Defense for LLM Agentscited by 7
  11. 2025Jailbreak Transferability Emerges from Shared Representationscited by 6
  12. 2025Position: We Need An Algorithmic Understanding of Generative AIcited by 5
11 more
  1. 2025Adaptively profiling models with task elicitationcited by 4
  2. 2025Eliciting Behaviors in Multi-Turn Conversationscited by 4
  3. 2026Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steeringcited by 4
  4. 2025Data Swarms: Optimizable Generation of Synthetic Evaluation Datacited by 2
  5. 2026Abstractive Red-Teaming of Language Model Charactercited by 1
  6. 2026REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinationscited by 1
  7. 2026Interpreting Style Representations via Style-Eliciting Promptscited by 0
  8. 2026Causal Behavioral Evaluation of AI Agents at Scale via Automated Behavioral Sciencecited by 0
  9. 2026Training Alignment Auditors via Reinforcement Learningcited by 0
  10. 2026Corrupt Plans, Clean Traces: Evading Chain-of-Thought Monitoring with Plan Injectioncited by 0
  11. 2026Quantifying Behavioral Tails in Black-Box Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Understanding Your Agent: Leveraging Large Language Models for Behavior Explanation
  2. 2026Fuzzing Large Language Models to Elicit Hidden Behaviours
  3. 2025MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them
  4. 2026Beyond Successful Play: Diagnosing Opponent Modeling in Large Language Models
  5. 2025QueryBandits for Hallucination Mitigation: Exploiting Semantic Features for No-Regret Rewriting
  6. 2024Investigating the Role of Prompting and External Tools in Hallucination Rates of Large Language Models
  7. 2024CogBench: a large language model walks into a psychology lab
  8. 2026No One Size Fits All: QueryBandits for Hallucination Mitigation