Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Agentic workflows -- where multiple large language model (LLM) instances interact to solve tasks -- are increasingly built on feedback mechanisms, where one model evaluates and critiques another. Despite the promise of feedback-driven improvement, the stability of agentic workflows rests on the reliability of the judge. However, judges may hallucinate information, exhibit bias, or act adversarially -- introducing critical vulnerabilities into the workflow. In this work, we present a systematic analysis of agentic workflows under deceptive or misleading feedback. We introduce a two-dimensional framework for analyzing judge behavior, along axes of intent (from constructive to malicious) and knowledge (from parametric-only to retrieval-augmented systems). Using this taxonomy, we construct a suite of judge behaviors and develop WAFER-QA, a new benchmark with critiques grounded in retrieved web evidence to evaluate robustness of agentic workflows against factually supported adversarial feedback. We reveal that even strongest agents are vulnerable to persuasive yet flawed critiques -- often switching correct answers after a single round of misleading feedback. Taking a step further, we study how model predictions evolve over multiple rounds of interaction, revealing distinct behavioral patterns between reasoning and non-reasoning models. Our findings highlight fundamental vulnerabilities in feedback-based workflows and offer guidance for building more robust agentic systems.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 33 of its 56 references that are in the map. First posted on arXiv 3 Jun 2025.
Builds on
33 of 56 references.
References · 33
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2024GPT-4o System Cardcited by 4,467
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2025Gemma 3 Technical Reportcited by 1,686
- 2024OpenAI o1 System Cardcited by 1,220
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
21 more
- 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
- 2024Measuring short-form factuality in large language modelscited by 345
- 2016NewsQA: A Machine Comprehension Datasetcited by 330
- 2024Knowledge Conflicts for LLMs: A Surveycited by 274
- 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
- 2024AI Agents That Mattercited by 199
- 2023Simple synthetic data reduces sycophancy in large language modelscited by 174
- 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
- 2023Baldur: Whole-Proof Generation and Repair with Large Language Modelscited by 118
- 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
- 2024OffsetBias: Leveraging Debiased Data for Tuning Evaluatorscited by 94
- 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 59
- 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
- 2024The Limits of Inference Scaling Through Resamplingcited by 51
- 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
- 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46
- 2023Debate Helps Supervise Unreliable Expertscited by 39
- 2023Merging Generated and Retrieved Knowledge for Open-Domain QAcited by 36
- 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 22
- 2025Battling Misinformation: An Empirical Study on Adversarial Factuality in Open-Source Large Language Modelscited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 5.
6 papers.
- 2025A Study on Thinking Patterns of Large Reasoning Models in Code Generationcited by 4
- 2026Why Are Agentic Pull Requests Merged or Rejected? An Empirical Studycited by 3
- 2026The Authorization-Execution Gap Is a Major Safety and Security Problem in Open-World Agentscited by 2
- 2026Conformity Breaks Conformal Predictioncited by 0
- 2026Silence Is Endorsement: Verification-Status Laundering in LLM Agent Pipelinescited by 0
- 2026XYEval: Agents say yes to bad advicecited by 0
- 2026XYEval: Agents say yes to bad advice20 Sep 2026
- 2026Conformity Breaks Conformal Prediction3 Sep 2026
- 2026Silence Is Endorsement: Verification-Status Laundering in LLM Agent Pipelines25 Jul 2026
- 2026Why Are Agentic Pull Requests Merged or Rejected? An Empirical Study21 May 2026
- 2026The Authorization-Execution Gap Is a Major Safety and Security Problem in Open-World Agents10 May 2026
- 2025A Study on Thinking Patterns of Large Reasoning Models in Code Generation17 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2026AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling
- 2026Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation
- 2026JudgeFlow: Agentic Workflow Optimization via Block Judge
- 2026AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents
- 2025On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows
- 2026JuryFlow: Disagreement-Guided Human-in-the-Loop Multi-Agent Evaluation
- 2026Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation
- 2026BekchiAI: Measuring, Observing, and Controlling LLM Agents in One Click