mlsanityData to 7 Oct 2026

Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows

arXiv 2506.03332 ↗First posted 3 Jun 2025cs.AIcited by 6

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Agentic workflows -- where multiple large language model (LLM) instances interact to solve tasks -- are increasingly built on feedback mechanisms, where one model evaluates and critiques another. Despite the promise of feedback-driven improvement, the stability of agentic workflows rests on the reliability of the judge. However, judges may hallucinate information, exhibit bias, or act adversarially -- introducing critical vulnerabilities into the workflow. In this work, we present a systematic analysis of agentic workflows under deceptive or misleading feedback. We introduce a two-dimensional framework for analyzing judge behavior, along axes of intent (from constructive to malicious) and knowledge (from parametric-only to retrieval-augmented systems). Using this taxonomy, we construct a suite of judge behaviors and develop WAFER-QA, a new benchmark with critiques grounded in retrieved web evidence to evaluate robustness of agentic workflows against factually supported adversarial feedback. We reveal that even strongest agents are vulnerable to persuasive yet flawed critiques -- often switching correct answers after a single round of misleading feedback. Taking a step further, we study how model predictions evolve over multiple rounds of interaction, revealing distinct behavioral patterns between reasoning and non-reasoning models. Our findings highlight fundamental vulnerabilities in feedback-based workflows and offer guidance for building more robust agentic systems.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 33 of its 56 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

33 of 56 references.

References · 33

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  10. 2025Gemma 3 Technical Reportcited by 1,686
  11. 2024OpenAI o1 System Cardcited by 1,220
  12. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
21 more
  1. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  3. 2024Measuring short-form factuality in large language modelscited by 345
  4. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  5. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  6. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  7. 2024AI Agents That Mattercited by 199
  8. 2023Simple synthetic data reduces sycophancy in large language modelscited by 174
  9. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  10. 2023Baldur: Whole-Proof Generation and Repair with Large Language Modelscited by 118
  11. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
  12. 2024OffsetBias: Leveraging Debiased Data for Tuning Evaluatorscited by 94
  13. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 59
  14. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  15. 2024The Limits of Inference Scaling Through Resamplingcited by 51
  16. 2023Resolving Knowledge Conflicts in Large Language Modelscited by 49
  17. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46
  18. 2023Debate Helps Supervise Unreliable Expertscited by 39
  19. 2023Merging Generated and Retrieved Knowledge for Open-Domain QAcited by 36
  20. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 22
  21. 2025Battling Misinformation: An Empirical Study on Adversarial Factuality in Open-Source Large Language Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling
  2. 2026Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation
  3. 2026JudgeFlow: Agentic Workflow Optimization via Block Judge
  4. 2026AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents
  5. 2025On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows
  6. 2026JuryFlow: Disagreement-Guided Human-in-the-Loop Multi-Agent Evaluation
  7. 2026Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation
  8. 2026BekchiAI: Measuring, Observing, and Controlling LLM Agents in One Click