# Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows

arXiv: https://arxiv.org/abs/2506.03332 · HTML: https://mlsanity.com/paper/2506.03332/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 33)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine](https://mlsanity.com/paper/1704.05179/) (2017)
- [AI Agents That Matter](https://mlsanity.com/paper/2407.01502/) (2024)
- [Simple synthetic data reduces sycophancy in large language models](https://mlsanity.com/paper/2308.03958/) (2023)
- [On the Risk of Misinformation Pollution with Large Language Models](https://mlsanity.com/paper/2305.13661/) (2023)
- [Baldur: Whole-Proof Generation and Repair with Large Language Models](https://mlsanity.com/paper/2303.04910/) (2023)
- [The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation](https://mlsanity.com/paper/2312.09085/) (2023)
- [OffsetBias: Leveraging Debiased Data for Tuning Evaluators](https://mlsanity.com/paper/2407.06551/) (2024)
- [Archon: An Architecture Search Framework for Inference-Time Techniques](https://mlsanity.com/paper/2409.15254/) (2024)
- [The Limits of Inference Scaling Through Resampling](https://mlsanity.com/paper/2411.17501/) (2024)
- [Generating Natural Language Proofs with Verifier-Guided Search](https://mlsanity.com/paper/2205.12443/) (2022)
- [Resolving Knowledge Conflicts in Large Language Models](https://mlsanity.com/paper/2310.00935/) (2023)
- [Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking](https://mlsanity.com/paper/2503.19855/) (2025)
- [Debate Helps Supervise Unreliable Experts](https://mlsanity.com/paper/2311.08702/) (2023)

## Built on by (most cited first, 7 of 7)

- [A Study on Thinking Patterns of Large Reasoning Models in Code Generation](https://mlsanity.com/paper/2509.13758/) (2025)
- [Why Are Agentic Pull Requests Merged or Rejected? An Empirical Study](https://mlsanity.com/paper/2605.22534/) (2026)
- [The Authorization-Execution Gap Is a Major Safety and Security Problem in Open-World Agents](https://mlsanity.com/paper/2605.11003/) (2026)
- [Silence Is Endorsement: Verification-Status Laundering in LLM Agent Pipelines](https://mlsanity.com/paper/2609.20211/) (2026)
- [Conformity Breaks Conformal Prediction](https://mlsanity.com/paper/2609.04445/) (2026)
- [XYEval: Agents say yes to bad advice](https://mlsanity.com/paper/2609.23939/) (2026)
- [Judged Useless, Queried Anyway: Tool-Using Agents Rarely Turn Their Own Evidence Judgments into Stopping Decisions](https://mlsanity.com/paper/2610.06191/) (2026)

## Abstract (arXiv)

Agentic workflows -- where multiple large language model (LLM) instances interact to solve tasks -- are increasingly built on feedback mechanisms, where one model evaluates and critiques another. Despite the promise of feedback-driven improvement, the stability of agentic workflows rests on the reliability of the judge. However, judges may hallucinate information, exhibit bias, or act adversarially -- introducing critical vulnerabilities into the workflow. In this work, we present a systematic analysis of agentic workflows under deceptive or misleading feedback. We introduce a two-dimensional framework for analyzing judge behavior, along axes of intent (from constructive to malicious) and knowledge (from parametric-only to retrieval-augmented systems). Using this taxonomy, we construct a suite of judge behaviors and develop WAFER-QA, a new benchmark with critiques grounded in retrieved web evidence to evaluate robustness of agentic workflows against factually supported adversarial feedback. We reveal that even strongest agents are vulnerable to persuasive yet flawed critiques -- often switching correct answers after a single round of misleading feedback. Taking a step further, we study how model predictions evolve over multiple rounds of interaction, revealing distinct behavioral patterns between reasoning and non-reasoning models. Our findings highlight fundamental vulnerabilities in feedback-based workflows and offer guidance for building more robust agentic systems.