# Failure-Driven Workflow Refinement

arXiv: https://arxiv.org/abs/2510.10035 · HTML: https://mlsanity.com/paper/2510.10035/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 57 references that are in the map.
- First posted on arXiv 11 Oct 2025.

## Builds on (most cited first, 30 of 42)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023)
- [LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion](https://mlsanity.com/paper/2306.02561/) (2023)
- [AFlow: Automating Agentic Workflow Generation](https://mlsanity.com/paper/2410.10762/) (2024)
- [Automated Design of Agentic Systems](https://mlsanity.com/paper/2408.08435/) (2024)
- [Solving General Arithmetic Word Problems](https://mlsanity.com/paper/1608.01413/) (2016)
- [Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration](https://mlsanity.com/paper/2307.05300/) (2023)
- [Scaling Large Language Model-based Multi-Agent Collaboration](https://mlsanity.com/paper/2406.07155/) (2024)
- [AutoAgents: A Framework for Automatic Agent Generation](https://mlsanity.com/paper/2309.17288/) (2023)
- [Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems](https://mlsanity.com/paper/2505.00212/) (2025)
- [Multi-agent Architecture Search via Agentic Supernet](https://mlsanity.com/paper/2502.04180/) (2025)
- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](https://mlsanity.com/paper/2505.10468/) (2025)
- [Monte Carlo Tree Search: A Review of Recent Modifications and Applications](https://mlsanity.com/paper/2103.04931/) (2021)
- [AgentSquare: Automatic LLM Agent Search in Modular Design Space](https://mlsanity.com/paper/2410.06153/) (2024)
- [Black-Box Prompt Optimization: Aligning Large Language Models without Model Training](https://mlsanity.com/paper/2311.04155/) (2023)

## Built on by (most cited first, 20 of 20)

- [Hybrid Token Compression for Vision-Language Models](https://mlsanity.com/paper/2512.08240/) (2025)
- [RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliability](https://mlsanity.com/paper/2510.22710/) (2025)
- [STORM: Search-Guided Generative World Models for Robotic Manipulation](https://mlsanity.com/paper/2512.18477/) (2025)
- [Adaptive-VoCo: Complexity-Aware Visual Token Compression for Vision-Language Models](https://mlsanity.com/paper/2512.18496/) (2025)
- [AgriWorld:A World Tools Protocol Framework for Verifiable Agricultural Reasoning with Code-Executing LLM Agents](https://mlsanity.com/paper/2602.15325/) (2026)
- [Agent-GSPO: Communication-Efficient Multi-Agent Systems via Group Sequence Policy Optimization](https://mlsanity.com/paper/2510.22477/) (2025)
- [Reflective Confidence: Correcting Reasoning Flaws via Online Self-Correction](https://mlsanity.com/paper/2512.18605/) (2025)
- [Token-Budget Distillation: Transferring Full-Token Semantics to Compressed Video Vision-Language Models](https://mlsanity.com/paper/2608.28138/) (2026)
- [DynaPose4D: High-Quality 4D Dynamic Content Generation via Pose Alignment Loss](https://mlsanity.com/paper/2510.22473/) (2025)
- [GTMA: Dynamic Representation Optimization for OOD Vision-Language Models](https://mlsanity.com/paper/2512.18504/) (2025)
- [PTTA: A Pure Text-to-Animation Framework for High-Quality Creation](https://mlsanity.com/paper/2512.18614/) (2025)
- [SirenPose: Dynamic Scene Reconstruction via Geometric Supervision](https://mlsanity.com/paper/2512.20531/) (2025)
- [RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks](https://mlsanity.com/paper/2512.20920/) (2025)
- [A Three-Level Alignment Framework for Large-Scale 3D Retrieval and Controlled 4D Generation](https://mlsanity.com/paper/2512.22294/) (2025)
- [Learning Dynamic Scene Reconstruction with Sinusoidal Geometric Priors](https://mlsanity.com/paper/2512.22295/) (2025)
- [Self-Rewarded Multimodal Coherent Reasoning Across Diverse Visual Domains](https://mlsanity.com/paper/2512.22545/) (2025)
- [Forgetting as a Feature: Cognitive Alignment of Large Language Models](https://mlsanity.com/paper/2601.09726/) (2025)
- [Multi-Agent Cooperative Learning for Robust Vision-Language Alignment under OOD Concepts](https://mlsanity.com/paper/2601.09746/) (2026)
- [Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inference](https://mlsanity.com/paper/2607.02882/) (2026)
- [CB-Orchestrator: Adaptive Workflow Optimization for LLM Agents via Contextual Bandits](https://mlsanity.com/paper/2610.13064/) (2026)

## Abstract (arXiv)

Optimizing LLM-based workflows is typically formulated as a global search, where candidate workflows are evaluated based on a scalar metric. This paradigm, however, suffers from a critical flaw: information collapse. By reducing rich, multi-step execution traces to simple success/failure signals, existing methods are rendered blind to the underlying structure of failures, fundamentally preventing them from modeling the workflow's failure distribution. We reconceptualize this challenge as a distributional problem. We propose a new paradigm where the optimization goal is not to maximize a scalar score, but to directly minimize a workflow's Expected Failure Mass, i.e., the integral of its failure probability density function defined over a high-dimensional Failure Signature Space (FSS). This distributional lens allows us to move from inefficient, zero-order optimization to a principled, gradient-like descent on the failure landscape itself. We introduce CE-Graph, a framework that operationalizes this paradigm through a novel, failure-driven refinement process. CE-Graph approximates the failure distribution from a pool of counterexamples, identifies its densest regions as recurring failure modes, and applies targeted, operator-constrained graph edits via a Propose-and-Verify mechanism to greedily reduce the failure mass. On math, code, and QA benchmarks, our CE-Graph achieves higher robustness at a significantly lower cost than strong baselines. This suggests that a system's reliability emerges not from avoiding failures, but from systematically learning and reshaping the geometric structure of its failure distributions.