mlsanityData to 14 Oct 2026

SynAct: Synergistic Active Feedback Learning for LLM Agents

arXiv 2610.12651 ↗First posted 8 Oct 2026cs.LGcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents also fits: Agent skill learning and evolution

Abstract (arXiv)

LLM agents executing multi-step workflows fail in two fundamentally different ways: recoverable slips, which further attempts can repair, and knowledge gaps, which retrying cannot resolve. Expert feedback can address the latter, but expert attention is scarce. The key challenge is therefore not whether to ask for feedback, but where feedback can change the outcome. Existing approaches either avoid expert feedback, allowing agents to reinforce their own errors, or allocate feedback at the task level without identifying consequential decisions. We present SynAct, an active learning framework that treats expert attention as the binding constraint and allocates feedback at the level of individual decisions. SynAct ranks decisions by the mutual information between the selected action and the terminal outcome across repeated rollouts, queries an expert at the highest-value decisions, and distills the resulting corrections into transferable rules that the agent can reuse without expert access at deployment. We theoretically characterize what this signal identifies under finite repeated sampling: with k rollouts and at least m observations per action, outcome-constant decisions have zero mutual information, while outcome-separating decisions have mutual information at least H2(m/k) bits, yielding exact separation for any threshold t in (0,H2(m/k)). This also explains why success-based acquisition can be blind when all rollouts fail: success is constant even when failed runs reach different outcomes. Across four agent benchmarks and four models spanning three families, SynAct improves held-out accuracy in all 16 evaluated settings, by +11.7pp on average. The same signal transfers to inference-time routing, where SynAct outperforms budget-matched cascade, uncertainty, and expert-judge routers in 15 of 16 settings, while a single targeted correction outperforms seven additional unaimed retries.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 42 of its 48 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

42 of 48 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning26 May
  2. 2025Group-in-Group Policy Optimization for LLM Agent Training16 May
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learning20 Mar

Other references · 39

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  9. 2025Gemma 3 Technical Reportcited by 1,824
  10. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  11. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  12. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
27 more
  1. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  2. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  3. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  4. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  5. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  6. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
  7. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 398
  8. 2024Agent Workflow Memorycited by 383
  9. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
  10. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
  11. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 185
  12. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 177
  13. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 143
  14. 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 98
  15. 2023CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalizationcited by 95
  16. 2024AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agentscited by 74
  17. 2025Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agentscited by 62
  18. 2024AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learningcited by 61
  19. 2025SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoningcited by 44
  20. 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedbackcited by 24
  21. 2025Test-Time Adaptation for LLM Agents via Environment Interactioncited by 20
  22. 2025RLTHF: Targeted Human Feedback for LLM Alignmentcited by 8
  23. 2026Uncertainty-Aware Clarification in LLM Agents with Information Gaincited by 8
  24. 2024Disentangled Representation Learning for Parametric Partial Differential Equationscited by 3
  25. 2026Tool-Making and Self-Evolving LLM Agents in Low-Latency Systemscited by 3
  26. 2026Beyond Pairs: Your Language Model is Secretly Optimizing a Preference Graphcited by 2
  27. 2026Eluna: An Agentic LLM System for Automating Warehouse Operations with Reasoning and Task Executioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Does Multi-Agent RL Improve LLM Workflows? Workflow, Scale, and Policy-Sharing Tradeoffs
  2. 2026CollabFlow: Recursive Self-Improvement of Agent Collaboration
  3. 2026Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents
  4. 2026CAFE: Self-Improving Search Agents Need Co-Evolving Feedback
  5. 2024Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance
  6. 2025Beyond Reactivity: Measuring Proactive Problem Solving in LLM Agents
  7. 2026CAPF: Guiding Search-Agent Rollouts with Credit-Attenuated Privileged Feedback
  8. 2026Sentry: Learning to Recover from LLM Agent Failures at Test Time