mlsanityData to 14 Oct 2026

Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents

arXiv 2608.21027 ↗First posted 21 Aug 2026cs.AIcited by 2

On the map: LLM agents and tool use → Agent skill learning and evolution

Abstract (arXiv)

LLM agents are emerging as an important paradigm for real-world tasks that require reasoning, tool use, and sequential decision-making. As these agents operate over longer horizons, runtime intervention offers a way to improve reliability without retraining the underlying actor. Effective intervention must provide a useful direction for recovery besides a warning. Existing approaches often rely on an expert solver or a critic that generates task-specific corrections, incurring either the cost of another capable solver or the capacity demands of a task-capable critic. We introduce Comparison-Only Tiny Advisor (COTA) for constructive runtime intervention, which reduces the learned intervention role to local action comparison. A lightweight comparator judges the actor's proposal against available alternatives, and preferred alternatives are returned as non-binding advice for replanning. The comparator is trained from same-prefix counterfactual branches. Across WebShop, ALFWorld, and tau^3-Retail with three LLM actors, COTA instantiated with a 0.5B comparator consistently improves the original actor and achieves the strongest overall performance--cost trade-off among the compared methods. These results suggest that effective runtime intervention need not itself be a task-solving problem: the intervention role can be separated from task solving and handled by a lightweight model specialized for local comparison.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 41 of its 49 references that are in the map. First posted on arXiv 21 Aug 2026.

Builds on

41 of 49 references.

References · 41

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  7. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  8. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  9. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  10. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  11. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  12. 2023Teaching Large Language Models to Self-Debugcited by 988
29 more
  1. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  3. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  4. 2017Safe Reinforcement Learning via Shieldingcited by 449
  5. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
  6. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  7. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 223
  8. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  9. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoningcited by 151
  10. 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoningcited by 124
  11. 2019Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Modelscited by 117
  12. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 103
  13. 2024CriticBench: Benchmarking LLMs for Critique-Correct Reasoningcited by 88
  14. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinementcited by 88
  15. 2024Prover-Verifier Games improve legibility of LLM outputscited by 85
  16. 2025Process Reward Models for LLM Agents: Practical Framework and Directionscited by 85
  17. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 77
  18. 2024Learning to Decode Collaboratively with Multiple Language Modelscited by 62
  19. 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 59
  20. 2025ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Predictioncited by 28
  21. 2025AgentRM: Enhancing Agent Generalization with Reward Modelingcited by 27
  22. 2025Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executorscited by 22
  23. 2021Reinforcement Learning for Classical Planning: Viewing Heuristics as Dense Reward Generatorscited by 21
  24. 2025The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvementcited by 20
  25. 2026Accurate Failure Prediction in Agents Does Not Imply Effective Failure Preventioncited by 15
  26. 2026Weak Critics Make Strong Learners: On-Policy Critique Distillation for Scalable Oversightcited by 6
  27. 2020Long-Term Visitation Value for Deep Exploration in Sparse Reward Reinforcement Learningcited by 5
  28. 2026Asymmetric Actor-Critic for Multi-turn LLM Agentscited by 4
  29. 2025On-line Policy Improvement using Monte-Carlo Searchcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026EvoSteer: Online Self-Evolving Graph Orchestration via Reference-Anchored Credit Assignmentcited by 0
  2. 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM-as-a-Verifier: A General-Purpose Verification Framework
  2. 2026Sentry: Learning to Recover from LLM Agent Failures at Test Time
  3. 2026Self-Harness: Harnesses That Improve Themselves
  4. 2026Harnessing LLM Agents with Skill Programs
  5. 2026SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revision
  6. 2026AgentEscapeBench: Evaluating Out-of-Domain Tool-Grounded Reasoning in LLM Agents
  7. 2025ReflexGrad: Within-Episode Failure Recovery in LLM Agents via Progress-Gated Dual-Process Routing
  8. 2026SkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories