mlsanityData to 14 Oct 2026

DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents

arXiv 2610.19376 ↗First posted 14 Oct 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks also fits: Reinforcement learning for LLM agents

Abstract (arXiv)

Language-model agents that run long tasks often redo work they have already done: they re-issue a tool call they made earlier and get back exactly the same observation, a step that adds no new information. We identify this failure as a loop. Loops appear natively in Qwen, Llama, and Gemini agents and become far more common once the agent's history is compressed to fit its context window. To reduce these loops, we propose DeLoop, a self-supervised training method. DeLoop builds on a simple observation: the agent's own rollout can provide the supervision needed to learn from its loops. We use a looping action as a negative example and the first later non-looping action in the same rollout as a positive example. DeLoop learns from these pairs through direct preference optimization and does not require task rewards, gold answers, or LLM judges. On CodeGym under compression, DeLoop reduces the share of Qwen3-8B tasks containing loops from 57.3% to 6.1%. It also uses about one third as many steps per task, with similar task success. On Qwen3-32B, DeLoop reduces loops and steps while raising task success from 37.7% to 44.8%.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 68 of its 72 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

68 of 72 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026U-Fold: Dynamic Intent-Aware Context Folding for User-Centric Agents26 Jan
  2. 2025Co-Evolving Agents: Learning from Failures as Hard Negatives27 Nov
  3. 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents4 Aug
  4. 2025Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments23 May
  5. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training20 Jan
  6. 2025AgentRefine: Enhancing Agent Generalization through Refinement Tuning3 Jan
  7. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents13 Aug
  8. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement17 Jun
  9. 2024ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy21 Mar
  10. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs19 Oct
  11. 2023FireAct: Toward Language Agent Fine-tuning9 Oct
  12. 2023AgentBench: Evaluating LLMs as Agents7 Aug

Other references · 56

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  9. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  10. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  11. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  12. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
38 more
  1. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  2. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  5. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  6. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  7. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  8. 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
  9. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  10. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  11. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  12. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  13. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
  14. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
  15. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 348
  16. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  17. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 273
  18. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
  19. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 238
  20. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 200
  21. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 197
  22. 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 148
  23. 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 141
  24. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 136
  25. 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 116
  26. 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Modelcited by 106
  27. 2025TRAIL: Trace Reasoning and Agentic Issue Localizationcited by 104
  28. 2022Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generationcited by 94
  29. 2024Direct Multi-Turn Preference Optimization for Language Agentscited by 64
  30. 2025Context as a Tool: Context Management for Long-Horizon SWE-Agentscited by 43
  31. 2026RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL Systemcited by 24
  32. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 18
  33. 2025Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planningcited by 17
  34. 2025Generalizable End-to-End Tool-Use RL with Synthetic CodeGymcited by 17
  35. 2026Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memorycited by 12
  36. 2026Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agentscited by 11
  37. 2026PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoningcited by 10
  38. 2026Toward Reliable Context Compression for Long-Horizon Agents: An Empirical Study of Execution Instabilitycited by 10

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training
  2. 2026Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agents
  3. 2026Spinning Straw into Gold: Relabeling LLM Agent Trajectories in Hindsight for Successful Demonstrations
  4. 2025Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks
  5. 2025SAND: Boosting LLM Agents with Self-Taught Action Deliberation
  6. 2026daVinci-Agency: Unlocking Long-Horizon Agency Data-Efficiently
  7. 2026StateComp: Learning When to Compress History in Long Horizon Agents
  8. 2026When Can Agents Forget Their Reasoning? ICLR for Long-Horizon Agent Context Compression