mlsanityData to 7 Oct 2026

Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning

arXiv 2606.20002 ↗First posted 18 Jun 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based AI agent gets deployed in an environment, it solves a long sequence of tasks while continuously exploring the environment, learning from its own experiences, and iteratively self-updating its context about the environment, thereby achieving progressively better performance on future tasks conditioned on the updated context. Major components of the CoD framework include: (1) algorithm design and infrastructure for end-to-end reinforcement learning (RL) with long rollout sequences interleaving solve-task and update-context episodes; (2) tasks and environments for incentivizing and eliciting the targeted meta-capability in LLMs during training, as well as for faithfully measuring progress during evaluation. We present proof-of-concept implementations of the CoD framework, including a GRPO-style RL algorithm with fine-grained credit assignment, as well as tasks and environments tailored to the targeted meta-capability (rather than domain-specific LLM capabilities or standard task-by-task RL). Empirical results with Qwen3-8B and Qwen3.6-27B validate the efficacy of end-to-end RL training in the CoD setting, and demonstrate the potential for out-of-distribution generalization -- within the training domains, across different domains, and from CoD to Ralph-loop settings -- of the elicited meta-capability. Our investigation of CoD connects several lines of prior work, and opens up new opportunities for advancing LLMs and AI agents. To facilitate further research and applications, we release our implementations at https://github.com/agentscope-ai/Trinity-RFT/tree/research/cod_v2/examples/research_cod.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 66 of its 93 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

66 of 93 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Meta-Reinforcement Learning with Self-Reflection for Agentic Search11 Mar
  2. 2026MAGE: Meta-Reinforcement Learning for Language Agents toward Strategic Exploration and Exploitation4 Mar
  3. 2026R3L: Reflect-then-Retry Reinforcement Learning with Language-Guided Exploration, Pivotal Credit, and Positive Amplification7 Jan
  4. 2025Meta-RL Induces Exploration in Language Agents18 Dec
  5. 2025Group-in-Group Policy Optimization for LLM Agent Training16 May
  6. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL29 Feb
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learning20 Mar

Other references · 58

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  10. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
  11. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  12. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
38 more
  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  2. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  3. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  4. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  5. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  6. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  7. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  8. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
  9. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
  10. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 256
  11. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  12. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 226
  13. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 220
  14. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  15. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 146
  16. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 129
  17. 2023A Tutorial on Meta-Reinforcement Learningcited by 102
  18. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 82
  19. 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 72
  20. 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learnerscited by 65
  21. 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Taskscited by 62
  22. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
  23. 2026CL-bench: A Benchmark for Context Learningcited by 46
  24. 2018Some Considerations on Learning to Explore via Meta-Reinforcement Learningcited by 43
  25. 2024Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Togethercited by 38
  26. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 35
  27. 2024Provable Scaling Laws for the Test-Time Compute of Large Language Modelscited by 25
  28. 2025Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Modelscited by 19
  29. 2026SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agentscited by 19
  30. 2026PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoningcited by 17
  31. 2025Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friendscited by 13
  32. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
  33. 2026Frontier-Eng: Benchmarking Self-Evolving Agents on Real-World Engineering Tasks with Generative Optimizationcited by 13
  34. 2026SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessmentcited by 12
  35. 2026Learning Agent-Compatible Context Management for Long-Horizon Taskscited by 12
  36. 2026Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environmentscited by 11
  37. 2026Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learningcited by 8
  38. 2026RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewardscited by 8

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking Agentic Reinforcement Learning In Large Language Models
  2. 2025Lifelong Learning of Large Language Model based Agents: A Roadmap
  3. 2026Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents
  4. 2023Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents
  5. 2026From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
  6. 2024Training Agents with Weakly Supervised Feedback from Large Language Models
  7. 2023Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach
  8. 2024Offline Training of Language Model Agents with Functions as Learnable Weights