mlsanityData to 14 Oct 2026

CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization

arXiv 2310.10134 ↗First posted 16 Oct 2023cs.CL cs.AI cs.LGcited by 95

Not on the map. Why?

Cited by 53 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Language agents have shown some ability to interact with an external environment, e.g., a virtual world such as ScienceWorld, to perform complex tasks, e.g., growing a plant, without the startup costs of reinforcement learning. However, despite their zero-shot capabilities, these agents to date do not continually improve over time beyond performance refinement on a specific task. Here we present CLIN, the first language-based agent to achieve this, so that it continually improves over multiple trials, including when both the environment and task are varied, and without requiring parameter updates. Our approach is to use a persistent, dynamic, textual memory centered on causal abstractions (rather than general "helpful hints") that is regularly updated after each trial so that the agent gradually learns useful knowledge for new trials. In the ScienceWorld benchmark, CLIN is able to continually improve on repeated trials on the same task and environment, outperforming state-of-the-art reflective language agents like Reflexion by 23 absolute points. CLIN can also transfer its learning to new environments (or new tasks), improving its zero-shot performance by 4 points (13 for new tasks) and can further improve performance there through continual memory updates, enhancing performance by an additional 17 points (7 for new tasks). This suggests a new architecture for agents built on frozen models that can still continually and rapidly improve over time.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). Builds on 28 of its 42 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

28 of 42 references.

References · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  9. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  10. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  11. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  12. 2017Hindsight Experience Replaycited by 1,136
16 more
  1. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  2. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  3. 2022A Generalist Agentcited by 808
  4. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  5. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  6. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Taskscited by 168
  7. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  8. 2021Replay-Guided Adversarial Environment Designcited by 111
  9. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  10. 2015Deep Reinforcement Learning with a Natural Language Action Spacecited by 88
  11. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
  12. 2023Large Language Models Are Semi-Parametric Reinforcement Learning Agentscited by 42
  13. 2022Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvementcited by 37
  14. 2020How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worldscited by 35
  15. 2023Introspective Tips: Large Language Model for In-Context Decision Makingcited by 28
  16. 2019Learning STRIPS Action Models with Classical Planningcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 30; 2025: 18; 2026: 45.

95 papers.

Sort:

By area:

  1. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779
  2. 2024Agent Workflow Memorycited by 383
  3. 2025Inducing Programmatic Skills for Agentic Taskscited by 111
  4. 2024DiscoveryBench: Towards Data-Driven Discovery with Large Language Modelscited by 109
  5. 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Modelcited by 106
  6. 2024DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agentscited by 100
  7. 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 93
  8. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 82
  9. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 77
  10. 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 75
  11. 2024AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learningcited by 61
  12. 2024A Survey on Robotics with Foundation Models: toward Embodied AIcited by 56
38 more
  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  2. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 40
  3. 2024ReGAL: Refactoring Programs to Discover Generalizable Abstractionscited by 36
  4. 2024Data-driven Discovery with Large Generative Modelscited by 34
  5. 2024Towards Uncertainty-Aware Language Agentcited by 32
  6. 2024Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skillscited by 26
  7. 2024SelfGoal: Your Language Agents Already Know How to Achieve High-level Goalscited by 21
  8. 2025RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Userscited by 21
  9. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 21
  10. 2024Revealing the Barriers of Language Agents in Planningcited by 20
  11. 2026RealMem: Benchmarking LLMs in Real-World Memory-Driven Interactioncited by 20
  12. 2026Useful Memories Become Faulty When Continuously Updated by LLMscited by 20
  13. 2024OPEx: A Component-Wise Analysis of LLM-Centric Agents in Embodied Instruction Followingcited by 14
  14. 2024PDDLEGO: Iterative Planning in Textual Environmentscited by 14
  15. 2024AgentKit: Structured LLM Reasoning with Dynamic Graphscited by 13
  16. 2024PreAct: Prediction Enhances Agent's Planning Abilitycited by 12
  17. 2024Few-shot Dialogue Strategy Learning for Motivational Interviewing via Inductive Reasoningcited by 11
  18. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  19. 2026When Continual Learning Moves to Memory: A Study of Experience Reuse in LLM Agentscited by 10
  20. 2024Enhancing Agent Learning through World Dynamics Modelingcited by 9
  21. 2024MSI-Agent: Incorporating Multi-Scale Insight into Embodied Agents for Superior Planning and Decision-Makingcited by 9
  22. 2025ARIA: Training Language Agents with Intention-Driven Reward Aggregationcited by 9
  23. 2024HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Modelscited by 8
  24. 2025Memento No More: Coaching AI Agents to Master Multiple Tasks via Hints Internalizationcited by 8
  25. 2025Web World Modelscited by 7
  26. 2026Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agentscited by 7
  27. 2024TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidancecited by 6
  28. 2026Neuro-Symbolic Synergy for World Modelingcited by 6
  29. 2026CoMAP: Co-Evolving World Models and Agent Policies for LLM Agentscited by 6
  30. 2023Interpreting User Requests in the Context of Natural Language Standing Instructionscited by 5
  31. 2024Towards Unified Alignment Between Agents, Humans, and Environmentcited by 5
  32. 2024MindForge: Empowering Embodied Agents with Theory of Mind for Lifelong Cultural Learningcited by 5
  33. 2025ModelingAgent: Bridging LLMs and Mathematical Modeling for Real-World Challengescited by 5
  34. 2025EvoEmpirBench: Dynamic Spatial Reasoning with Agent-ExpVercited by 5
  35. 2026Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streamscited by 5
  36. 2026Always-OnAgents:A Survey of Persistent Memory, State, and Governance in LLMAgentscited by 5
  37. 2024Towards Generalizable Agents in Text-Based Educational Environments: A Study of Integrating RL with LLMscited by 4
  38. 2026Multi-Agent Transactive Memorycited by 4

50 most cited of 95 All 95 (JSON)