mlsanityData to 7 Oct 2026

StreamBench: Towards Benchmarking Continuous Improvement of Language Agents

arXiv 2406.08747 ↗First posted 13 Jun 2024cs.CLcited by 44

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Recent works have shown that large language model (LLM) agents are able to improve themselves from experience, which is an important ability for continuous enhancement post-deployment. However, existing benchmarks primarily evaluate their innate capabilities and do not assess their ability to improve over time. To address this gap, we introduce StreamBench, a pioneering benchmark designed to evaluate the continuous improvement of LLM agents over an input-feedback sequence. StreamBench simulates an online learning environment where LLMs receive a continuous flow of feedback stream and iteratively enhance their performance. In addition, we propose several simple yet effective baselines for improving LLMs on StreamBench, and provide a comprehensive analysis to identify critical components that contribute to successful streaming strategies. Our work serves as a stepping stone towards developing effective online learning strategies for LLMs, paving the way for more adaptive AI systems in streaming scenarios. Source code: https://github.com/stream-bench/stream-bench. Benchmark website: https://stream-bench.github.io.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 31 of its 44 references that are in the map. First posted on arXiv 13 Jun 2024.

Builds on

31 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023ExpeL: LLM Agents Are Experiential Learners20 Aug

Other references · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  11. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  12. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
18 more
  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  2. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  3. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  4. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  5. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  6. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  7. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  8. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  9. 2023Larger language models do in-context learning differentlycited by 386
  10. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  11. 2018Online Learning: A Comprehensive Surveycited by 203
  12. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  13. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  14. 2023Unified Demonstration Retriever for In-Context Learningcited by 141
  15. 2022DDXPlus: A New Dataset For Automatic Medical Diagnosiscited by 87
  16. 2023Meta-Learning Online Adaptation of Language Modelscited by 43
  17. 2023Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrationscited by 42
  18. 2023MoT: Memory-of-Thought Enables ChatGPT to Self-Improvecited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 15; 2026: 26.

44 papers.

Sort:

By area:

  1. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  2. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  3. 2025Survey on Evaluation of LLM-based Agentscited by 190
  4. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 129
  5. 2024Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Modelscited by 90
  6. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  7. 2026SkillOS: Learning Skill Curation for Self-Evolving Agentscited by 45
  8. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 29
  9. 2025MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosiscited by 23
  10. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 22
  11. 2025RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Videocited by 21
  12. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 19
32 more
  1. 2026EvoRoute: Experience-Driven Self-Routing LLM Agent Systemscited by 13
  2. 2025Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learningcited by 12
  3. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Surveycited by 12
  4. 2024Evaluation-Driven Development and Operations of LLM Agents: A Process Model and Reference Architecturecited by 10
  5. 2025Layered Chain-of-Thought Prompting for Multi-Agent LLM Systems: A Comprehensive Approach to Explainable Large Language Modelscited by 10
  6. 2026EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfercited by 8
  7. 2025Survey: Multi-Armed Bandits Meet Large Language Modelscited by 6
  8. 2026Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Modelscited by 6
  9. 2026Uncertainty-Aware Clarification in LLM Agents with Information Gaincited by 6
  10. 2025CogStream: Context-guided Streaming Video Question Answeringcited by 5
  11. 2026AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agentscited by 5
  12. 2026Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stagescited by 4
  13. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  14. 2026MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Modelscited by 2
  15. 2026Meta-TTL: Meta-Learning Self-Improvement Policies for Language Agentscited by 2
  16. 2026AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?cited by 2
  17. 2024Scaffolded Language Models with Language Supervision for Mixed-Autonomy: A Surveycited by 1
  18. 2025Benchmarking In-context Experiential Learning Through Repeated Product Recommendationscited by 1
  19. 2026EpiEvolve: Self-Evolving Agents for Streaming Pandemic Forecasting under Regime Shiftscited by 1
  20. 2026From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memorycited by 1
  21. 2026OPD-Evolver: Cultivating Holistic Agent Evolver via On-Policy Distillationcited by 1
  22. 2026Compressing History into Memory: Distilling Transformers into Recurrent Transformerscited by 1
  23. 2026Same Agent, Different Answers: A Repeat-Aware Audit of Corpus-Induced Answer Churn in Retrieval-Augmented QAcited by 1
  24. 2026ME-IQA: Memory-Enhanced Image Quality Assessment via Re-Rankingcited by 0
  25. 2026Supplement Generation Training for Enhancing Agentic Task Performancecited by 0
  26. 2026RIZZ: Routing Interactions to Near Zero-Interference Zones for Continual Adaptation of Black-Box Agentscited by 0
  27. 2026CIGPO: Contextual Information-Gain Policy Optimization for Multi-Turn Evidence-Reading LLM Agentscited by 0
  28. 2026SkillMentor: LLM Agent Self-Evolution via Learning Blind-Spot Diagnosiscited by 0
  29. 2026PACE-Bench: Benchmarking Physics Adaptation via Code Evolution in Dynamic Environmentscited by 0
  30. 2026Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvementcited by 0
  31. 2026Semantic Projection for Continual Self-Evolution of Language Agentscited by 0
  32. 2026RefCon: Iterative Refinement and Contrastive Memory Extraction for Context-Evolving Agentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023AgentBench: Evaluating LLMs as Agents
  2. 2024AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agents
  3. 2025GuideBench: Benchmarking Domain-Oriented Guideline Following for LLM Agents
  4. 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation
  5. 2026LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth
  6. 2025CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Tasks
  7. 2025A Self-Improving Coding Agent
  8. 2025Towards Adaptive ML Benchmarks: Web-Agent-Driven Construction, Domain Expansion, and Metric Optimization