mlsanityData to 7 Oct 2026

ExpSeek: Self-Triggered Experience Seeking for Web Agents

arXiv 2601.08605 ↗First posted 13 Jan 2026cs.CL cs.AIcited by 23

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Experience intervention in web agents emerges as a promising technical paradigm, enhancing agent interaction capabilities by providing valuable insights from accumulated experiences. However, existing methods predominantly inject experience passively as global context before task execution, struggling to adapt to dynamically changing contextual observations during agent-environment interaction. We propose ExpSeek, which shifts experience toward step-level proactive seeking: (1) estimating step-level entropy thresholds to determine intervention timing using the model's intrinsic signals; (2) designing step-level tailored experience content. Experiments on Qwen3-8B and 32B models across four challenging web agent benchmarks demonstrate that ExpSeek achieves absolute improvements of 9.3% and 7.5%, respectively. Our experiments validate the feasibility and advantages of entropy as a self-triggering signal, reveal that even a small-scale 4B experience model can significantly boost the performance of larger agent models. The code is released at https://github.com/WYRipple/ExpSeek.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 47 of its 91 references that are in the map. First posted on arXiv 13 Jan 2026.

Builds on

47 of 91 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents12 Jan
  2. 2025Memory in the Age of AI Agents15 Dec
  3. 2025Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution11 Dec
  4. 2025Contextual Experience Replay for Self-Improvement of Language Agents7 Jun
  5. 2024A Survey on the Memory Mechanism of Large Language Model based Agents21 Apr

Other references · 42

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  4. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  5. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  6. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  7. 2023Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Controlcited by 180
  8. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 180
  9. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 177
  10. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 171
  11. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 162
  12. 2025Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RLcited by 144
30 more
  1. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
  2. 2025xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluationscited by 118
  3. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
  4. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  5. 2024Beyond Browsing: API-Based Web Agentscited by 77
  6. 2025SealQA: Raising the Bar for Reasoning in Search-Augmented Language Modelscited by 65
  7. 2024Offline Reinforcement Learning for LLM Multi-Step Reasoningcited by 53
  8. 2025Training-Free Group Relative Policy Optimizationcited by 53
  9. 2025FLEX: Continuous Agent Evolution via Forward Learning from Experiencecited by 51
  10. 2025Rethinking Entropy Interventions in RLVR: An Entropy Change Perspectivecited by 50
  11. 2025Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agentscited by 43
  12. 2025EvolveSearch: An Iterative Self-Evolving Search Agentcited by 42
  13. 2025Curriculum-RLAIF: Curriculum Alignment with Reinforcement Learning from AI Feedbackcited by 30
  14. 2026UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervisioncited by 23
  15. 2025CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarioscited by 19
  16. 2025SOTOPIA-Omega: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agentscited by 16
  17. 2025Learning to Contextualize Web Pages for Enhanced Decision Making by LLM Agentscited by 16
  18. 2025LoopTool: Closing the Data-Training Loop for Robust LLM Tool Callscited by 16
  19. 2025Adaptive Social Learning via Mode Policy Optimization for Language Agentscited by 15
  20. 2025Who Stole Your Data? A Method for Detecting Unauthorized RAG Theftcited by 15
  21. 2025Queries Are Not Alone: Clustering Text Embeddings for Video Searchcited by 14
  22. 2025Improving Consistency in Large Language Models through Chain of Guidancecited by 13
  23. 2025DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Actioncited by 11
  24. 2025When TableQA Meets Noise: A Dual Denoising Framework for Complex Questions and Large-scale Tablescited by 10
  25. 2024Don't Half-listen: Capturing Key-part Information in Continual Instruction Tuningcited by 9
  26. 2024Revealing and Mitigating the Challenge of Detecting Character Knowledge Errors in LLM Role-Playingcited by 9
  27. 2025COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligencecited by 9
  28. 2025Self-Guided Function Calling in Large Language Models via Stepwise Experience Recallcited by 7
  29. 2025ExpeTrans: LLMs Are Experiential Transfer Learnerscited by 2
  30. 2025ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interactioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 23.

23 papers.

Sort:

By area:

  1. 2026ReCreate: Reasoning and Creating Domain Agents Driven by Experiencecited by 22
  2. 2026GAM: Hierarchical Graph-based Agentic Memory for LLM Agentscited by 19
  3. 2026TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrievalcited by 13
  4. 2026MemPO: Self-Memory Policy Optimization for Long-Horizon Agentscited by 11
  5. 2026Mem2Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillationcited by 10
  6. 2026ATTNPO: Attention-Guided Process Supervision for Efficient Reasoningcited by 9
  7. 2026From log π to π: Taming Divergence in Soft Clipping via Bilateral Decoupled Decay of Probability Gradient Weightcited by 6
  8. 2026Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Gamescited by 6
  9. 2026TSPO: Breaking the Double Homogenization Dilemma in Multi-turn Search Policy Optimizationcited by 5
  10. 2026What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoningcited by 4
  11. 2026TopoDIM: One-shot Topology Generation of Diverse Interaction Modes for Multi-Agent Systemscited by 3
  12. 2026A New Role for Relevance: Guiding Corpus Interaction in Agentic Searchcited by 3
11 more
  1. 2026R^2-Mem: Reflective Experience for Memory Searchcited by 2
  2. 2026Context-Agent: Dynamic Discourse Trees for Non-Linear Dialoguecited by 1
  3. 2026Rethinking Experience Utilization in Self-Evolving Language Model Agentscited by 1
  4. 2026Context Training with Active Information Seekingcited by 1
  5. 2026Agents in the Large: Perception-Centered Architecture for Persistent Agentscited by 1
  6. 2026ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detectioncited by 0
  7. 2026Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agentscited by 0
  8. 2026DRIVE: Modeling Skills at the Reasoning and Interaction Levels for Web Agents under Continual Learningcited by 0
  9. 2026Causal Evidence Extraction and Triangulation in Crisis Reports using Large Language Models: A ReliefWeb-based Studycited by 0
  10. 2026Success Leaves Detours: Learning Executable Walkthroughs for Long-Horizon Agentscited by 0
  11. 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EXG: Self-Evolving Agents with Experience Graphs
  2. 2026ExpVoyager: Direct Experience Navigation for Dynamic Agent Skill Synthesis
  3. 2025WebATLAS: An LLM Agent with Experience-Driven Memory and Action Simulation
  4. 2026Guides That Cause Actions: An Offline Study of Guide-Action Mutual Reinforcement in Multimodal Web Agents
  5. 2026LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues
  6. 2025WebCoach: Self-Evolving Web Agents with Cross-Session Memory Guidance
  7. 2026ExpWeaver: LLM Agents Learn from Experience via Latent RAG
  8. 2026Modeling Distinct Human Interaction in Web Agents