# Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents

arXiv: https://arxiv.org/abs/2606.15017 · HTML: https://mlsanity.com/paper/2606.15017/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 12 Jun 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 23 of 23)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models](https://mlsanity.com/paper/2401.13919/) (2024)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills](https://mlsanity.com/paper/2504.07079/) (2025)
- [An Illusion of Progress? Assessing the Current State of Web Agents](https://mlsanity.com/paper/2504.01382/) (2025)
- [The BrowserGym Ecosystem for Web Agent Research](https://mlsanity.com/paper/2412.05467/) (2024)
- [Inducing Programmatic Skills for Agentic Tasks](https://mlsanity.com/paper/2504.06821/) (2025)
- [WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks](https://mlsanity.com/paper/2407.05291/) (2024)
- [Beyond Browsing: API-Based Web Agents](https://mlsanity.com/paper/2410.16464/) (2024)
- [REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites](https://mlsanity.com/paper/2504.11543/) (2025)
- [PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agents](https://mlsanity.com/paper/2603.03296/) (2026)
- [WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Tasks](https://mlsanity.com/paper/2506.01952/) (2025)
- [PolySkill: Learning Generalizable Skills Through Polymorphic Abstraction](https://mlsanity.com/paper/2510.15863/) (2025)
- [WebXSkill: Skill Learning for Autonomous Web Agents](https://mlsanity.com/paper/2604.13318/) (2026)
- [WAREX: Web Agent Reliability Evaluation on Existing Benchmarks](https://mlsanity.com/paper/2510.03285/) (2025)
- [Experiential Reflective Learning for Self-Improving LLM Agents](https://mlsanity.com/paper/2603.24639/) (2026)
- [Terminal Agents Suffice for Enterprise Automation](https://mlsanity.com/paper/2604.00073/) (2026)
- [JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents Adaptation](https://mlsanity.com/paper/2510.04373/) (2025)

## Built on by (most cited first, 6 of 6)

- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Routing Is Least Learnable Where It Is Most Valuable: Bounds on Representation Routing for Web Agents](https://mlsanity.com/paper/2608.06171/) (2026)
- [What Does a Skill Actually Do? Estimands and Evaluation Validity for Tool and Skill Use in LLM Agents: A Critical Review](https://mlsanity.com/paper/2609.33153/) (2026)
- [StraTune: Adaptive Selection of Revision Operators for Self-Evolving LLM Skills](https://mlsanity.com/paper/2609.32886/) (2026)
- [What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agents](https://mlsanity.com/paper/2609.39989/) (2026)
- [Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents](https://mlsanity.com/paper/2610.01787/) (2026)

## Abstract (arXiv)

Online web agents often augment a base actor with memory, workflow, or skill modules. These modules can improve performance, but they also consume test-time tokens, a cost rarely reported alongside the actor's inference cost. We study online augmentation, where this overhead is paid on every task, and re-evaluate its benefits under a fixed total inference budget. We compare AWM, ASI, and ReasoningBank with a token-matched vanilla baseline that uses the same budget for additional actor steps. Across four WebArena domains and three models, Gemini 3 Flash, GPT-5.4-mini, and Qwen 3.6-27B, the vanilla baseline matches or surpasses all three augmentation methods in aggregate success rate while often using fewer total tokens. We observe a similar trend on WorkArena-L1 with Qwen 3.6-27B, indicating that the effect extends to enterprise knowledge-work tasks. Our results suggest that skills and workflow memory can be useful in specific domains, but their apparent gains often vanish against a budget-matched actor. We further show that run-to-run variance materially affects outcomes and should be reported as a core evaluation criterion for online web agents.