mlsanityData to 14 Oct 2026

Marginal Advantage Accumulation for Memory-Driven Agent Self-Evolution

arXiv 2606.20475 ↗First posted 18 Jun 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

In batch-style trace distillation, the same memory operation may receive contradictory feedback across different batches. Existing methods lack a cross-batch, operation-level evidence accumulation mechanism, making it impossible to distinguish stably effective operations from accidental hits. This paper formalizes the requirement as two structural conditions, alignability and comparability, and proposes Marginal Advantage Accumulation (MAA). MAA constructs differential signals to make them comparable across batches, accumulates signed evidence per operation via EMA, and ensures cross-batch traceability through semantic identity merging. As a post-processing architecture, MAA achieves the best results in 14 out of 16 settings across 4 benchmarks and 4 target models, consistently outperforming existing batch-level distillation baselines and matching or surpassing online alternatives in most settings, while reducing optimization-phase token consumption by approximately 75%.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 39 of its 41 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

39 of 41 references.

References · 39

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  8. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  9. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  10. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  11. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  12. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
27 more
  1. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  2. 2023Large Language Models as Optimizerscited by 864
  3. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  4. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  5. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  6. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  7. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
  8. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
  9. 2024Agent Workflow Memorycited by 383
  10. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
  11. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 235
  12. 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skillscited by 187
  13. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 148
  14. 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 148
  15. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 142
  16. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 137
  17. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 136
  18. 2025AgentEvolver: Towards Efficient Self-Evolving Agent Systemcited by 85
  19. 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 69
  20. 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agentscited by 69
  21. 2026DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolutioncited by 26
  22. 2026SkillGrad: Optimizing Agent Skills Like Gradient Descentcited by 26
  23. 2026ReCreate: Reasoning and Creating Domain Agents Driven by Experiencecited by 24
  24. 2025MARS: Multi-Agent Adaptive Reasoning with Socratic Guidance for Automated Prompt Optimizationcited by 15
  25. 2026Hybrid Self-evolving Structured Memory for GUI Agentscited by 13
  26. 2026Live-Evo: Online Evolution of Agentic Memory from Continuous Feedbackcited by 8
  27. 2026MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Gamescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Continual Learning in Transitioncited by 0