mlsanityData to 14 Oct 2026

Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design

arXiv 2609.22086 ↗First posted 18 Sep 2026cs.AI cs.CVcited by 1

New in LLM agents and tool use; no direction yet. LLM agents and tool use

Abstract (arXiv)

Professional graphic design is a long-horizon agentic task in which structured, editable artifacts emerge from many interdependent actions, yet outcomes admit no reliable programmatic oracle. We introduce a continual adaptation framework in which a frozen frontier model operates professional design software through more than 230 tools, while an external procedural memory of natural-language skills accumulates and refines reusable design procedures from experience. The memory widens by acquiring procedures for recurring uncovered subtasks and deepens by revising existing procedures against their own successful and failed executions, while a matched replay gate admits only changes that repair failures without regressing observed successes. Five rounds over 1,406 real user briefs and 1,869 automatically graded trajectories, with no weight updates and no human labels, grow the bank from 76 documentation-derived skills to 139 and raise GenEval2 execution success on Claude-Sonnet-4 from 72.7% to 99.3% (+11.99 points in generation quality), with 61.8% and 67.6% win rates against the no-skill agent across four specialized design benchmarks on Claude-Sonnet-4 and Claude-Opus-4.6. We further show the two mechanisms are effective in combination: on 200 held-out briefs from user-traffic benchmark, widening or deepening alone reaches a 49.4% / 48.6% win rate over the no-skill agent, while their combination reaches 58.5% (p = 0.025). Procedural memory offers a practical route to continual adaptation of agents under noisy, unverifiable feedback.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 47 of its 62 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

47 of 62 references.

References · 47

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  8. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  9. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  10. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  11. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  12. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
35 more
  1. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  2. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  3. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  4. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  5. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 398
  6. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
  7. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  8. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 163
  9. 2017Safe Policy Improvement with Baseline Bootstrappingcited by 144
  10. 2021CanvasVAE: Learning to Generate Vector Graphic Documentscited by 95
  11. 2025OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generationcited by 87
  12. 2023PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layoutcited by 61
  13. 2026Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skillscited by 55
  14. 2025VisPlay: Self-Evolving Vision-Language Models from Imagescited by 47
  15. 2026Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agentscited by 47
  16. 2023COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Designcited by 43
  17. 2024OpenCOLE: Towards Reproducible Automatic Graphic Design Generationcited by 37
  18. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
  19. 2024PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generationcited by 35
  20. 2026Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Gradingcited by 31
  21. 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agentscited by 27
  22. 2026From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Modelscited by 23
  23. 2026MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Datacited by 21
  24. 2025Learning Hierarchical Procedural Memory for LLM Agents through Bayesian Selection and Contrastive Refinementcited by 20
  25. 2025CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic Design Generationcited by 19
  26. 2026Milestone-Guided Policy Learning for Long-Horizon Language Agentscited by 19
  27. 2026G-Zero: Self-Play for Open-Ended Generation from Zero Datacited by 17
  28. 2024From Elements to Design: A Layered Approach for Automatic Graphic Design Compositioncited by 13
  29. 2025BannerAgency: Advertising Banner Design with Multimodal LLM Agentscited by 11
  30. 2026SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditingcited by 11
  31. 2025Rethinking Layered Graphic Design Generation with a Top-Down Approachcited by 10
  32. 2026COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflowscited by 6
  33. 2026LaDe: Unified Multi-Layered Graphic Media Generation and Decompositioncited by 5
  34. 2025GraphicBench: A Planning Benchmark for Graphic Design with Language Agentscited by 4
  35. 2026CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructionscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CONTRAMEM: Learning Self-Evolving Procedural Memory from Contrasting Multi-Model Trajectories
  2. 2026From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agents
  3. 2026Managing Procedural Memory in LLM Agents: Control, Adaptation, and Evaluation
  4. 2025Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution
  5. 2026EvoDiagram: Agentic Editable Diagram Creation via Design Expertise Evolution
  6. 2026Enabling Creative Exploration for Vibe Design Agents
  7. 2026DeskCraft: Benchmarking Desktop Agents on Professional Workflows and Human-in-the-Loop Collaboration
  8. 2026AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design