# Distilling Feedback into Memory-as-a-Tool

arXiv: https://arxiv.org/abs/2601.05960 · HTML: https://mlsanity.com/paper/2601.05960/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 9 Jan 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 19 of 19)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains](https://mlsanity.com/paper/2507.17746/) (2025)
- [Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning](https://mlsanity.com/paper/2508.19828/) (2025)
- [Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Tools](https://mlsanity.com/paper/2502.04644/) (2025)
- [Feedback Loops With Language Models Drive In-Context Reward Hacking](https://mlsanity.com/paper/2402.06627/) (2024)
- [MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory](https://mlsanity.com/paper/2404.11672/) (2024)
- [From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users](https://mlsanity.com/paper/2508.17281/) (2025)
- [RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems](https://mlsanity.com/paper/2508.01415/) (2025)
- [Learning with Memory Embeddings](https://mlsanity.com/paper/1511.07972/) (2015)
- [Configurable Preference Tuning with Rubric-Guided Synthetic Data](https://mlsanity.com/paper/2506.11702/) (2025)
- [Specification Self-Correction: Mitigating In-Context Reward Hacking Through Test-Time Refinement](https://mlsanity.com/paper/2507.18742/) (2025)
- [MetaSC: Test-Time Safety Specification Optimization for Language Models](https://mlsanity.com/paper/2502.07985/) (2025)
- [Interactive Reinforcement Learning with Dynamic Reuse of Prior Knowledge from Human/Agent's Demonstration](https://mlsanity.com/paper/1805.04493/) (2018)

## Built on by (most cited first, 2 of 2)

- [SkillX: Automatically Constructing Skill Knowledge Bases for Agents](https://mlsanity.com/paper/2604.04804/) (2026)
- [STEM Agent: A Self-Adapting, Tool-Enabled, Extensible Architecture for Multi-Protocol AI Agent Systems](https://mlsanity.com/paper/2603.22359/) (2026)

## Abstract (arXiv)

We propose a framework that amortizes the cost of inference-time reasoning by converting transient critiques into retrievable guidelines, through a file-based memory system and agent-controlled tool calls. We evaluate this method on the Rubric Feedback Bench, a novel dataset for rubric-based learning. Experiments demonstrate that our augmented LLMs rapidly match the performance of test-time refinement pipelines while drastically reducing inference cost.