mlsanityData to 7 Oct 2026

PSD: Pseudo Self-Distillation of Memory Representation Capabilities for LLM Agents

arXiv 2609.23449 ↗First posted 20 Sep 2026cs.IR cs.CL cs.LGcited by 0

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Memory systems are becoming a core component of LLM agents, but constructing and maintaining memory remains expensive because it relies on repeated calls to large proprietary language models. This cost creates a major barrier to deploying memory-enhanced agents at scale. In this paper, we present Pseudo Self-Distillation (PSD), a framework that enables small language models (SLMs) to construct hierarchical memory representations by distilling behavior from a strong black-box oracle through a multi-stage training pipeline. Standard distillation methods require access to teacher logits or hidden states, which closed models do not expose. Unlike conventional self-distillation settings, where supervision is derived from a model's own predictions, sampled rollouts, or aggregated outputs, PSD enables a single-model distillation setup while channeling external oracle knowledge through the prompt. PSD uses a single small model in two roles: a teacher that sees a privileged prompt containing the oracle's answer as reference context, and a student that sees only the task prompt. The student learns to reproduce the teacher's output distribution, absorbing oracle-guided behavior into its own weights without accessing the oracle's internals. On LoCoMo, PSD-trained Qwen3-0.6B, 1.7B, and 4B match or exceed GPT-4.1-mini on downstream retrieval at a fraction of the deployment cost, with off-policy PSD achieving the strongest results across most conditions. We further show that this memory-construction capability transfers out-of-distribution to LongMemEval, despite the students being trained exclusively on LoCoMo with no exposure to LongMemEval data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 22 of its 33 references that are in the map. First posted on arXiv 20 Sep 2026.

Builds on

22 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning27 Aug
  2. 2025What Deserves Memory: Adaptive Memory Distillation for LLM Agents5 Aug
  3. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents27 Feb

Other references · 19

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  5. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  6. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  7. 2016Sequence-Level Knowledge Distillationcited by 761
  8. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
  9. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
  10. 2024Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Samplingcited by 85
  11. 2026Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routingcited by 65
  12. 2026Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervisioncited by 58
7 more
  1. 2025Black-Box On-Policy Distillation of Large Language Modelscited by 53
  2. 2026Stable On-Policy Distillation through Adaptive Target Reformulationcited by 47
  3. 2026Embarrassingly Simple Self-Distillation Improves Code Generationcited by 36
  4. 2026OPSDL: On-Policy Self-Distillation for Long-Context Language Modelscited by 29
  5. 2026Rethinking On-Policy Self-Distillation for Thinking Modelscited by 28
  6. 2026Purified OPSD: On-Policy Self-Distillation Without Losing How to Thinkcited by 23
  7. 2026Memora: A Harmonic Memory Representation Balancing Abstraction and Specificitycited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory
  2. 2026Towards Autonomous Memory Agents
  3. 2026MemTrain: Self-Supervised Context Memory Training
  4. 2025Learn to Memorize: Optimizing LLM-based Agents with Adaptive Memory Framework
  5. 2026Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents
  6. 2026Evaluating Memory Structure in LLM Agents
  7. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
  8. 2025Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topics