# Internalizing Agent Experience into Diffusion Model Weights via On-Policy Context Distillation

arXiv: https://arxiv.org/abs/2610.07250 · HTML: https://mlsanity.com/paper/2610.07250/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 5 Oct 2026.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 20 of 20)

- [Flow-GRPO: Training Flow Matching Models via Online RL](https://mlsanity.com/paper/2505.05470/) (2025)
- [SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning](https://mlsanity.com/paper/2602.08234/) (2026)
- [WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2503.07265/) (2025)
- [On-Policy Context Distillation for Language Models](https://mlsanity.com/paper/2602.12275/) (2026)
- [MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents](https://mlsanity.com/paper/2602.02474/) (2026)
- [Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA](https://mlsanity.com/paper/2304.06027/) (2023)
- [GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluation](https://mlsanity.com/paper/2512.16853/) (2025)
- [Continual Harness: Online Adaptation for Self-Improving Foundation Agents](https://mlsanity.com/paper/2605.09998/) (2026)
- [MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wild](https://mlsanity.com/paper/2603.17187/) (2026)
- [Flow-OPD: On-Policy Distillation for Flow Matching Models](https://mlsanity.com/paper/2605.08063/) (2026)
- [SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning](https://mlsanity.com/paper/2607.14777/) (2026)
- [GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning](https://mlsanity.com/paper/2601.18543/) (2026)
- [D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models](https://mlsanity.com/paper/2605.05204/) (2026)
- [GEMS: Agent-Native Multimodal Generation with Memory and Skills](https://mlsanity.com/paper/2603.28088/) (2026)
- [GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation](https://mlsanity.com/paper/2605.21605/) (2026)
- [ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generation](https://mlsanity.com/paper/2511.11483/) (2025)
- [Qwen-Image-Agent: Bridging the Context Gap in Real-World Image Generation](https://mlsanity.com/paper/2606.26907/) (2026)
- [Co-Evolving Skill Generation and Policy Optimization](https://mlsanity.com/paper/2606.08755/) (2026)
- [CRAFT: Continuous Reasoning and Agentic Feedback Tuning for Multimodal Text-to-Image Generation](https://mlsanity.com/paper/2512.20362/) (2025)
- [MemoGen: Can Past Experience Improve Future Text-to-Image Generation?](https://mlsanity.com/paper/2606.03243/) (2026)

## Abstract (arXiv)

Wrapping an image generation model in an agentic harness can effectively boost Text-to-Image task performance: the harness can leverage memory, skills, workflow orchestration, result verification, and iterative refinement to continually construct and revise prompts, thereby eliciting better images. These gains, however, remain external to the diffusion model and are realized only while the full harness runs. We propose Diffusion On-Policy Context Distillation (D-OPCD), which treats the agent-improved prompt as privileged context and distills the knowledge encoded in the agent harness into the weights of the diffusion model, so that the model retains part of the harness's benefit when conditioned on the original query alone. Using a Text-to-Image agent equipped with our proposed Auto Skill Evolver (ASE), we show that D-OPCD can internalize harness capabilities into the generator's weights, raising the average direct-generation score from 60.52 to 65.09 across four benchmarks. With this knowledge absorbed into the weights, the harness can shed its saturated skills and resume evolving: a second ASE round on the updated generator improves on a skill-free harness by additional 1.83 points, pointing toward text-to-image systems in which harness and model keep improving each other through continual co-evolution.