# MIRROR: From Imitation to Internalization in LLM Personalization

arXiv: https://arxiv.org/abs/2610.09795 · HTML: https://mlsanity.com/paper/2610.09795/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Placed in

- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Builds on (most cited first, 27 of 27)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models](https://mlsanity.com/paper/2601.18734/) (2026)
- [Reinforcement Learning via Self-Distillation](https://mlsanity.com/paper/2601.20802/) (2026)
- [Self-Distillation Enables Continual Learning](https://mlsanity.com/paper/2601.19897/) (2026)
- [MemOS: A Memory OS for AI System](https://mlsanity.com/paper/2507.03724/) (2025)
- [Kimi K3: Open Frontier Intelligence](https://mlsanity.com/paper/2607.24653/) (2026)
- [Personalized Language Modeling from Personalized Human Feedback](https://mlsanity.com/paper/2402.05133/) (2024)
- [AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution](https://mlsanity.com/paper/2603.01145/) (2026)
- [LongLaMP: A Benchmark for Personalized Long-form Text Generation](https://mlsanity.com/paper/2407.11016/) (2024)
- [OPSDL: On-Policy Self-Distillation for Long-Context Language Models](https://mlsanity.com/paper/2604.17535/) (2026)
- [Self-Distilled Policy Gradient](https://mlsanity.com/paper/2606.04036/) (2026)
- [POPI: Personalizing LLMs via Optimized Natural Language Preference Inference](https://mlsanity.com/paper/2510.17881/) (2025)
- [OPSD Compresses What RLVR Teaches: A Post-RL Compaction Stage for Reasoning Models](https://mlsanity.com/paper/2605.06188/) (2026)
- [Hierarchical Memory Orchestration for Personalized Persistent Agents](https://mlsanity.com/paper/2604.01670/) (2026)
- [Rethinking Personalization in Large Language Models at the Token Level](https://mlsanity.com/paper/2603.06595/) (2026)
- [Beyond Retrieval: Learning Compact User Representations for Scalable LLM Personalization](https://mlsanity.com/paper/2606.04547/) (2026)
- [An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift](https://mlsanity.com/paper/2601.05882/) (2026)
- [Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answering](https://mlsanity.com/paper/2605.12645/) (2026)
- [PGMem: Tightly Coupled Persona-Memory Graph for Lifelong Personalized Agents](https://mlsanity.com/paper/2608.01708/) (2026)
- [Locating and Controlling Implicit Personalization in Large Language Models](https://mlsanity.com/paper/2608.11735/) (2026)
- [EviDx: Evidence-Aware Active Diagnosis with Scaffolded LLM Agents](https://mlsanity.com/paper/2608.24570/) (2026)
- [From Volume to Value: Preference-Aligned Memory Construction for On-Device RAG](https://mlsanity.com/paper/2605.18271/) (2026)
- [Fine-Tuning Without Forgetting via Loss-Adaptive Learning Rates](https://mlsanity.com/paper/2605.20005/) (2026)
- [Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifacts](https://mlsanity.com/paper/2606.10334/) (2026)
- [Hypotheses-Guided Self Distillation for Continual Personalization](https://mlsanity.com/paper/2609.00251/) (2026)

## Abstract (arXiv)

The demand for personalized LLMs is shifting from style imitation toward content quality. We investigate whether self-distillation can bridge this gap in existing fine-tuning paradigm. To address this limitation, we introduce MIRROR(Meta- personalization by Internalizing Reference-Revealed On-policy Reflections), a novel self-distillation framework that shifts LLM personalization from imitation toward preference internalization. First, we replace reference-token imitation with reference-revealed on-policy self-distillation, aligning the model's next-token distributions along its own generation trajectories with those of its reference-conditioned self, thereby internalizing user preferences rather than reproducing reference wording.Second, we introduce MIRROR-F, a focal plug-in that augments on-policy distributional alignment with selective supervision over informative reference tokens, thereby strengthening content generation while preserving user-specific expression. Across three personalized generation benchmarks, two model scales, and complementary reference-based and LLM-based evaluations, MIRROR and MIRROR-F achieve leading overall personalization performance and superior text quality, while exhibiting less catastrophic forgetting than SFT-based baselines on three unseen personalized generation tasks. The gains are consistent across model scales and application scenarios, translating to improved performance in LLM personalization tasks.