mlsanityData to 14 Oct 2026

From Retrieval to Weights: Parametric Individualization of Small Language Models with Individual Text Corpora

arXiv 2609.10155 ↗First posted 9 Sep 2026cs.CL cs.IRcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

We approach a cognitive simulation perspective on episodic and semantic memory in multiple-choice question answering by incorporating text from individual text corpora (ITC) into retrieval-augmented generation and DoRA fine-tuning. We web-crawl the search histories of 515 participants who answered 36 multiple-choice knowledge items and analyze a stratified subsample of 150 participants. For each participant, one DoRA adapter consolidates their ITC into a small language model (SLM) whose baseline correctness falls below the participants' lowest quartile. The adapter measurably writes the ITC into the weights: it fits its own participant's held-out text better than other participants' texts (dz =1.27), an individuality effect that increases with ITC size in rank order. On the generalized knowledge test, however, the adapter adds knowledge rather than alignment with the individual: log-loss match improves, whereas match accuracy under a bias-corrected PMI readout does not, and retrieval adds nothing on top. Our results demonstrate that ITCs can be consolidated into the weights of SLMs, an encouraging basis for individualized tutoring agents, and we discuss how to move from there toward a realistic simulation of episodic and semantic memory at the individual level.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 13 of its 45 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

13 of 45 references.

References · 13

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Language Models as Knowledge Bases?cited by 1,936
  3. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  4. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  5. 2022Using cognitive psychology to understand GPT-3cited by 398
  6. 2024LoRA Learns Less and Forgets Lesscited by 328
  7. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 263
  8. 2024Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuningcited by 90
  9. 2022Continual Pre-Training Mitigates Forgetting in Language and Visioncited by 83
  10. 2024Human-inspired Episodic Memory for Infinite Context LLMscited by 62
  11. 2024Psychometric Alignment: Capturing Human Knowledge Distributions via Language Modelscited by 18
  12. 2025Do LLMs Make Mistakes Like Students? Exploring Natural Alignment between Language Models and Human Error Patternscited by 8
1 more
  1. 2024Individual Text Corpora Predict Openness, Interests, Knowledge and Level of Educationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Individual Text Corpora Predict User-Specific Knowledge: Benchmarks of Individualized Knowledge Simulation ↗
  2. 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Models
  3. 2025Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home
  4. 2024Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis
  5. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries
  6. 2025Self-Routing RAG: Binding Selective Retrieval with Knowledge Verbalization
  7. 2026To Memorize or to Retrieve: Scaling the Interaction Between Pretraining and Retrieval
  8. 2026EAR: Entity-Aware Partitioning Approach for Retrieval-Augmented Generation Development