Autolearn: Learn by Surprise, Commit by Proof
Not on the map. Why?
Abstract (arXiv)
We propose Autolearn, a framework that enables language models to learn from documents they read, with no external supervision. Passages that produce anomalously high per-token loss are flagged, verified through a self-generated Q&A chain, and trained on with conviction-proportional β2 adjustment. We introduce the perturbation gap (paraphrase-to-original perplexity ratio) as a metric that distinguishes memorization from understanding. The key mechanism is the training data format: Q&A-format training drives the perturbation gap below the pre-trained baseline (2.098 vs. 2.204, Delta = -0.106, > 10σ), suppressing token-sequence memorization, while standard fine-tuning's best attempt remains within noise (Delta = -0.010, < 1σ). Across four models spanning Qwen3 and Phi-4 families, Autolearn is the only method that enters this regime. Stochastic evaluation reveals passage-specific knowledge acquisition: the probability of generating a correct novel fact rises from 6% to 54% after training (p < 10-4), and Q&A format outperforms standard fine-tuning on genuinely novel facts. The system is self-extinguishing: learned content reduces surprisal below threshold and is skipped on re-encounter.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 2 Apr 2026.
Builds on
25 of 38 references.
References · 25
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
13 more
- 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
- 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
- 2023Do Large Language Models Know What They Don't Know?cited by 238
- 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
- 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 157
- 2024Distinguishing the Knowable from the Unknowable with Language Modelscited by 45
- 2024Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teachingcited by 17
- 2025Pre-training Limited Memory Language Models with Internal and External Knowledgecited by 11
- 2025Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probingcited by 7
- 2025ALAS: Autonomous Learning Agent for Self-Updating Language Modelscited by 6
- 2025Rote Learning Considered Useful: Generalizing over Memorized Data in LLMscited by 5
- 2025Self-Evolving LLMs via Continual Instruction Tuningcited by 4
- 2025STABLE: Gated Continual Learning for Large Language Modelscited by 2
Built on by
Not cited yet.