mlsanityData to 7 Oct 2026

Forking: Sudden Overfitting Under Replay

arXiv 2610.00394 ↗First posted 30 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper studies forking, a generalization failure discovered in NanoGPT autoresearch. Under data replay, models with an over-encoding n-gram memory branch show a sharp separation of training and validation loss at epoch boundaries, resembling the shape of forks. We study this phenomenon in a controlled vanilla NanoGPT setting and reproduce it in a DeepSeek-style model with Engram. Mechanistically, repeated updates sharpen the continuations observed in training while suppressing the probability of unseen continuations, whose loss grows with each pass. The n-gram module creates weakly interacting context-specific subspaces, amplifying this effect. Low-frequency contexts contribute most of the gap, whereas larger training budgets and heavily crowded tables suppress it. We also observe forking in short-budget, heavily repeated SFT and RL-like regimes. The contributions of this paper are twofold: (1) Forking reveals yet another curious phenomenon in deep learning, in addition to grokking and double descent. (2) Forking is an unexpected and unpleasant by-product of tricks proposed by autoresearch agents. While these agents produce an enormous number of results that seem useful, we should always be careful with their results.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 53 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

36 of 53 references.

References · 36

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  3. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  5. 2023LIMA: Less Is More for Alignmentcited by 1,080
  6. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  7. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  8. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  9. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  11. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  12. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
24 more
  1. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  2. 2023Scaling Data-Constrained Language Modelscited by 391
  3. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
  4. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  5. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  6. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  7. 2026Towards End-to-End Automation of AI Researchcited by 150
  8. 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 148
  9. 2022Scaling Laws and Interpretability of Learning from Repeated Datacited by 147
  10. 2023Emergent and Predictable Memorization in Large Language Modelscited by 147
  11. 2024Byte Latent Transformer: Patches Scale Better Than Tokenscited by 137
  12. 2019Large Memory Layers with Product Keyscited by 120
  13. 2023To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisiscited by 88
  14. 2016Improving Neural Language Models with a Continuous Cachecited by 81
  15. 2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Modelscited by 78
  16. 2025Over-Tokenized Transformer: Vocabulary is Generally Worth Scalingcited by 42
  17. 2024Memory Layers at Scalecited by 39
  18. 2026DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compressioncited by 34
  19. 2023Why do Nearest Neighbor Language Models Work?cited by 29
  20. 2025Scaling Embedding Layers in Language Modelscited by 19
  21. 2022N-Grammer: Augmenting Transformers with latent n-gramscited by 13
  22. 2024The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generationcited by 9
  23. 2026Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memorycited by 4
  24. 2026Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Datacited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
  2. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training
  3. 2026Fork-Think with Confidence
  4. 2026Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
  5. 2025Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models
  6. 2025A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective
  7. 2026Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
  8. 2026A Theory of Generalization in Deep Learning