mlsanityData to 7 Oct 2026

Implicit meta-learning may lead language models to trust more reliable sources

arXiv 2310.15047 ↗First posted 23 Oct 2023cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

We demonstrate that LLMs may learn indicators of document usefulness and modulate their updates accordingly. We introduce random strings ("tags") as indicators of usefulness in a synthetic fine-tuning dataset. Fine-tuning on this dataset leads to implicit meta-learning (IML): in further fine-tuning, the model updates to make more use of text that is tagged as useful. We perform a thorough empirical investigation of this phenomenon, finding (among other things) that (i) it occurs in both pretrained LLMs and those trained from scratch, as well as on a vision task, and (ii) larger models and smaller batch sizes tend to give more IML. We also use probing to examine how IML changes the way models store knowledge in their parameters. Finally, we reflect on what our results might imply about capabilities, risks, and controllability of future AI systems. Our code can be found at https://github.com/krasheninnikov/internalization.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

39 of 52 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2022A ConvNet for the 2020scited by 3,545
  6. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  11. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  12. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
27 more
  1. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  2. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  3. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 762
  4. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  5. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  6. 2021Fast Model Editing at Scalecited by 491
  7. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  8. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  9. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  10. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  11. 2021Gradient Matching for Domain Generalizationcited by 258
  12. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  13. 2021On the Origin of Implicit Regularization in Stochastic Gradient Descentcited by 183
  14. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  15. 2022Large Language Models with Controllable Working Memorycited by 169
  16. 2020Editable Neural Networkscited by 154
  17. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  18. 2020Learning explanations that are hard to varycited by 132
  19. 2023Taken out of context: On measuring situational awareness in LLMscited by 131
  20. 2019Stiffness: A New Perspective on Generalization in Neural Networkscited by 46
  21. 2020Hidden Incentives for Auto-Induced Distributional Shiftcited by 42
  22. 2021Cross-document Coreference Resolution over Predicted Mentionscited by 29
  23. 2021Implicit Gradient Alignment in Distributed and Federated Learningcited by 29
  24. 2022Estimating and Penalizing Induced Preference Shifts in Recommender Systemscited by 29
  25. 2021Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learningcited by 16
  26. 2023Memory-Based Meta-Learning on Non-Stationary Distributionscited by 14
  27. 2021SGD Implicitly Regularizes Generalization Errorcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 2.

9 papers.

  1. 2024On Targeted Manipulation and Deception when Optimizing LLMs for User Feedbackcited by 45
  2. 2024Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?cited by 24
  3. 2024Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Factscited by 13
  4. 2025Fresh in memory: Training-order recency is linearly encoded in language model activationscited by 5
  5. 2026Constitutional Midtraining: Content Presence Drives Alignment Gainscited by 3
  6. 2025When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammarscited by 2
  7. 2025Programming by Backprop: An Instruction is Worth 100 Examples When Finetuning LLMscited by 2
  8. 2026Epistemic Goggles: A Pretrained Module that Induces an Epistemic Frame via Gradient Editingcited by 1
  9. 2025Out-of-Context Abduction: LLMs Make Inferences About Procedural Data Leveraging Declarative Facts in Earlier Training Datacited by 0