# From Style to Facts: Mapping the Boundaries of Knowledge Injection with Finetuning

arXiv: https://arxiv.org/abs/2503.05919 · HTML: https://mlsanity.com/paper/2503.05919/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 7 Mar 2025.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"](https://mlsanity.com/paper/2309.12288/) (2023)
- [LoRA Learns Less and Forgets Less](https://mlsanity.com/paper/2405.09673/) (2024)
- [Physics of Language Models: Part 3.1, Knowledge Storage and Extraction](https://mlsanity.com/paper/2309.14316/) (2023)
- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://mlsanity.com/paper/2405.05904/) (2024)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Taken out of context: On measuring situational awareness in LLMs](https://mlsanity.com/paper/2309.00667/) (2023)
- [Instruction-tuned Language Models are Better Knowledge Learners](https://mlsanity.com/paper/2402.12847/) (2024)
- [Understanding Finetuning for Factual Knowledge Extraction](https://mlsanity.com/paper/2406.14785/) (2024)
- [Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts](https://mlsanity.com/paper/2412.04614/) (2024)
- [Beyond Memorization: The Challenge of Random Memory Access in Language Models](https://mlsanity.com/paper/2403.07805/) (2024)

## Built on by (most cited first, 8 of 8)

- [Learning Facts at Scale with Active Reading](https://mlsanity.com/paper/2508.09494/) (2025)
- [LLMLagBench: Identifying Temporal Training Boundaries in Large Language Models](https://mlsanity.com/paper/2511.12116/) (2025)
- [Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts](https://mlsanity.com/paper/2604.08519/) (2026)
- [Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMs](https://mlsanity.com/paper/2510.09885/) (2025)
- [REFLEX: Self-Refining Explainable Fact-Checking via Verdict-Anchored Style Control](https://mlsanity.com/paper/2511.20233/) (2025)
- [ECG-LLM -- training and evaluation of domain-specific large language models for electrocardiography](https://mlsanity.com/paper/2510.18339/) (2025)
- [ContinuousBench: Can Differentially Private Synthetic Text Improve Capabilities?](https://mlsanity.com/paper/2606.01849/) (2026)
- [Learning a Fact Is Not Learning How to Retrieve It](https://mlsanity.com/paper/2610.03251/) (2026)

## Abstract (arXiv)

Finetuning provides a scalable and cost-effective means of customizing language models for specific tasks or response styles, with greater reliability than prompting or in-context learning. In contrast, the conventional wisdom is that injecting knowledge via finetuning results in brittle performance and poor generalization. We argue that the dichotomy of "task customization" (e.g., instruction tuning) and "knowledge injection" (e.g., teaching new facts) is a distinction without a difference. We instead identify concrete factors that explain the heterogeneous effectiveness observed with finetuning. To this end, we conduct a large-scale experimental study of finetuning the frontier Gemini v1.5 model family on a spectrum of datasets that are artificially engineered to interpolate between the strengths and failure modes of finetuning. Our findings indicate that question-answer training data formats provide much stronger knowledge generalization than document/article-style training data, numerical information can be harder for finetuning to retain than categorical information, and models struggle to apply finetuned knowledge during multi-step reasoning even when trained on similar examples -- all factors that render "knowledge injection" to be especially difficult, even after controlling for considerations like data augmentation and information volume. On the other hand, our findings also indicate that it is not fundamentally more difficult to finetune information about a real-world event than information about what a model's writing style should be.