# Bayesian scaling laws for in-context learning

arXiv: https://arxiv.org/abs/2410.16531 · HTML: https://mlsanity.com/paper/2410.16531/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 41 of its 49 references that are in the map.
- First posted on arXiv 21 Oct 2024.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 30 of 41)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://mlsanity.com/paper/2208.01066/) (2022)
- [What learning algorithm is in-context learning? Investigations with linear models](https://mlsanity.com/paper/2211.15661/) (2022)
- [Safety Alignment Should Be Made More Than Just a Few Tokens Deep](https://mlsanity.com/paper/2406.05946/) (2024)
- [Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations](https://mlsanity.com/paper/2310.06387/) (2023)
- [Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection](https://mlsanity.com/paper/2306.04637/) (2023)
- [Transformers learn to implement preconditioned gradient descent for in-context learning](https://mlsanity.com/paper/2306.00297/) (2023)
- [Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models](https://mlsanity.com/paper/2307.14539/) (2023)
- [Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive](https://mlsanity.com/paper/2402.13228/) (2024)
- [Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications](https://mlsanity.com/paper/2402.05162/) (2024)
- [Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning](https://mlsanity.com/paper/2010.15875/) (2020)
- [Many-Shot In-Context Learning](https://mlsanity.com/paper/2404.11018/) (2024)
- [A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity](https://mlsanity.com/paper/2401.01967/) (2024)
- [Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs](https://mlsanity.com/paper/2407.15549/) (2024)
- [Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking](https://mlsanity.com/paper/2402.14811/) (2024)
- [Revisiting Neural Scaling Laws in Language and Vision](https://mlsanity.com/paper/2209.06640/) (2022)
- [The Learnability of In-Context Learning](https://mlsanity.com/paper/2303.07895/) (2023)
- [What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization](https://mlsanity.com/paper/2305.19420/) (2023)
- [Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks](https://mlsanity.com/paper/2311.12786/) (2023)
- [Broken Neural Scaling Laws](https://mlsanity.com/paper/2210.14891/) (2022)

## Built on by (most cited first, 19 of 19)

- [The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs](https://mlsanity.com/paper/2509.09677/) (2025)
- [How Do Large Language Monkeys Get Their Power (Laws)?](https://mlsanity.com/paper/2502.17578/) (2025)
- [Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering](https://mlsanity.com/paper/2511.00617/) (2025)
- [In-Context Learning Is Provably Bayesian Inference: A Generalization Theory for Meta-Learning](https://mlsanity.com/paper/2510.10981/) (2025)
- [A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoning](https://mlsanity.com/paper/2411.04105/) (2024)
- [ContextNav: Towards Agentic Multimodal In-Context Learning](https://mlsanity.com/paper/2510.04560/) (2025)
- [Transformer learns the cross-task prior and regularization for in-context learning](https://mlsanity.com/paper/2505.12138/) (2025)
- [Next-token pretraining implies in-context learning](https://mlsanity.com/paper/2505.18373/) (2025)
- [Why is prompting hard? Understanding prompts on binary sequence predictors](https://mlsanity.com/paper/2502.10760/) (2025)
- [Transformers Don't In-Context Learn Least Squares Regression](https://mlsanity.com/paper/2507.09440/) (2025)
- [Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space](https://mlsanity.com/paper/2605.12412/) (2026)
- [Evaluating the Robustness of Chinchilla Compute-Optimal Scaling](https://mlsanity.com/paper/2509.23963/) (2025)
- [Learning to Remember, Learn, and Forget in Attention-Based Models](https://mlsanity.com/paper/2602.09075/) (2026)
- [Item Response Scaling Laws: A Measurement Theory Approach for Efficient and Generalizable Neural Scaling Estimation](https://mlsanity.com/paper/2606.07616/) (2026)
- [On the Relationship Between the Choice of Representation and In-Context Learning](https://mlsanity.com/paper/2510.08372/) (2025)
- [Aligning Probabilistic Beliefs under Informative Missingness: LLM Steerability in Clinical Reasoning](https://mlsanity.com/paper/2512.00479/) (2025)
- [Limits of Reliability and Scaling in Language Models](https://mlsanity.com/paper/2607.14112/) (2026)
- [You Are What You Read: Misalignment via In-Context Persona Induction](https://mlsanity.com/paper/2609.06851/) (2026)
- [Linear RNN Scaling Laws: When Longer Sequences Beat More Sequences](https://mlsanity.com/paper/2609.27964/) (2026)

## Abstract (arXiv)

In-context learning (ICL) is a powerful technique for getting language models to perform complex tasks with no training updates. Prior work has established strong correlations between the number of in-context examples provided and the accuracy of the model's predictions. In this paper, we seek to explain this correlation by showing that ICL approximates a Bayesian learner. This perspective gives rise to a novel Bayesian scaling law for ICL. In experiments with mboxGPT-2 models of different sizes, our scaling law matches existing scaling laws in accuracy while also offering interpretable terms for task priors, learning efficiency, and per-example probabilities. To illustrate the analytic power that such interpretable scaling laws provide, we report on controlled synthetic dataset experiments designed to inform real-world studies of safety alignment. In our experimental protocol, we use SFT or DPO to suppress an unwanted existing model capability and then use ICL to try to bring that capability back (many-shot jailbreaking). We then study ICL on real-world instruction-tuned LLMs using capabilities benchmarks as well as a new many-shot jailbreaking dataset. In all cases, Bayesian scaling laws accurately predict the conditions under which ICL will cause suppressed behaviors to reemerge, which sheds light on the ineffectiveness of post-training at increasing LLM safety.