# Constitutional Midtraining: Content Presence Drives Alignment Gains

arXiv: https://arxiv.org/abs/2607.26654 · HTML: https://mlsanity.com/paper/2607.26654/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [Safety Alignment Should Be Made More Than Just a Few Tokens Deep](https://mlsanity.com/paper/2406.05946/) (2024)
- [A mathematical theory of semantic development in deep neural networks](https://mlsanity.com/paper/1810.10531/) (2018)
- [Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks](https://mlsanity.com/paper/2006.13198/) (2020)
- [Agentic Misalignment: How LLMs Could Be Insider Threats](https://mlsanity.com/paper/2510.05179/) (2025)
- [The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems](https://mlsanity.com/paper/2503.03750/) (2025)
- [Language Models Resist Alignment: Evidence From Data Compression](https://mlsanity.com/paper/2406.06144/) (2024)
- [Safety Pretraining: Toward the Next Generation of Safe AI](https://mlsanity.com/paper/2504.16980/) (2025)
- [Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment](https://mlsanity.com/paper/2601.10160/) (2026)
- [Model Spec Midtraining: Improving How Alignment Training Generalizes](https://mlsanity.com/paper/2605.02087/) (2026)
- [How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining](https://mlsanity.com/paper/2511.18903/) (2025)
- [Implicit meta-learning may lead language models to trust more reliable sources](https://mlsanity.com/paper/2310.15047/) (2023)
- [When Should We Introduce Safety Interventions During Pretraining?](https://mlsanity.com/paper/2601.07087/) (2026)
- [Curriculum Learning for LLM Pretraining: An Analysis of Learning Dynamics](https://mlsanity.com/paper/2601.21698/) (2026)

## Built on by (most cited first, 4 of 4)

- [Constitutional adapters: Inference-time interventions for misalignment and misuse](https://mlsanity.com/paper/2609.36657/) (2026)
- [Character Training for Risk-Averse Agents](https://mlsanity.com/paper/2609.38093/) (2026)
- [Pre-training interventions, ex post facto: Grafting model beliefs across checkpoints](https://mlsanity.com/paper/2610.00767/) (2026)
- [Predicting Alignment Generalization with Value Representations](https://mlsanity.com/paper/2610.12410/) (2026)

## Abstract (arXiv)

Post-training alignment is often shallow, eroding under fine-tuning. It remains untested as to whether constitutional midtraining interventions can produce durable alignment when cleanly isolated from post-training. We build a 394M-token constitutional corpus from Anthropic's Constitution and apply constitutional midtraining at 120B scale, where principled, values-based content is inserted into midtraining. A 2x2 design (curriculum ordering x deliberative reasoning) was used to produce four constitutionally midtrained conditions, plus a control, which were evaluated on self-generated and established benchmarks including alignment under pressure, value conflict resolution, blackmail, and emergent misalignment. All models were evaluated across three stages: post-midtraining, post-SFT, and post-benign fine-tuning. Constitutionally midtrained models outperformed the control on alignment generalization and durability, notably on blackmail: SFT instilled a blackmail propensity in all models, but constitutional midtraining blunted it, with the advantage surviving benign fine-tuning (-17.5pp). This durability did not extend to settings that required active resistance to in-context pressure or conflict, where the advantage attenuates after SFT. The presence of constitutional content at midtraining also mattered more than its structure, and constitutional midtraining incurred no capability cost, on average, at any stage (MMLU, ARC-Easy, piqa, GSM8K). A modest amount of constitutional content at midtraining could therefore yield broad, persistent alignment gains, offering a cheap, complementary addition to SFT-centered pipelines. Code, data, and models are available.