# Compiling Learning Problems into Adaptation Programs for Language Models

arXiv: https://arxiv.org/abs/2609.37371 · HTML: https://mlsanity.com/paper/2609.37371/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 8 of 8)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2303.10512/) (2023)
- [Sparse Structure Search for Parameter-Efficient Tuning](https://mlsanity.com/paper/2206.07382/) (2022)
- [Aletheia: Gradient-Guided Layer Selection for Efficient LoRA Fine-Tuning Across Architectures](https://mlsanity.com/paper/2604.15351/) (2026)
- [Understanding and Guiding Layer Placement in Parameter-Efficient Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2602.04019/) (2026)
- [Rethinking Adapter Placement: A Dominant Adaptation Module Perspective](https://mlsanity.com/paper/2605.06183/) (2026)
- [Localized Adaptation Reveals Distinct Learning Signatures in Transformers](https://mlsanity.com/paper/2607.25663/) (2026)

## Abstract (arXiv)

Model adaptation is typically governed by a fixed recipe, even though different update programs can produce substantially different behavioral outcomes. We introduce adaptation compilation, which reframes where, how, and to what extent a model should adapt as a joint prediction and decision problem. Rather than searching over candidate programs anew for each learning episode, a compiler learns from prior adaptations to predict a vector-valued counterfactual response surface over candidate programs---their expected effects on acquisition, transfer, boundedness, and preservation---and selects a program before adaptation begins. Because this predicted geometry captures multiple behavioral consequences rather than a single winner or scalar score, it can be reused under different downstream priorities without retraining. Across five learning types, preferred programs vary meaningfully across episodes, and this variation is predictable from pre-adaptation information. On Llama-3.1-8B, compiler-selected programs approach exhaustive search while outperforming global and objective-specific defaults. Replication on Gemma-2-9B preserves program heterogeneity and selection headroom, but shows that exploiting this headroom requires accounting for uncertainty when departing from strong defaults. Together, these results show that adaptation search can be amortized across related learning problems, turning prior adaptation experience into a basis for deciding how future learning should occur.