# The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Order

arXiv: https://arxiv.org/abs/2606.24993 · HTML: https://mlsanity.com/paper/2606.24993/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 15 of 15)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [Multi-Task Learning as a Bargaining Game](https://mlsanity.com/paper/2202.01017/) (2022)
- [Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models](https://mlsanity.com/paper/2010.05874/) (2020)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning](https://mlsanity.com/paper/2110.02600/) (2021)
- [Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2601.18699/) (2026)
- [Commute Your Domains: Trajectory Optimality Criterion for Multi-Domain Learning](https://mlsanity.com/paper/2501.15556/) (2025)
- [Sequence Transferability and Task Order Selection in Continual Learning](https://mlsanity.com/paper/2502.06544/) (2025)

## Built on by (most cited first, 9 of 9)

- [Optimizer Memory Makes Shuffle Order a First-Order Source of Fine-Tuning Noise](https://mlsanity.com/paper/2606.29554/) (2026)
- [First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers](https://mlsanity.com/paper/2607.16821/) (2026)
- [Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfer](https://mlsanity.com/paper/2608.20442/) (2026)
- [Fiber Fingerprints of Hidden Learning-State Dynamics](https://mlsanity.com/paper/2608.15976/) (2026)
- [Activation-Space Order-Swap Geometry: A Site-Asymmetry Audit](https://mlsanity.com/paper/2608.25315/) (2026)
- [Off-Target Effects of Response-Style Alignment in a Korean 27B Language Model](https://mlsanity.com/paper/2609.11291/) (2026)
- [The Free-Recipe Limit: Every Recipe Effect Measures Which Premise of an Idealised Learner Broke](https://mlsanity.com/paper/2609.26160/) (2026)
- [Commutator Memory: Sparse, Path-Local Reading and Steering in Language Models](https://mlsanity.com/paper/2609.34348/) (2026)
- [Conflicting Supervision Moves Commitment, Not Capability: A 12.29σ arrangement effect that is exactly zero under a convention-agnostic score](https://mlsanity.com/paper/2610.00234/) (2026)

## Abstract (arXiv)

Sequential learning is order-dependent: from Pile-style next-token domain adaptation to instruction-SFT and DPO, N candidate sources induce N! possible curricula. We show that the local order effect is governed by a computable geometric quantity, the Lie-bracket commutator of gradient update fields, yielding a pairwise score for whether A->B or B->A is better for a target domain. The pairwise bracket primitive also defines a Lie-Bracket Tournament: with a shared theta_0 target-gradient reference, Hessian symmetry gives Borda/row-sum scores from one Hessian-vector product per source, O(N) dot products, and an O(N log N) sort, without materializing the O(N^2) edge matrix. Empirically, the planner reaches 98.1%/98.9% pairwise accuracy at k=1 for instruction-SFT/DPO, remains at 73.1%/72.2% at k=20, and preserves the original pretraining-domain evidence with 82.4-92.0% accuracy across four LLMs and 91.1% on diffusion. At curriculum scale, it recovers the best of all 3! schedules in 87.5% of trials, ranks 85 Stack programming-language source domains for a Python target in the 99th sampled percentile, and reaches the 99.0-99.6th sampled percentile on 56 MMLU subjects, sharply above the reported descending gradient-norm baseline. These results reframe sequential learning as a geometric tournament problem: commutators provide both local pairwise order information and a scalable primitive for many-domain schedules.