The Geometry of Sequential Learning: Lie-Bracket Prediction of Transfer Order
On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods
Abstract (arXiv)
Sequential learning is order-dependent: from Pile-style next-token domain adaptation to instruction-SFT and DPO, N candidate sources induce N! possible curricula. We show that the local order effect is governed by a computable geometric quantity, the Lie-bracket commutator of gradient update fields, yielding a pairwise score for whether A->B or B->A is better for a target domain. The pairwise bracket primitive also defines a Lie-Bracket Tournament: with a shared theta_0 target-gradient reference, Hessian symmetry gives Borda/row-sum scores from one Hessian-vector product per source, O(N) dot products, and an O(N log N) sort, without materializing the O(N^2) edge matrix. Empirically, the planner reaches 98.1%/98.9% pairwise accuracy at k=1 for instruction-SFT/DPO, remains at 73.1%/72.2% at k=20, and preserves the original pretraining-domain evidence with 82.4-92.0% accuracy across four LLMs and 91.1% on diffusion. At curriculum scale, it recovers the best of all 3! schedules in 87.5% of trials, ranks 85 Stack programming-language source domains for a Python target in the 99th sampled percentile, and reaches the 99.0-99.6th sampled percentile on 56 MMLU subjects, sharply above the reported descending gradient-norm baseline. These results reframe sequential learning as a geometric tournament problem: commutators provide both local pairwise order information and a scalable primitive for many-domain schedules.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 15 of its 34 references that are in the map. First posted on arXiv 23 Jun 2026.
Builds on
15 of 34 references.
References · 15
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
- 2020Gradient Surgery for Multi-Task Learningcited by 1,210
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
- 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
- 2022Multi-Task Learning as a Bargaining Gamecited by 209
- 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 167
- 2020Understanding Self-Training for Gradual Domain Adaptationcited by 149
- 2021Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learningcited by 16
3 more
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026Optimizer Memory Makes Shuffle Order a First-Order Source of Fine-Tuning Noisecited by 6
- 2026First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformerscited by 3
- 2026Fiber Fingerprints of Hidden Learning-State Dynamicscited by 2
- 2026Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfercited by 1
- 2026Activation-Space Order-Swap Geometry: A Site-Asymmetry Auditcited by 0
- 2026Off-Target Effects of Response-Style Alignment in a Korean 27B Language Modelcited by 0
- 2026The Free-Recipe Limit: Every Recipe Effect Measures Which Premise of an Idealised Learner Brokecited by 0
- 2026Commutator Memory: Sparse, Path-Local Reading and Steering in Language Modelscited by 0
- 2026Commutator Memory: Sparse, Path-Local Reading and Steering in Language Models28 Sep 2026
- 2026Off-Target Effects of Response-Style Alignment in a Korean 27B Language Model10 Sep 2026
- 2026Activation-Space Order-Swap Geometry: A Site-Asymmetry Audit26 Aug 2026
- 2026Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfer20 Aug 2026
- 2026Fiber Fingerprints of Hidden Learning-State Dynamics17 Aug 2026
- 2026The Free-Recipe Limit: Every Recipe Effect Measures Which Premise of an Idealised Learner Broke9 Aug 2026
- 2026First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers18 Jul 2026
- 2026Optimizer Memory Makes Shuffle Order a First-Order Source of Fine-Tuning Noise28 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026RIFT: Reordered Instruction Following Testbed To Evaluate Instruction Following in Singular Multistep Prompt Structures
- 2025Memory Determines Learning Direction: A Theory of Gradient-Based Optimization in State Space Models
- 2026Learning What to Imitate: Entropy-Aware Distribution Mixing
- 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions
- 2026Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarks
- 2026A Comparative Analysis of Attention versus State-Space Models for In-Context Learning
- 2026The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling
- 2026ForkLeft: Entropy-First Rollouts for Prefix-Aligned Autoregressive-to-Diffusion Distillation