# Dynamic Latent Routing

arXiv: https://arxiv.org/abs/2605.14323 · HTML: https://mlsanity.com/paper/2605.14323/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 55 references that are in the map.
- First posted on arXiv 14 May 2026.

## Builds on (most cited first, 12 of 12)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Think before you speak: Training Language Models With Pause Tokens](https://mlsanity.com/paper/2310.02226/) (2023)
- [Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning](https://mlsanity.com/paper/2502.03275/) (2025)
- [Temporal Abstraction in Reinforcement Learning with the Successor Representation](https://mlsanity.com/paper/2110.05740/) (2021)
- [RLP: Reinforcement as a Pretraining Objective](https://mlsanity.com/paper/2510.01265/) (2025)
- [Understanding Addition and Subtraction in Transformers](https://mlsanity.com/paper/2402.02619/) (2024)
- [Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought](https://mlsanity.com/paper/2604.22709/) (2026)
- [Why Representation Engineering Works: A Theoretical and Empirical Study in Vision-Language Models](https://mlsanity.com/paper/2503.22720/) (2025)
- [Zero-Shot Compositional Policy Learning via Language Grounding](https://mlsanity.com/paper/2004.07200/) (2020)
- [SteeringSafety: Benchmarking Representation Steering in LLMs Across Safety Perspectives](https://mlsanity.com/paper/2509.13450/) (2025)

## Abstract (arXiv)

We investigate the temporal concatenation of sub-policies in Markov Decision Processes (MDP) with time-varying reward functions. We introduce General Dijkstra Search (GDS), and prove that globally optimal goal-reaching policies can be recovered through temporal composition of intermediate optimal sub-policies. Motivated by the "search, select, update" principle underlying GDS, we propose Dynamic Latent Routing (DLR), a language-model post-training method that jointly learns discrete latent codes, routing policies, and model parameters through dynamic search in a single training stage. In low-data fine-tuning settings, DLR matches or outperforms supervised fine-tuning across four datasets and six models, achieving a mean gain of +6.6 percentage points, while prior discrete-latent baselines consistently underperform SFT. Mechanistic analyses and targeted code ablations show that DLR learns structured routing behaviors with distinct causal roles.