mlsanityData to 14 Oct 2026

Enhancing Latent Computation in Transformers with Latent Tokens

arXiv 2505.12629 ↗First posted 19 May 2025cs.LG cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Augmenting large language models (LLMs) with auxiliary tokens has emerged as a promising strategy for enhancing model performance. In this work, we introduce a lightweight method termed latent tokens; these are dummy tokens that may be non-interpretable in natural language but steer the autoregressive decoding process of a Transformer-based LLM via the attention mechanism. The proposed latent tokens can be seamlessly integrated with a pre-trained Transformer, trained in a parameter-efficient manner, and applied flexibly at inference time, while adding minimal complexity overhead to the existing infrastructure of standard Transformers. We propose several hypotheses about the underlying mechanisms of latent tokens and design synthetic tasks accordingly to verify them. Numerical results confirm that the proposed method noticeably outperforms the baselines, particularly in the out-of-distribution generalization scenarios, highlighting its potential in improving the adaptability of LLMs.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 32 of its 34 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

32 of 34 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  11. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  12. 2023Vision Transformers Need Registerscited by 871
20 more
  1. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  2. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  3. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  4. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  5. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  6. 2023Think before you speak: Training Language Models With Pause Tokenscited by 264
  7. 2022Recurrent Memory Transformercited by 203
  8. 2020Rethinking Positional Encoding in Language Pre-trainingcited by 197
  9. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 191
  10. 2024The pitfalls of next-token predictioncited by 160
  11. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 137
  12. 2023Guiding Language Model Reasoning with Planning Tokenscited by 58
  13. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 50
  14. 2018Learning To Split and Rephrase From Wikipedia Edit Historycited by 38
  15. 2023Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encodingcited by 32
  16. 2023Understanding In-Context Learning from Repetitionscited by 29
  17. 2024On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoningcited by 26
  18. 2024Deliberation in Latent Space via Differentiable Cache Augmentationcited by 26
  19. 2023Adaptive Computation with Elastic Input Sequencecited by 22
  20. 2024Rethinking Thinking Tokens: Understanding Why They Underperform in Practicecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 8.

14 papers.

Sort:

By area:

  1. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 109
  2. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 78
  3. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 43
  4. 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
  5. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 19
  6. 2026Residual Context Diffusion Language Modelscited by 10
  7. 2025LatentEvolve: Self-Evolving Test-Time Scaling in Latent Spacecited by 9
  8. 2025LaRe: Latent Refocusing for Multimodal Reasoningcited by 8
  9. 2026From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoningcited by 5
  10. 2025Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Spacecited by 4
  11. 2026Capabilities and Fundamental Limits of Latent Chain-of-Thoughtcited by 4
  12. 2026Exploring Collaboration between a language and a non-language agentcited by 1
2 more
  1. 2026RuPLaR : Efficient Latent Compression of LLM Reasoning Chains with Rule-Based Priors From Multi-Step to One-Stepcited by 0
  2. 2026CRAMER: Control via Request-Aware Masking for Editing Recommenderscited by 0