mlsanityData to 7 Oct 2026

Latent Chain-of-Thought Improves Structured-Data Transformers

arXiv 2605.11262 ↗First posted 11 May 2026cs.LGcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought and more broadly test-time compute are known to augment the expressive capabilities of language models and have led to major innovations in reasoning. Motivated by this success, this paper explores latent chain-of-thought as well as the impact of depth and looping for time-series and tabular data. We propose a recurrent scheme in which a structured-data transformer, after an initial forward pass, compresses its query-position hidden states into feedback tokens that are appended to the input and processed again, allowing multiple rounds of latent computation before prediction. We compare CoT models against a same-depth no-CoT baseline, a deeper baseline matched to the CoT model in effective depth, and a looped transformer with weight-tied recurrence but no additional chain-of-thought tokens. Across 36 datasets in time-series forecasting and tabular prediction, latent chain-of-thought improves over the baseline on 7/9 time-series datasets (+12.63% average gain) and 23/27 tabular datasets (+3.25% average gain), with CoT models performing best on average in both settings. We also show that the benefit of CoT extends to pretrained foundation models: applying latent CoT to nanoTabPFN, a small open-source tabular foundation model, improves its performance above the much larger TabPFN-v2 on TabArena. Together, these results demonstrate that chain-of-thought is a useful axis for scaling test-time compute for structured data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 22 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

18 of 22 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Continuous Chain of Thought Enables Parallel Exploration and Reasoning29 May
  2. 2024Training Large Language Models to Reason in a Continuous Latent Space9 Dec
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 15

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  4. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  5. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  6. 2023A decoder-only foundation model for time-series forecastingcited by 650
  7. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 643
  8. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  9. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  10. 2025TabICL: A Tabular Foundation Model for In-Context Learning on Large Datacited by 267
  11. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  12. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoningcited by 118
3 more
  1. 2025TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Modelscited by 115
  2. 2026Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Modelscited by 4
  3. 2025Mantis: A Foundation Model for Mechanistic Disease Forecastingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer
  2. 2025Eliciting Chain-of-Thought Reasoning for Time Series Analysis using Reinforcement Learning
  3. 2026Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
  4. 2026Latent Reasoning with Normalizing Flows
  5. 2025Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model
  6. 2025A Formal Comparison Between Chain of Thought and Latent Thought
  7. 2024Autoregressive + Chain of Thought = Recurrent: Recurrence's Role in Language Models' Computability and a Revisit of Recurrent Transformer
  8. 2024From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency