mlsanityData to 7 Oct 2026

Step-resolved data attribution for looped transformers

arXiv 2602.10097 ↗First posted 10 Feb 2026cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

We study how individual training examples shape the internal computation of looped transformers, where a shared block is applied for τ recurrent iterations to enable latent reasoning. Existing training-data influence estimators such as TracIn yield a single scalar score that aggregates over all loop iterations, obscuring when during the recurrent computation a training example matters. We introduce textitStep-Decomposed Influence (SDI), which decomposes TracIn into a length-τ influence trajectory by unrolling the recurrent computation graph and attributing influence to specific loop iterations. To make SDI practical at transformer scale, we propose a TensorSketch implementation that never materialises per-example gradients. Experiments on looped GPT-style models and algorithmic reasoning tasks show that SDI scales excellently, matches full-gradient baselines with low error and supports a broad range of data attribution and interpretability tasks with per-step insights into the latent reasoning process.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 32 of its 42 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

32 of 42 references.

References · 32

  1. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  2. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  3. 2018Universal Transformerscited by 613
  4. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  5. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  6. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  7. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  8. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  9. 2023Looped Transformers as Programmable Computerscited by 224
  10. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 223
  11. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  12. 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
20 more
  1. 2021Deep Learning Through the Lens of Example Difficultycited by 139
  2. 2023Looped Transformers are Better at Learning Learning Algorithmscited by 135
  3. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
  4. 2025Thought Anchors: Which LLM Reasoning Steps Matter?cited by 126
  5. 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 120
  6. 2024Looped Transformers for Length Generalizationcited by 91
  7. 2020TimeSHAP: Explaining Recurrent Models through Sequence Perturbationscited by 73
  8. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 72
  9. 2025A Survey on Latent Reasoningcited by 71
  10. 2024RecurrentGemma: Moving Past Transformers for Efficient Open Language Modelscited by 61
  11. 2025Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrencecited by 41
  12. 2023Learning to Solve Constraint Satisfaction Problems with Recurrent Transformercited by 21
  13. 2020Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functionscited by 18
  14. 2025Block-Recurrent Dynamics in Vision Transformerscited by 16
  15. 2019On Attribution of Recurrent Neural Network Predictions via Additive Decompositioncited by 15
  16. 2025Universal Reasoning Modelcited by 11
  17. 2022Influence Functions for Sequence Tagging Modelscited by 9
  18. 2024Revisit, Extend, and Enhance Hessian-Free Influence Functionscited by 8
  19. 2025GraSS: Scalable Data Attribution with Gradient Sparsification and Sparse Projectioncited by 8
  20. 2025Tensor Sketch: Fast and Scalable Polynomial Kernel Approximationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulationcited by 48
  2. 2026When Does Recurrence Become an Algorithm? Convergence Selection in Weight-Tied Looped Transformerscited by 5
  3. 2026T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning with Dynamic Routingcited by 1
  4. 2026SMolLM: Small Language Models Learn Small Molecular Grammarcited by 0
  5. 2026Loop Dropout: Regularizing Shared Updates in Looped Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond the Training Horizon: Mechanisms and Limits of Length Generalization in Looped Transformers
  2. 2026LoopMTP: A looped transformer guided by latent multi-token prediction
  3. 2026Training-Free Looped Transformers
  4. 2025Closed-Loop Transformers: Autoregressive Modeling as Iterative Latent Equilibrium
  5. 2026Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
  6. 2026Stability and Generalization in Looped Transformers
  7. 2026Adaptive Loops and Memory in Transformers: Think Harder or Know More?
  8. 2026One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models