# Step-resolved data attribution for looped transformers

arXiv: https://arxiv.org/abs/2602.10097 · HTML: https://mlsanity.com/paper/2602.10097/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 42 references that are in the map.
- First posted on arXiv 10 Feb 2026.

## Builds on (most cited first, 30 of 32)

- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://mlsanity.com/paper/2502.05171/) (2025)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Reasoning with Latent Thoughts: On the Power of Looped Transformers](https://mlsanity.com/paper/2502.17416/) (2025)
- [Looped Transformers as Programmable Computers](https://mlsanity.com/paper/2301.13196/) (2023)
- [SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver](https://mlsanity.com/paper/1905.12149/) (2019)
- [Scaling Latent Reasoning via Looped Language Models](https://mlsanity.com/paper/2510.25741/) (2025)
- [Looped Transformers are Better at Learning Learning Algorithms](https://mlsanity.com/paper/2311.12424/) (2023)
- [Deep Learning Through the Lens of Example Difficulty](https://mlsanity.com/paper/2106.09647/) (2021)
- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation](https://mlsanity.com/paper/2507.10524/) (2025)
- [Thought Anchors: Which LLM Reasoning Steps Matter?](https://mlsanity.com/paper/2506.19143/) (2025)
- [Looped Transformers for Length Generalization](https://mlsanity.com/paper/2409.15647/) (2024)
- [A Survey on Latent Reasoning](https://mlsanity.com/paper/2507.06203/) (2025)
- [Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning](https://mlsanity.com/paper/2505.16782/) (2025)
- [TimeSHAP: Explaining Recurrent Models through Sequence Perturbations](https://mlsanity.com/paper/2012.00073/) (2020)
- [RecurrentGemma: Moving Past Transformers for Efficient Open Language Models](https://mlsanity.com/paper/2404.07839/) (2024)
- [Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence](https://mlsanity.com/paper/2511.07384/) (2025)
- [Learning to Solve Constraint Satisfaction Problems with Recurrent Transformer](https://mlsanity.com/paper/2307.04895/) (2023)
- [Block-Recurrent Dynamics in Vision Transformers](https://mlsanity.com/paper/2512.19941/) (2025)
- [Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functions](https://mlsanity.com/paper/2006.13707/) (2020)
- [On Attribution of Recurrent Neural Network Predictions via Additive Decomposition](https://mlsanity.com/paper/1903.11245/) (2019)
- [Universal Reasoning Model](https://mlsanity.com/paper/2512.14693/) (2025)
- [Influence Functions for Sequence Tagging Models](https://mlsanity.com/paper/2210.14177/) (2022)
- [Revisit, Extend, and Enhance Hessian-Free Influence Functions](https://mlsanity.com/paper/2405.17490/) (2024)

## Built on by (most cited first, 5 of 5)

- [LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation](https://mlsanity.com/paper/2602.11451/) (2026)
- [When Does Recurrence Become an Algorithm? Convergence Selection in Weight-Tied Looped Transformers](https://mlsanity.com/paper/2607.20594/) (2026)
- [T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning with Dynamic Routing](https://mlsanity.com/paper/2609.15160/) (2026)
- [SMolLM: Small Language Models Learn Small Molecular Grammar](https://mlsanity.com/paper/2605.06322/) (2026)
- [Loop Dropout: Regularizing Shared Updates in Looped Language Models](https://mlsanity.com/paper/2609.34218/) (2026)

## Abstract (arXiv)

We study how individual training examples shape the internal computation of looped transformers, where a shared block is applied for τ recurrent iterations to enable latent reasoning. Existing training-data influence estimators such as TracIn yield a single scalar score that aggregates over all loop iterations, obscuring when during the recurrent computation a training example matters. We introduce textitStep-Decomposed Influence (SDI), which decomposes TracIn into a length-τ influence trajectory by unrolling the recurrent computation graph and attributing influence to specific loop iterations. To make SDI practical at transformer scale, we propose a TensorSketch implementation that never materialises per-example gradients. Experiments on looped GPT-style models and algorithmic reasoning tasks show that SDI scales excellently, matches full-gradient baselines with low error and supports a broad range of data attribution and interpretability tasks with per-step insights into the latent reasoning process.