# Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs

arXiv: https://arxiv.org/abs/2602.01914 · HTML: https://mlsanity.com/paper/2602.01914/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 11 of its 28 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 11 of 11)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [Measuring the Mixing of Contextual Information in the Transformer](https://mlsanity.com/paper/2203.04212/) (2022)
- [AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution](https://mlsanity.com/paper/2411.15102/) (2024)
- [ReAGent: A Model-agnostic Feature Attribution Method for Generative Language Models](https://mlsanity.com/paper/2402.00794/) (2024)
- [Explaining the Reasoning of Large Language Models Using Attribution Graphs](https://mlsanity.com/paper/2512.15663/) (2025)

## Built on by (most cited first, 7 of 7)

- [Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference](https://mlsanity.com/paper/2606.05922/) (2026)
- [AttriMem: Attribution-Guided Process Feedback for Agent Memory Construction](https://mlsanity.com/paper/2607.21106/) (2026)
- [Directional Reasoning Trajectory Change (DRTC): Identifying Critical Trace Segments in Reasoning Models](https://mlsanity.com/paper/2602.15332/) (2026)
- [TRAM: Enhancing Multimodal Reasoning with Trajectory-Derived Auxiliary Memory](https://mlsanity.com/paper/2608.01922/) (2026)
- [Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning](https://mlsanity.com/paper/2609.04194/) (2026)
- [Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning](https://mlsanity.com/paper/2609.37656/) (2026)
- [From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representations](https://mlsanity.com/paper/2610.15412/) (2026)

## Abstract (arXiv)

Token attribution methods provide intuitive explanations for language model outputs by identifying causally important input tokens. However, as modern LLMs increasingly rely on extended reasoning chains, existing schemes face two critical challenges: (1) efficiency bottleneck, where attributing a target span of M tokens within a context of length N requires O(M*N) operations, making long-context attribution prohibitively slow; and (2) faithfulness drop, where intermediate reasoning tokens absorb attribution mass, preventing importance from propagating back to the original input. To address these, we introduce FlashTrace, an efficient multi-token attribution method that employs span-wise aggregation to compute attribution over multi-token targets in a single pass, while maintaining faithfulness. Moreover, we design a recursive attribution mechanism that traces importance through intermediate reasoning chains back to source inputs. Extensive experiments on long-context retrieval (RULER) and multi-step reasoning (MATH, MorehopQA) tasks demonstrate that FlashTrace achieves over 130x speedup over existing baselines while maintaining superior faithfulness. We further analyze the dynamics of recursive attribution, showing that even a single recursive hop improves faithfulness by tracing importance through the reasoning chain.