mlsanityData to 14 Oct 2026

TokenScope: Token-Level Explainability and Interpretability for Code-Oriented Tasks in Large Language Models

arXiv 2607.01235 ↗First posted 30 Apr 2026cs.CL cs.AI cs.SEcited by 0

On the map: Mechanistic interpretability → Knowledge and representations inside LLMs

Abstract (arXiv)

Understanding how Large Language Models (LLMs) make token-level decisions during code generation remains a major challenge for both researchers and practitioners. While recent tools provide insights into model internals or generation outcomes, they often lack decoding-time signals, fine-grained uncertainty measures, and interactive mechanisms for exploring alternative generation paths. We present TokenScope, an interactive interpretability and analysis tool for decoder-based LLMs that exposes token-level metrics, attention patterns, and structural information during generation. TokenScope supports interactive token replacement, counterfactual branching, and code-aware aggregation via abstract syntax trees. By unifying decoding-time signals with structural program analysis, TokenScope enables systematic investigation of LLM behaviour during code generation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability. Builds on 5 of its 7 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

5 of 7 references.

References · 5

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  3. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  4. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  5. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Enabling Global, Human-Centered Explanations for LLMs:From Tokens to Interpretable Code and Test Generation
  2. 2026Compressed code: the hidden effects of quantization and distillation on programming tokens
  3. 2025EquiBench: Benchmarking Large Language Models' Reasoning about Program Semantics via Equivalence Checking
  4. 2024Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations
  5. 2025CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models
  6. 2026Evaluating Language Models on Cross-Language Code Functional Equivalence
  7. 2025A Code Comprehension Benchmark for Large Language Models for Code
  8. 2025A Tool for In-depth Analysis of Code Execution Reasoning of Large Language Models