mlsanityData to 14 Oct 2026

CircuitTree: Composing Circuits into A Syllable Transcriber

arXiv 2610.12520 ↗First posted 7 Oct 2026cs.LG cs.AI cs.SDcited by 0

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

The goal of circuit discovery is to decompose a monolithic neural network into tiny modular subnetworks which are interpretable in the sense that each does an identifiable subtask. We develop a practical method for composing arbitrarily many circuits into a single model. We propose a greedy, density-scaled composition algorithm that overcomes limitations of previous attempts to compose circuits by naive union, then generalize it to N circuits via CircuitTree, a new data structure built on a Theta(Nlog N) tree-compose schedule. We compose circuits pruned from HuBERT for classifying individual phonemes into a syllable classifier, a capability none of the 39 phoneme circuits has individually. Chaining CircuitTrees into a hierarchy-of-circuits pipeline, we transcribe open syllables at 64.8% accuracy, within 1.2 points of a 66.0% ceiling that shares no circuitry at all, with 99.5% of circuit components shared.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 15 of its 18 references that are in the map. First posted on arXiv 7 Oct 2026.

Builds on

15 of 18 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs12 May
  2. 2025K-MSHC: Unmasking Minimally Sufficient Head Circuits in Large Language Models with Experiments on Syntactic Classification Tasks18 May
  3. 2023Circuit Component Reuse Across Tasks in Transformer Language Models12 Oct
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretability28 Apr
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small1 Nov

Other references · 10

  1. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  2. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  3. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
  4. 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 403
  5. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 104
  6. 2024Finding Transformer Circuits with Edge Pruningcited by 60
  7. 2024Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularitycited by 14
  8. 2024Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Modelscited by 11
  9. 2021Neural Network Module Decomposition and Recompositioncited by 10
  10. 2026Sparse Weight and Edge Circuit Discovery in Transformer-based Acoustic Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CircuitFormer: A Circuit Language Model for Analog Topology Design from Natural Language Prompt
  2. 2026Data-driven Circuit Discovery for Interpretability of Language Models
  3. 2026Hierarchical Sparse Circuit Extraction from Billion-Parameter Language Models through Scalable Attribution Graph Decomposition
  4. 2024Transcoders Find Interpretable LLM Feature Circuits
  5. 2025Position-aware Automatic Circuit Discovery
  6. 2026Scalable Circuit Learning for Interpreting Large Language Models
  7. 2023Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models
  8. 2026CSP-Atlas: Concept-Specific Neural Circuits in a Sparse Python Transformer