mlsanityData to 7 Oct 2026

Compositionality Unlocks Deep Interpretable Models

arXiv 2504.02667 ↗First posted 3 Apr 2025cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We propose chi-net, an intrinsically interpretable architecture combining the compositional multilinear structure of tensor networks with the expressivity and efficiency of deep neural networks. chi-nets retain equal accuracy compared to their baseline counterparts. Our novel, efficient diagonalisation algorithm, ODT, reveals linear low-rank structure in a multilayer SVHN model. We leverage this toward formal weight-based interpretability and model compression.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 11 of its 19 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

11 of 19 references.

References · 11

  1. 2020GLU Variants Improve Transformercited by 1,724
  2. 2016Why does deep and cheap learning work so well?cited by 100
  3. 2021QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computercited by 65
  4. 2020Deep Polynomial Neural Networkscited by 63
  5. 2020Towards falsifiable interpretability researchcited by 53
  6. 2018Quantum Entanglement in Deep Learning Architecturescited by 49
  7. 2024Bilinear MLPs enable weight-based mechanistic interpretabilitycited by 28
  8. 2024What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?cited by 22
  9. 2023A technical note on bilinear layers for interpretabilitycited by 12
  10. 2021Compositionality as we see it, everywhere around uscited by 11
  11. 2018Learning with tree-based tensor formatscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoderscited by 5
  2. 2025Finding Manifolds With Bilinear Autoencoderscited by 5
  3. 2026PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decodingcited by 3
  4. 2026Bilinear autoencoders find interpretable manifoldscited by 1
  5. 2026From Mechanistic to Compositional Interpretabilitycited by 1
  6. 2026Benign Loss Landscapes Can Coexist with Worst-Case Hardnesscited by 0