mlsanityData to 7 Oct 2026

Graphical einops: bridging tensor networks and computation graphs

arXiv 2605.31485 ↗First posted 29 May 2026cs.LG math.CTcited by 0

Not on the map. Why?

Abstract (arXiv)

Architecture diagrams are ubiquitous in deep learning, but they are usually only representational: the tensor-program identities they suggest are still proved by prose and tensor-axis manipulation. We introduce a formal graphical calculus for the structural fragment of tensor programming underlying einops, making such diagrams proof-enabling. Our calculus represents tensor axes as nested graded tubes around a base type. The tube boundary recovers the undirected tensor-network view of axes, while the directed interior retains the operational reading of computation graphs. The key rewrite is grade-naturality: sliding spectacles over tubes. Standard equivariance proofs become short diagrammatic derivations. We additionally demonstrate how our rewrite system may be applied to convert attention masks into pre-processing operations, recovering efficient implementations of sparse attention blocks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 34 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

17 of 34 references.

References · 17

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  7. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  9. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  10. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  11. 2019Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernelcited by 211
  12. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 148
5 more
  1. 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 71
  2. 2024FlashMask: Efficient and Rich Mask Extension of FlashAttentioncited by 18
  3. 2024Neural Circuit Diagrams: Robust Diagrams for the Communication, Implementation, and Analysis of Deep Learning Architecturescited by 8
  4. 2024An introduction to graphical tensor notation for mechanistic interpretabilitycited by 7
  5. 2025Tape Diagrams for Monoidal Monadscited by 1

Built on by

Not cited yet.