mlsanityData to 7 Oct 2026

The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks

arXiv 2306.17844 ↗First posted 30 Jun 2023cs.LGcited by 153

Not on the map. Why?

Abstract (arXiv)

Do neural networks, trained on well-understood algorithmic tasks, reliably rediscover known algorithms for solving those tasks? Several recent studies, on tasks ranging from group arithmetic to in-context linear regression, have suggested that the answer is yes. Using modular addition as a prototypical problem, we show that algorithm discovery in neural networks is sometimes more complex. Small changes to model hyperparameters and initializations can induce the discovery of qualitatively different algorithms from a fixed training set, and even parallel implementations of multiple such algorithms. Some networks trained to perform modular addition implement a familiar Clock algorithm; others implement a previously undescribed, less intuitive, but comprehensible procedure which we term the Pizza algorithm, or a variety of even more complex procedures. Our results show that even simple learning problems can admit a surprising diversity of solutions, motivating the development of new tools for characterizing the behavior of neural networks across their algorithmic phase space.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). Builds on 23 of its 29 references that are in the map. First posted on arXiv 30 Jun 2023.

Builds on

23 of 29 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Emergent Abilities of Large Language Modelscited by 2,483
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  4. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  5. 2022In-context Learning and Induction Headscited by 930
  6. 2022Toy Models of Superpositioncited by 928
  7. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  8. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  9. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  10. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
  11. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  12. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
11 more
  1. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 229
  2. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 184
  3. 2022Omnigrok: Grokking Beyond Algorithmic Datacited by 142
  4. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 141
  5. 2023The Quantization Model of Neural Scalingcited by 137
  6. 2019Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discoverycited by 93
  7. 2022Mechanistic Mode Connectivitycited by 50
  8. 2020Learning to Unknotcited by 40
  9. 2021Machine-Learning Mathematical Structurescited by 25
  10. 2022How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformerscited by 20
  11. 2015Visual Learning of Arithmetic Operationscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 55; 2025: 49; 2026: 34.

153 papers.

Sort:

By area:

  1. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  3. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  4. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
  5. 2023What Algorithms can Transformers Learn? A Study in Length Generalizationcited by 198
  6. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 123
  7. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  8. 2024KAN 2.0: Kolmogorov-Arnold Networks Meet Sciencecited by 116
  9. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  10. 2024Universal Neurons in GPT2 Language Modelscited by 100
  11. 2024Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristicscited by 94
  12. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
38 more
  1. 2024A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Taskcited by 49
  2. 2025The Origins of Representation Manifolds in Large Language Modelscited by 46
  3. 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 43
  4. 2023Learning the greatest common divisor: explaining transformer predictionscited by 33
  5. 2023Feature emergence via margin maximization: case studies in algebraic taskscited by 33
  6. 2024Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysiscited by 31
  7. 2025(How) Do Language Models Track State?cited by 31
  8. 2023Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammarscited by 30
  9. 2024A phase transition between positional and semantic learning in a solvable model of dot-product attentioncited by 30
  10. 2024Standards for Belief Representations in LLMscited by 30
  11. 2024Hypothesis Testing the Circuit Hypothesis in LLMscited by 29
  12. 2024Out-of-distribution generalization via composition: a lens through induction heads in Transformerscited by 28
  13. 2024Emergence in non-neural models: grokking modular arithmetic via average gradient outer productcited by 26
  14. 2024Language Models Need Inductive Biases to Count Inductivelycited by 25
  15. 2024Mechanistic?cited by 25
  16. 2023To grok or not to grok: Disentangling generalization and memorization on corrupted algorithmic datasetscited by 24
  17. 2023Interpretability Illusions in the Generalization of Simplified Modelscited by 24
  18. 2024InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniquescited by 24
  19. 2024Language Models as Models of Languagecited by 24
  20. 2023Grokking Group Multiplication with Cosetscited by 22
  21. 2024Towards Uncovering How Large Language Model Works: An Explainability Perspectivecited by 22
  22. 2025Mixture of Experts Made Intrinsically Interpretablecited by 22
  23. 2025The Geometry of Reasoning: Flowing Logics in Representation Spacecited by 21
  24. 2024Opening the AI black box: program synthesis via mechanistic interpretabilitycited by 19
  25. 2024A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcutscited by 19
  26. 2024Grokking Modular Polynomialscited by 18
  27. 2024How transformers learn structured data: insights from hierarchical filteringcited by 18
  28. 2024Algorithmic Capabilities of Random Transformerscited by 17
  29. 2025Probing for Arithmetic Errors in Language Modelscited by 17
  30. 2025Aligned but Blind: Alignment Increases Implicit Bias by Reducing Awareness of Racecited by 15
  31. 2025Hidden Breakthroughs in Language Model Trainingcited by 15
  32. 2024Towards Empirical Interpretation of Internal Circuits and Properties in Grokked Transformers on Modular Polynomialscited by 14
  33. 2024Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformerscited by 14
  34. 2023In-Context Learning Dynamics with Random Binary Sequencescited by 13
  35. 2024Does ChatGPT Have a Mind?cited by 13
  36. 2024Position: An Inner Interpretability Framework for AI Inspired by Lessons from Cognitive Neurosciencecited by 11
  37. 2024Emergent properties with repeated examplescited by 11
  38. 2025Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networkscited by 11

50 most cited of 153 All 153 (JSON)