mlsanityData to 7 Oct 2026

Dendritic In-Context Learning in a Single-Layer Spiking Neural Network

arXiv 2607.02283 ↗First posted 2 Jul 2026cs.NE cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL) operates via implicit gradient descent embedded in the forward pass of modern AI architectures -- Transformers, Mamba, state-space models, and MLPs. Capturing this capability in biologically plausible Spiking Neural Networks (SNNs) has remained an open challenge: existing SNNs fail the Garg-2022 benchmark at non-trivial task dimensions. We trace this failure to a structural assumption: prior SNN designs route adaptation through inference-time synaptic plasticity, viewing the dendritic compartment as a passive conduit for error or teacher signals. We challenge this assumption. The subthreshold dynamics of a single dendritic compartment already implement a complete online learning algorithm. By treating the compartment as the computational substrate rather than a passive conduit, we propose DendriCL -- a single-layer compartmental spiking architecture whose apical recurrence is structurally identical to leaky online Widrow-Hoff LMS. This dynamics-only update collapses the architectural depth required for general-purpose ICL to a single layer. DendriCL is uniquely seed-stable at super-dimensional Garg-2022 ICL -- where dense Transformers exhibit grokking-style instability and fail past moderate task dimension -- and a linear probe recovers the reference online-LMS trajectory directly from the apical membrane at R^2 = 0.93, showing the algorithm is structurally embedded in the dynamics rather than implicitly discovered during training. Taken together, ICL requires neither attention, depth, nor inference-time plasticity: a single compartment with online-LMS dynamics is sufficient.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 50 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

27 of 50 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  5. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  6. 2022In-context Learning and Induction Headscited by 930
  7. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  8. 2022Transformers learn in-context by gradient descentcited by 714
  9. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  10. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  11. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  12. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
15 more
  1. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  2. 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 274
  3. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
  4. 2018Long short-term memory and learning-to-learn in networks of spiking neuronscited by 179
  5. 2020Liquid Time-constant Networkscited by 160
  6. 2024The Evolution of Statistical Induction Heads: In-Context Learning Markov Chainscited by 115
  7. 2024Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Taskscited by 104
  8. 2024Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chipscited by 97
  9. 2018Dendritic cortical microcircuits approximate the backpropagation algorithmcited by 96
  10. 2018Differentiable plasticity: training plastic neural networks with backpropagationcited by 83
  11. 2021Closed-form Continuous-time Neural Modelscited by 65
  12. 2024Is Mamba Capable of In-Context Learning?cited by 45
  13. 2024Linear Transformers are Versatile In-Context Learnerscited by 28
  14. 2024SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Modelscited by 19
  15. 2024MLPs Learn In-Context on Regression and Classification Taskscited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Spiking World Model with Multi-Compartment Neurons for Model-based Reinforcement Learning
  2. 2026SpikingVLA: Asynchronous Spiking Vision-Language-Action Models
  3. 2025Spiking Decision Transformers: Local Plasticity, Phase-Coding, and Dendritic Routing for Low-Power Sequence Control
  4. 2025Biologically Inspired Spiking Diffusion Model with Adaptive Lateral Selection Mechanism
  5. 2024P-SpikeSSM: Harnessing Probabilistic Spiking State Space Models for Long-Range Dependency Tasks
  6. 2023Fully Spiking Denoising Diffusion Implicit Models
  7. 2023Spiking Denoising Diffusion Probabilistic Models
  8. 2026The Forward-Backward Disconnect: State Dynamics, Credit Assignment, and Biological Grounding in Neural Computation ↗