# Exploiting heterogeneous delays for efficient computation in low-bit neural networks

arXiv: https://arxiv.org/abs/2510.27434 · HTML: https://mlsanity.com/paper/2510.27434/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 64 references that are in the map.
- First posted on arXiv 31 Oct 2025.

## Builds on (most cited first, 23 of 23)

- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Trained Ternary Quantization](https://mlsanity.com/paper/1612.01064/) (2016)
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](https://mlsanity.com/paper/1910.07407/) (2019)
- [Pathways: Asynchronous Distributed Dataflow for ML](https://mlsanity.com/paper/2203.12533/) (2022)
- [Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings](https://mlsanity.com/paper/2306.17670/) (2023)
- [The neuroconnectionist research programme](https://mlsanity.com/paper/2209.03718/) (2022)
- [A Comprehensive Study on Quantization Techniques for Large Language Models](https://mlsanity.com/paper/2411.02530/) (2024)
- [DenRAM: Neuromorphic Dendritic Architecture with RRAM for Efficient Temporal Processing with Delays](https://mlsanity.com/paper/2312.08960/) (2023)
- [Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks](https://mlsanity.com/paper/2212.01232/) (2022)
- [Axonal Delay As a Short-Term Memory for Feed Forward Deep Spiking Neural Networks](https://mlsanity.com/paper/2205.02115/) (2022)
- [A Robot Web for Distributed Many-Device Localisation](https://mlsanity.com/paper/2202.03314/) (2022)
- [A Spatiotemporal Perspective on Dynamical Computation in Neural Information Processing Systems](https://mlsanity.com/paper/2409.13669/) (2024)
- [DelRec: learning delays in recurrent spiking neural networks](https://mlsanity.com/paper/2509.24852/) (2025)
- [Beyond Weights: Deep learning in Spiking Neural Networks with pure synaptic-delay training](https://mlsanity.com/paper/2306.06237/) (2023)
- [Delayed Memory Unit: Modelling Temporal Dependency Through Delay Gate](https://mlsanity.com/paper/2310.14982/) (2023)
- [Memory via Temporal Delays in weightless Spiking Neural Network](https://mlsanity.com/paper/2202.07132/) (2022)
- [Spikemax: Spike-based Loss Methods for Classification](https://mlsanity.com/paper/2205.09845/) (2022)
- [The specificity and robustness of long-distance connections in weighted, interareal connectomes](https://mlsanity.com/paper/1711.03809/) (2017)
- [PETAH: Parameter Efficient Task Adaptation for Hybrid Transformers in a resource-limited Context](https://mlsanity.com/paper/2410.17661/) (2024)
- [Efficient Synaptic Delay Implementation in Digital Event-Driven AI Accelerators](https://mlsanity.com/paper/2501.13610/) (2025)

## Built on by (most cited first, 2 of 2)

- [Three factor delay learning rules for spiking neural networks](https://mlsanity.com/paper/2601.00668/) (2026)
- [Delays in Spiking Neural Networks: A State Space Model Approach](https://mlsanity.com/paper/2512.01906/) (2025)

## Abstract (arXiv)

Neural networks rely on learning synaptic weights. However, this overlooks other neural parameters that can also be learned and may be utilized by the brain. One such parameter is the delay: the brain exhibits complex temporal dynamics with heterogeneous delays, where signals are transmitted asynchronously between neurons. It has been theorized that this delay heterogeneity, rather than a cost to be minimized, can be exploited in embodied contexts where task-relevant information naturally sits contextually in the time domain. We test this hypothesis by training spiking neural networks to modify not only their weights but also their delays at different levels of precision. We find that delay heterogeneity enables state-of-the-art performance on temporally complex neuromorphic problems and can be achieved even when weights are extremely imprecise (1.58-bit ternary precision: just positive, negative, or absent). By enabling high performance with extremely low-precision weights, delay heterogeneity allows memory-efficient solutions that maintain state-of-the-art accuracy even when weights are compressed over an order of magnitude more aggressively than typically studied weight-only networks. We show how delays and time-constants adaptively trade-off, and reveal through ablation that task performance depends on task-appropriate delay distributions, with temporally-complex tasks requiring longer delays. Our results suggest temporal heterogeneity is an important principle for efficient computation, particularly when task-relevant information is temporal - as in the physical world - with implications for embodied intelligent systems and neuromorphic hardware.