# Artificial Neural Networks generated by Low Discrepancy Sequences

arXiv: https://arxiv.org/abs/2103.03543 · HTML: https://mlsanity.com/paper/2103.03543/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 5 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [Sparse Networks from Scratch: Faster Training without Losing Performance](https://mlsanity.com/paper/1907.04840/) (2019)
- [Trained Ternary Quantization](https://mlsanity.com/paper/1612.01064/) (2016)
- [Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask](https://mlsanity.com/paper/1905.01067/) (2019)
- [QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions](https://mlsanity.com/paper/1910.10261/) (2019)
- [Pre-defined Sparsity for Low-Complexity Convolutional Neural Networks](https://mlsanity.com/paper/2001.10710/) (2020)
- [Pre-Defined Sparse Neural Networks with Hardware Acceleration](https://mlsanity.com/paper/1812.01164/) (2018)
- [Instant Quantization of Neural Networks using Monte Carlo Methods](https://mlsanity.com/paper/1905.12253/) (2019)
- [A Pre-defined Sparse Kernel Based Convolution for Deep CNNs](https://mlsanity.com/paper/1910.00724/) (2019)
- [Characterizing Sparse Connectivity Patterns in Neural Networks](https://mlsanity.com/paper/1711.02131/) (2017)
- [Accelerating Training of Deep Neural Networks via Sparse Edge Processing](https://mlsanity.com/paper/1711.01343/) (2017)
- [Interleaver Design for Deep Neural Networks](https://mlsanity.com/paper/1711.06935/) (2017)

## Built on by (most cited first, 1 of 1)

- [Lattice-based Deep Neural Networks: Regularity and Tailored Regularization](https://mlsanity.com/paper/2603.02809/) (2026)

## Abstract (arXiv)

Artificial neural networks can be represented by paths. Generated as random walks on a dense network graph, we find that the resulting sparse networks allow for deterministic initialization and even weights with fixed sign. Such networks can be trained sparse from scratch, avoiding the expensive procedure of training a dense network and compressing it afterwards. Although sparse, weights are accessed as contiguous blocks of memory. In addition, enumerating the paths using deterministic low discrepancy sequences, for example the Sobol' sequence, amounts to connecting the layers of neural units by progressive permutations, which naturally avoids bank conflicts in parallel computer hardware. We demonstrate that the artificial neural networks generated by low discrepancy sequences can achieve an accuracy within reach of their dense counterparts at a much lower computational complexity.