# Theoretical Properties for Neural Networks with Weight Matrices of Low Displacement Rank

arXiv: https://arxiv.org/abs/1703.00144 · HTML: https://mlsanity.com/paper/1703.00144/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Mar 2017.

## Built on by (most cited first, 11 of 11)

- [The Hessian perspective into the Nature of Convolutional Neural Networks](https://mlsanity.com/paper/2305.09088/) (2023)
- [A Secure and Efficient Federated Learning Framework for NLP](https://mlsanity.com/paper/2201.11934/) (2022)
- [Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning](https://mlsanity.com/paper/2406.17740/) (2024)
- [Ortho-Shot: Low Displacement Rank Regularization with Data Augmentation for Few-Shot Learning](https://mlsanity.com/paper/2110.09374/) (2021)
- [Block Circulant Adapter for Large Language Models](https://mlsanity.com/paper/2505.00582/) (2025)
- [Hardware-Efficient Photonic Tensor Core: Accelerating Deep Neural Networks with Structured Compression](https://mlsanity.com/paper/2502.01670/) (2025)
- [Arithmetic Circuits, Structured Matrices and (not so) Deep Learning](https://mlsanity.com/paper/2206.12490/) (2022)
- [Neural-HSS: Hierarchical Semi-Separable Neural PDE Solver](https://mlsanity.com/paper/2602.18248/) (2026)
- [A Low-complexity Structured Neural Network Approach to Intelligently Realize Wideband Multi-beam Beamformers](https://mlsanity.com/paper/2503.20694/) (2025)
- [EdgeLDR: Quaternion Low-Displacement Rank Neural Networks for Edge-Efficient Deep Learning](https://mlsanity.com/paper/2601.05379/) (2026)
- [Compact Circulant Layers with Spectral Priors](https://mlsanity.com/paper/2602.21965/) (2026)

## Abstract (arXiv)

Recently low displacement rank (LDR) matrices, or so-called structured matrices, have been proposed to compress large-scale neural networks. Empirical results have shown that neural networks with weight matrices of LDR matrices, referred as LDR neural networks, can achieve significant reduction in space and computational complexity while retaining high accuracy. We formally study LDR matrices in deep learning. First, we prove the universal approximation property of LDR neural networks with a mild condition on the displacement operators. We then show that the error bounds of LDR neural networks are as efficient as general neural networks with both single-layer and multiple-layer structure. Finally, we propose back-propagation based training algorithm for general LDR neural networks.