# Neural network is heterogeneous: Phase matters more

arXiv: https://arxiv.org/abs/2111.02014 · HTML: https://mlsanity.com/paper/2111.02014/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 14 references that are in the map.
- First posted on arXiv 3 Nov 2021.

## Builds on (most cited first, 6 of 6)

- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [Wave Physics as an Analog Recurrent Neural Network](https://mlsanity.com/paper/1904.12831/) (2019)
- [Complex-valued Neural Networks with Non-parametric Activation Functions](https://mlsanity.com/paper/1802.08026/) (2018)
- [Evaluation of Complex-Valued Neural Networks on Real-Valued Classification Tasks](https://mlsanity.com/paper/1811.12351/) (2018)

## Built on by (most cited first, 3 of 3)

- [A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges](https://mlsanity.com/paper/2406.11903/) (2024)
- [Fully complex-valued deep learning model for visual perception](https://mlsanity.com/paper/2212.07146/) (2022)
- [Phase-Aware Wavelet-Based-Scattering Encoder-Decoder for Dense Predictions](https://mlsanity.com/paper/2605.24621/) (2026)

## Abstract (arXiv)

We find a heterogeneity in both complex and real valued neural networks with the insight from wave optics, claiming a much more important role of phase than its amplitude counterpart in the weight matrix. In complex-valued neural networks, we show that among different types of pruning, the weight matrix with only phase information preserved achieves the best accuracy, which holds robustly under various settings of depth and width. The conclusion can be generalized to real-valued neural networks, where signs take the place of phases. These inspiring findings enrich the techniques of network pruning and binary computation.