# Realizing data features by deep nets

arXiv: https://arxiv.org/abs/1901.00130 · HTML: https://mlsanity.com/paper/1901.00130/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jan 2019.

## Built on by (most cited first, 6 of 6)

- [Universal Consistency of Deep Convolutional Neural Networks](https://mlsanity.com/paper/2106.12498/) (2021)
- [Generalization Performance of Empirical Risk Minimization on Over-parameterized Deep ReLU Nets](https://mlsanity.com/paper/2111.14039/) (2021)
- [Optimal Approximation and Learning Rates for Deep Convolutional Neural Networks](https://mlsanity.com/paper/2308.03259/) (2023)
- [Component-based Sketching for Deep ReLU Nets](https://mlsanity.com/paper/2409.14174/) (2024)
- [Lifting the Veil: Unlocking the Power of Depth in Q-learning](https://mlsanity.com/paper/2310.17915/) (2023)
- [Feature Qualification by Deep Nets: A Constructive Approach](https://mlsanity.com/paper/2503.18676/) (2025)

## Abstract (arXiv)

This paper considers the power of deep neural networks (deep nets for short) in realizing data features. Based on refined covering number estimates, we find that, to realize some complex data features, deep nets can improve the performances of shallow neural networks (shallow nets for short) without requiring additional capacity costs. This verifies the advantage of deep nets in realizing complex features. On the other hand, to realize some simple data feature like the smoothness, we prove that, up to a logarithmic factor, the approximation rate of deep nets is asymptotically identical to that of shallow nets, provided that the depth is fixed. This exhibits a limitation of deep nets in realizing simple features.