mlsanityData to 7 Oct 2026

The Foes of Neural Network's Data Efficiency Among Unnecessary Input Dimensions

arXiv 2107.06409 ↗First posted 13 Jul 2021cs.LG cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Datasets often contain input dimensions that are unnecessary to predict the output label, e.g. background in object recognition, which lead to more trainable parameters. Deep Neural Networks (DNNs) are robust to increasing the number of parameters in the hidden layers, but it is unclear whether this holds true for the input layer. In this letter, we investigate the impact of unnecessary input dimensions on a central issue of DNNs: their data efficiency, ie. the amount of examples needed to achieve certain generalization performance. Our results show that unnecessary input dimensions that are task-unrelated substantially degrade data efficiency. This highlights the need for mechanisms that remove {task-unrelated} dimensions to enable data efficiency gains.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 14 of its 25 references that are in the map. First posted on arXiv 13 Jul 2021.

Builds on

14 of 25 references.

References · 14

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  3. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  4. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  5. 2018Recognition in Terra Incognitacited by 655
  6. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  7. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  8. 2017High-dimensional dynamics of generalization error in neural networkscited by 238
  9. 2016Object Recognition with and without Objectscited by 39
  10. 2019Dimensionality compression and expansion in Deep Neural Networkscited by 38
  11. 2015Foveation-based Mechanisms Alleviate Adversarial Examplescited by 32
  12. 2014Object Detection Through Exploration With A Foveated Visual Fieldcited by 14
2 more
  1. 2017Do Deep Neural Networks Suffer from Crowding?cited by 7
  2. 2021Contemplating real-world object classificationcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022Relating Regularization and Generalization through the Intrinsic Dimension of Activationscited by 9