mlsanityData to 7 Oct 2026

Scaling Down Deep Learning with MNIST-1D

arXiv 2011.14439 ↗First posted 29 Nov 2020cs.LG cs.NE stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

Although deep learning models have taken on commercial and political relevance, key aspects of their training and operation remain poorly understood. This has sparked interest in science of deep learning projects, many of which require large amounts of time, money, and electricity. But how much of this research really needs to occur at scale? In this paper, we introduce MNIST-1D: a minimalist, procedurally generated, low-memory, and low-compute alternative to classic deep learning benchmarks. Although the dimensionality of MNIST-1D is only 40 and its default training set size only 4000, MNIST-1D can be used to study inductive biases of different deep architectures, find lottery tickets, observe deep double descent, metalearn an activation function, and demonstrate guillotine regularization in self-supervised learning. All these experiments can be conducted on a GPU or often even on a CPU within minutes, allowing for fast prototyping, educational use cases, and cutting-edge research on a low budget.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 3; 2023: 1; 2024: 5; 2025: 3; 2026: 9.

23 papers.

Sort:

  1. 2024Better than classical? The subtle art of benchmarking quantum machine learning modelscited by 82
  2. 2023Some Fundamental Aspects about Lipschitz Continuity of Neural Networkscited by 25
  3. 2022Phenomenology of Double Descent in Finite-Width Neural Networkscited by 11
  4. 2024Benchmarking quantum machine learning kernel training for classification taskscited by 10
  5. 2024Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?cited by 10
  6. 2024Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyondcited by 9
  7. 2021Sparsity-Probe: Analysis tool for Deep Learning Modelscited by 4
  8. 2024Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglementcited by 4
  9. 2025Not All Explanations for Deep Learning Phenomena Are Equally Valuablecited by 3
  10. 2026Structured Progressive Knowledge Activation for LLM-Driven Neural Architecture Searchcited by 2
  11. 2026Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Taskscited by 2
  12. 2026GraphIR: Architecture-Level Search States for LLM-Guided Neural Architecture Evolutioncited by 2
11 more
  1. 2026RanSOM: Second-Order Momentum with Randomized Scaling for Constrained and Unconstrained Optimizationcited by 1
  2. 2021Neko: a Library for Exploring Neuromorphic Learning Rulescited by 0
  3. 2022Neuronal diversity can improve machine learning for physics and beyondcited by 0
  4. 2022Do We Need Another Explainable AI Method? Toward Unifying Post-hoc XAI Evaluation Methods into an Interactive and Multi-dimensional Benchmarkcited by 0
  5. 2025Harnessing uncertainty when learning through Equilibrium Propagation in neural networkscited by 0
  6. 2025Towards Scalable Backpropagation-Free Gradient Estimationcited by 0
  7. 2026Early Classification of Time Series in Non-Stationary Cost Regimescited by 0
  8. 2026Training deep physical neural networks with local physical information bottleneckcited by 0
  9. 2026Exploring the Performance of ML/DL Architectures on the MNIST-1D Datasetcited by 0
  10. 2026Cautious optimism for deep parameterized quantum circuitscited by 0
  11. 2026End-to-end Early Classification of Time Series in Non-Stationary Environmentscited by 0