# Scaling Down Deep Learning with MNIST-1D

arXiv: https://arxiv.org/abs/2011.14439 · HTML: https://mlsanity.com/paper/2011.14439/

## Key facts

- Cited by 23 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 29 Nov 2020.

## Built on by (most cited first, 23 of 23)

- [Better than classical? The subtle art of benchmarking quantum machine learning models](https://mlsanity.com/paper/2403.07059/) (2024)
- [Some Fundamental Aspects about Lipschitz Continuity of Neural Networks](https://mlsanity.com/paper/2302.10886/) (2023)
- [Phenomenology of Double Descent in Finite-Width Neural Networks](https://mlsanity.com/paper/2203.07337/) (2022)
- [Benchmarking quantum machine learning kernel training for classification tasks](https://mlsanity.com/paper/2408.10274/) (2024)
- [Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?](https://mlsanity.com/paper/2408.11979/) (2024)
- [Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond](https://mlsanity.com/paper/2411.00247/) (2024)
- [Sparsity-Probe: Analysis tool for Deep Learning Models](https://mlsanity.com/paper/2105.06849/) (2021)
- [Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement](https://mlsanity.com/paper/2410.02147/) (2024)
- [Not All Explanations for Deep Learning Phenomena Are Equally Valuable](https://mlsanity.com/paper/2506.23286/) (2025)
- [Structured Progressive Knowledge Activation for LLM-Driven Neural Architecture Search](https://mlsanity.com/paper/2605.04057/) (2026)
- [Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Tasks](https://mlsanity.com/paper/2607.17624/) (2026)
- [GraphIR: Architecture-Level Search States for LLM-Guided Neural Architecture Evolution](https://mlsanity.com/paper/2608.01633/) (2026)
- [RanSOM: Second-Order Momentum with Randomized Scaling for Constrained and Unconstrained Optimization](https://mlsanity.com/paper/2602.06824/) (2026)
- [Neko: a Library for Exploring Neuromorphic Learning Rules](https://mlsanity.com/paper/2105.00324/) (2021)
- [Neuronal diversity can improve machine learning for physics and beyond](https://mlsanity.com/paper/2204.04348/) (2022)
- [Do We Need Another Explainable AI Method? Toward Unifying Post-hoc XAI Evaluation Methods into an Interactive and Multi-dimensional Benchmark](https://mlsanity.com/paper/2207.14160/) (2022)
- [Harnessing uncertainty when learning through Equilibrium Propagation in neural networks](https://mlsanity.com/paper/2503.22810/) (2025)
- [Towards Scalable Backpropagation-Free Gradient Estimation](https://mlsanity.com/paper/2511.03110/) (2025)
- [Early Classification of Time Series in Non-Stationary Cost Regimes](https://mlsanity.com/paper/2602.00918/) (2026)
- [Training deep physical neural networks with local physical information bottleneck](https://mlsanity.com/paper/2602.09569/) (2026)
- [Exploring the Performance of ML/DL Architectures on the MNIST-1D Dataset](https://mlsanity.com/paper/2602.13348/) (2026)
- [Cautious optimism for deep parameterized quantum circuits](https://mlsanity.com/paper/2607.21409/) (2026)
- [End-to-end Early Classification of Time Series in Non-Stationary Environments](https://mlsanity.com/paper/2608.20044/) (2026)

## Abstract (arXiv)

Although deep learning models have taken on commercial and political relevance, key aspects of their training and operation remain poorly understood. This has sparked interest in science of deep learning projects, many of which require large amounts of time, money, and electricity. But how much of this research really needs to occur at scale? In this paper, we introduce MNIST-1D: a minimalist, procedurally generated, low-memory, and low-compute alternative to classic deep learning benchmarks. Although the dimensionality of MNIST-1D is only 40 and its default training set size only 4000, MNIST-1D can be used to study inductive biases of different deep architectures, find lottery tickets, observe deep double descent, metalearn an activation function, and demonstrate guillotine regularization in self-supervised learning. All these experiments can be conducted on a GPU or often even on a CPU within minutes, allowing for fast prototyping, educational use cases, and cutting-edge research on a low budget.