# An Ensemble of Simple Convolutional Neural Network Models for MNIST Digit Recognition

arXiv: https://arxiv.org/abs/2008.10400 · HTML: https://mlsanity.com/paper/2008.10400/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2020.

## Built on by (most cited first, 30 of 30)

- [ExSpliNet: An interpretable and expressive spline-based neural network](https://mlsanity.com/paper/2205.01510/) (2022)
- [WaveMix: A Resource-efficient Neural Network for Image Analysis](https://mlsanity.com/paper/2205.14375/) (2022)
- [An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems](https://mlsanity.com/paper/2205.12755/) (2022)
- [Neuro-Symbolic Learning of Answer Set Programs from Raw Data](https://mlsanity.com/paper/2205.12735/) (2022)
- [Towards out of distribution generalization for problems in mechanics](https://mlsanity.com/paper/2206.14917/) (2022)
- [Spiking Neural Operators for Scientific Machine Learning](https://mlsanity.com/paper/2205.10130/) (2022)
- [Explaining Classifications to Non Experts: An XAI User Study of Post Hoc Explanations for a Classifier When People Lack Expertise](https://mlsanity.com/paper/2212.09342/) (2022)
- [Model-agnostic Measure of Generalization Difficulty](https://mlsanity.com/paper/2305.01034/) (2023)
- [Unsupervised physics-informed disentanglement of multimodal data for high-throughput scientific discovery](https://mlsanity.com/paper/2202.03242/) (2022)
- [EXACT: How to Train Your Accuracy](https://mlsanity.com/paper/2205.09615/) (2022)
- [Recoverable Privacy-Preserving Image Classification through Noise-like Adversarial Examples](https://mlsanity.com/paper/2310.12707/) (2023)
- [A Voting Approach for Explainable Classification with Rule Learning](https://mlsanity.com/paper/2311.07323/) (2023)
- [On Linear Separability of the MNIST Handwritten Digits Dataset](https://mlsanity.com/paper/2603.12850/) (2026)
- [Problem-dependent attention and effort in neural networks with applications to image resolution and model selection](https://mlsanity.com/paper/2201.01415/) (2022)
- [Accelerating Machine Learning via the Weber-Fechner Law](https://mlsanity.com/paper/2204.11834/) (2022)
- [Evaluating the Impact of Loss Function Variation in Deep Learning for Classification](https://mlsanity.com/paper/2210.16003/) (2022)
- [Rule Learning by Modularity](https://mlsanity.com/paper/2212.12335/) (2022)
- [Random Function Descent](https://mlsanity.com/paper/2305.01377/) (2023)
- [Enhancing LLM Evaluations: The Garbling Trick](https://mlsanity.com/paper/2411.01533/) (2024)
- [Hardware-aware vs. Hardware-agnostic Energy Estimation for SNN in Space Applications](https://mlsanity.com/paper/2508.19654/) (2025)
- [Quantity vs Quality: Investigating the Trade-Off between Sample Size and Label Reliability](https://mlsanity.com/paper/2204.09462/) (2022)
- [Maintaining Performance with Less Data](https://mlsanity.com/paper/2208.02007/) (2022)
- [Low-power multi-mode fiber projector overcomes shallow neural networks classifiers](https://mlsanity.com/paper/2210.04745/) (2022)
- [High-Level Parallelism and Nested Features for Dynamic Inference Cost and Top-Down Attention](https://mlsanity.com/paper/2308.05128/) (2023)
- [Neural Networks for Programming Quantum Annealers](https://mlsanity.com/paper/2308.06807/) (2023)
- [Deep Concept Removal](https://mlsanity.com/paper/2310.05755/) (2023)
- [The Implications of Decentralization in Blockchained Federated Learning: Evaluating the Impact of Model Staleness and Inconsistencies](https://mlsanity.com/paper/2310.07471/) (2023)
- [Pre-Deployment Complexity Estimation for Federated Perception Systems](https://mlsanity.com/paper/2603.28282/) (2026)
- [Flow Matching with Arbitrary Auxiliary Paths](https://mlsanity.com/paper/2605.06364/) (2026)
- [Rethinking Handwritten Character Recognition](https://mlsanity.com/paper/2609.10572/) (2026)

## Abstract (arXiv)

We report that a very high accuracy on the MNIST test set can be achieved by using simple convolutional neural network (CNN) models. We use three different models with 3x3, 5x5, and 7x7 kernel size in the convolution layers. Each model consists of a set of convolution layers followed by a single fully connected layer. Every convolution layer uses batch normalization and ReLU activation, and pooling is not used. Rotation and translation is used to augment training data, which is frequently used in most image classification tasks. A majority voting using the three models independently trained on the training data set can achieve up to 99.87% accuracy on the test set, which is one of the state-of-the-art results. A two-layer ensemble, a heterogeneous ensemble of three homogeneous ensemble networks, can achieve up to 99.91% test accuracy. The results can be reproduced by using the code at: https://github.com/ansh941/MnistSimpleCNN