# MNIST-Gen: A Modular MNIST-Style Dataset Generation Using Hierarchical Semantics, Reinforcement Learning, and Category Theory

arXiv: https://arxiv.org/abs/2507.11821 · HTML: https://mlsanity.com/paper/2507.11821/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 16 Jul 2025.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 12 of 12)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Learning Visual Features from Large Weakly Supervised Data](https://mlsanity.com/paper/1511.02251/) (2015)
- [Can LLMs Improve Multimodal Fact-Checking by Asking Relevant Questions?](https://mlsanity.com/paper/2410.04616/) (2024)
- [A Semi-supervised Fake News Detection using Sentiment Encoding and LSTM with Self-Attention](https://mlsanity.com/paper/2407.19332/) (2024)
- [A Multimodal Physics-Informed Neural Network Approach for Mean Radiant Temperature Modeling](https://mlsanity.com/paper/2503.08482/) (2025)
- [Sentiment and Social Signals in the Climate Crisis: A Survey on Analyzing Social Media Responses to Extreme Weather Events](https://mlsanity.com/paper/2504.18837/) (2025)
- [DocParseNet: Advanced Semantic Segmentation and OCR Embeddings for Efficient Scanned Document Annotation](https://mlsanity.com/paper/2406.17591/) (2024)
- [MID-L: Matrix-Interpolated Dropout Layer with Layer-wise Neuron Selection](https://mlsanity.com/paper/2505.11416/) (2025)

## Built on by (most cited first, 2 of 2)

- [Explainable Human-in-the-Loop Segmentation via Critic Feedback Signals](https://mlsanity.com/paper/2510.09945/) (2025)
- [Tri-Accel: Curvature-Aware Precision-Adaptive and Memory-Elastic Optimization for Efficient GPU Usage](https://mlsanity.com/paper/2508.16905/) (2025)

## Abstract (arXiv)

Neural networks are often benchmarked using standard datasets such as MNIST, FashionMNIST, or other variants of MNIST, which, while accessible, are limited to generic classes such as digits or clothing items. For researchers working on domain-specific tasks, such as classifying trees, food items, or other real-world objects, these data sets are insufficient and irrelevant. Additionally, creating and publishing a custom dataset can be time consuming, legally constrained, or beyond the scope of individual projects. We present MNIST-Gen, an automated, modular, and adaptive framework for generating MNIST-style image datasets tailored to user-specified categories using hierarchical semantic categorization. The system combines CLIP-based semantic understanding with reinforcement learning and human feedback to achieve intelligent categorization with minimal manual intervention. Our hierarchical approach supports complex category structures with semantic characteristics, enabling fine-grained subcategorization and multiple processing modes: individual review for maximum control, smart batch processing for large datasets, and fast batch processing for rapid creation. Inspired by category theory, MNIST-Gen models each data transformation stage as a composable morphism, enhancing clarity, modularity, and extensibility. As proof of concept, we generate and benchmark two novel datasets-textitTree-MNIST and textitFood-MNIST-demonstrating MNIST-Gen's utility for producing task-specific evaluation data while achieving 85% automatic categorization accuracy and 80% time savings compared to manual approaches.