# Manas: Mining Software Repositories to Assist AutoML

arXiv: https://arxiv.org/abs/2112.03395 · HTML: https://mlsanity.com/paper/2112.03395/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 63 references that are in the map.
- First posted on arXiv 6 Dec 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Large-Scale Evolution of Image Classifiers](https://mlsanity.com/paper/1703.01041/) (2017)
- [Designing Neural Network Architectures using Reinforcement Learning](https://mlsanity.com/paper/1611.02167/) (2016)
- [Net2Net: Accelerating Learning via Knowledge Transfer](https://mlsanity.com/paper/1511.05641/) (2015)
- [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018)
- [Auto-Keras: An Efficient Neural Architecture Search System](https://mlsanity.com/paper/1806.10282/) (2018)
- [Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution](https://mlsanity.com/paper/1804.09081/) (2018)
- [Genetic CNN](https://mlsanity.com/paper/1703.01513/) (2017)
- [DetNAS: Backbone Search for Object Detection](https://mlsanity.com/paper/1903.10979/) (2019)
- [Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science](https://mlsanity.com/paper/1603.06212/) (2016)
- [Network Morphism](https://mlsanity.com/paper/1603.01670/) (2016)
- [A Comprehensive Study on Deep Learning Bug Characteristics](https://mlsanity.com/paper/1906.01388/) (2019)
- [A Genetic Programming Approach to Designing Convolutional Neural Network Architectures](https://mlsanity.com/paper/1704.00764/) (2017)
- [Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness](https://mlsanity.com/paper/2005.12379/) (2020)
- [Simple And Efficient Architecture Search for Convolutional Neural Networks](https://mlsanity.com/paper/1711.04528/) (2017)
- [DeepLocalize: Fault Localization for Deep Neural Networks](https://mlsanity.com/paper/2103.03376/) (2021)
- [XNAS: Neural Architecture Search with Expert Advice](https://mlsanity.com/paper/1906.08031/) (2019)
- [Repairing Deep Neural Networks: Fix Patterns and Challenges](https://mlsanity.com/paper/2005.00972/) (2020)
- [Understanding the Factors that Impact the Popularity of GitHub Repositories](https://mlsanity.com/paper/1606.04984/) (2016)
- [FireCaffe: near-linear acceleration of deep neural network training on compute clusters](https://mlsanity.com/paper/1511.00175/) (2015)
- [Automating biomedical data science through tree-based pipeline optimization](https://mlsanity.com/paper/1601.07925/) (2016)
- [The Art and Practice of Data Science Pipelines: A Comprehensive Study of Data Science Pipelines In Theory, In-The-Small, and In-The-Large](https://mlsanity.com/paper/2112.01590/) (2021)
- [Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detection](https://mlsanity.com/paper/1909.02293/) (2019)

## Built on by (most cited first, 5 of 5)

- [DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs](https://mlsanity.com/paper/2112.04036/) (2021)
- [A Retrieval-Augmented Generation Approach to Extracting Algorithmic Logic from Neural Networks](https://mlsanity.com/paper/2512.04329/) (2025)
- [Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoML](https://mlsanity.com/paper/2306.09297/) (2023)
- [DREAM: Debugging and Repairing AutoML Pipelines](https://mlsanity.com/paper/2401.00379/) (2023)
- [Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deployment](https://mlsanity.com/paper/2401.14628/) (2024)

## Abstract (arXiv)

Today deep learning is widely used for building software. A software engineering problem with deep learning is that finding an appropriate convolutional neural network (CNN) model for the task can be a challenge for developers. Recent work on AutoML, more precisely neural architecture search (NAS), embodied by tools like Auto-Keras aims to solve this problem by essentially viewing it as a search problem where the starting point is a default CNN model, and mutation of this CNN model allows exploration of the space of CNN models to find a CNN model that will work best for the problem. These works have had significant success in producing high-accuracy CNN models. There are two problems, however. First, NAS can be very costly, often taking several hours to complete. Second, CNN models produced by NAS can be very complex that makes it harder to understand them and costlier to train them. We propose a novel approach for NAS, where instead of starting from a default CNN model, the initial model is selected from a repository of models extracted from GitHub. The intuition being that developers solving a similar problem may have developed a better starting point compared to the default model. We also analyze common layer patterns of CNN models in the wild to understand changes that the developers make to improve their models. Our approach uses commonly occurring changes as mutation operators in NAS. We have extended Auto-Keras to implement our approach. Our evaluation using 8 top voted problems from Kaggle for tasks including image classification and image regression shows that given the same search time, without loss of accuracy, Manas produces models with 42.9% to 99.6% fewer number of parameters than Auto-Keras' models. Benchmarked on GPU, Manas' models train 30.3% to 641.6% faster than Auto-Keras' models.