# Design Smells in Deep Learning Programs: An Empirical Study

arXiv: https://arxiv.org/abs/2107.02279 · HTML: https://mlsanity.com/paper/2107.02279/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 5 Jul 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [An Analysis of Deep Neural Network Models for Practical Applications](https://mlsanity.com/paper/1605.07678/) (2016)
- [Taxonomy of Real Faults in Deep Learning Systems](https://mlsanity.com/paper/1910.11015/) (2019)
- [Convolutional Neural Networks at Constrained Time Cost](https://mlsanity.com/paper/1412.1710/) (2014)
- [A Comprehensive Study on Deep Learning Bug Characteristics](https://mlsanity.com/paper/1906.01388/) (2019)
- [Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift](https://mlsanity.com/paper/1801.05134/) (2018)
- [Repairing Deep Neural Networks: Fix Patterns and Challenges](https://mlsanity.com/paper/2005.00972/) (2020)
- [Towards Principled Design of Deep Convolutional Networks: Introducing SimpNet](https://mlsanity.com/paper/1802.06205/) (2018)
- [Deep Convolutional Neural Network Design Patterns](https://mlsanity.com/paper/1611.00847/) (2016)

## Built on by (most cited first, 10 of 10)

- [Bugs in Machine Learning-based Systems: A Faultload Benchmark](https://mlsanity.com/paper/2206.12311/) (2022)
- [Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Vision](https://mlsanity.com/paper/2303.07476/) (2023)
- [Quality issues in Machine Learning Software Systems](https://mlsanity.com/paper/2208.08982/) (2022)
- [Challenges in Migrating Imperative Deep Learning Programs to Graph Execution: An Empirical Study](https://mlsanity.com/paper/2201.09953/) (2022)
- [Quality Issues in Machine Learning Software Systems](https://mlsanity.com/paper/2306.15007/) (2023)
- [On Security Weaknesses and Vulnerabilities in Deep Learning Systems](https://mlsanity.com/paper/2406.08688/) (2024)
- [MixFunn: A Neural Network for Differential Equations with Improved Generalization and Interpretability](https://mlsanity.com/paper/2503.22528/) (2025)
- [Speculative Automated Refactoring of Imperative Deep Learning Programs to Graph Execution](https://mlsanity.com/paper/2504.05424/) (2025)
- [Analysing Python Machine Learning Notebooks with Moose](https://mlsanity.com/paper/2509.11748/) (2025)
- [From AI Technical Debt to Agentic Technical Debt: A Systematic Mapping of Root Causes and Manifestations in Agentic AI Systems](https://mlsanity.com/paper/2608.01001/) (2026)

## Abstract (arXiv)

Nowadays, we are witnessing an increasing adoption of Deep Learning (DL) based software systems in many industries. Designing a DL program requires constructing a deep neural network (DNN) and then training it on a dataset. This process requires that developers make multiple architectural (e.g., type, size, number, and order of layers) and configuration (e.g., optimizer, regularization methods, and activation functions) choices that affect the quality of the DL models, and consequently software quality. An under-specified or poorly-designed DL model may train successfully but is likely to perform poorly when deployed in production. Design smells in DL programs are poor design and-or configuration decisions taken during the development of DL components, that are likely to have a negative impact on the performance (i.e., prediction accuracy) and then quality of DL based software systems. In this paper, we present a catalogue of 8 design smells for a popular DL architecture, namely deep Feedforward Neural Networks which is widely employed in industrial applications. The design smells were identified through a review of the existing literature on DL design and a manual inspection of 659 DL programs with performance issues and design inefficiencies. The smells are specified by describing their context, consequences, and recommended refactorings. To provide empirical evidence on the relevance and perceived impact of the proposed design smells, we conducted a survey with 81 DL developers. In general, the developers perceived the proposed design smells as reflective of design or implementation problems, with agreement levels varying between 47% and 68%.