# Neural Network Libraries: A Deep Learning Framework Designed from Engineers' Perspectives

arXiv: https://arxiv.org/abs/2102.06725 · HTML: https://mlsanity.com/paper/2102.06725/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 12 Feb 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [State-of-the-art Speech Recognition With Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01769/) (2017)
- [ReenactGAN: Learning to Reenact Faces via Boundary Transfer](https://mlsanity.com/paper/1807.11079/) (2018)
- [Data Cleansing for Models Trained with SGD](https://mlsanity.com/paper/1906.08473/) (2019)
- [Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation](https://mlsanity.com/paper/1811.09393/) (2018)

## Built on by (most cited first, 7 of 7)

- [Fine-grained Image Editing by Pixel-wise Guidance Using Diffusion Models](https://mlsanity.com/paper/2212.02024/) (2022)
- [A Low Memory Footprint Quantized Neural Network for Depth Completion of Very Sparse Time-of-Flight Depth Maps](https://mlsanity.com/paper/2205.12918/) (2022)
- [Data Cleansing for Deep Neural Networks with Storage-efficient Approximation of Influence Functions](https://mlsanity.com/paper/2103.11807/) (2021)
- [NDJIR: Neural Direct and Joint Inverse Rendering for Geometry, Lights, and Materials of Real Object](https://mlsanity.com/paper/2302.00675/) (2023)
- [Latte: Cross-framework Python Package for Evaluation of Latent-Based Generative Models](https://mlsanity.com/paper/2112.10638/) (2021)
- [Thinking the Fusion Strategy of Multi-reference Face Reenactment](https://mlsanity.com/paper/2202.10758/) (2022)
- [PRAISE: A Reproducible Workflow for Non-Programmer Scientists to Implement Published AI Models](https://mlsanity.com/paper/2610.16980/) (2026)

## Abstract (arXiv)

While there exist a plethora of deep learning tools and frameworks, the fast-growing complexity of the field brings new demands and challenges, such as more flexible network design, speedy computation on distributed setting, and compatibility between different tools. In this paper, we introduce Neural Network Libraries (https://nnabla.org), a deep learning framework designed from engineer's perspective, with emphasis on usability and compatibility as its core design principles. We elaborate on each of our design principles and its merits, and validate our attempts via experiments.