# TaxoNN: A Light-Weight Accelerator for Deep Neural Network Training

arXiv: https://arxiv.org/abs/2010.05197 · HTML: https://mlsanity.com/paper/2010.05197/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2020.

## Built on by (most cited first, 2 of 2)

- [The Backpropagation algorithm for a math student](https://mlsanity.com/paper/2301.09977/) (2023)
- [Precision-Scalable Microscaling Datapaths with Optimized Reduction Tree for Efficient NPU Integration](https://mlsanity.com/paper/2511.06313/) (2025)

## Abstract (arXiv)

Emerging intelligent embedded devices rely on Deep Neural Networks (DNNs) to be able to interact with the real-world environment. This interaction comes with the ability to retrain DNNs, since environmental conditions change continuously in time. Stochastic Gradient Descent (SGD) is a widely used algorithm to train DNNs by optimizing the parameters over the training data iteratively. In this work, first we present a novel approach to add the training ability to a baseline DNN accelerator (inference only) by splitting the SGD algorithm into simple computational elements. Then, based on this heuristic approach we propose TaxoNN, a light-weight accelerator for DNN training. TaxoNN can easily tune the DNN weights by reusing the hardware resources used in the inference process using a time-multiplexing approach and low-bitwidth units. Our experimental results show that TaxoNN delivers, on average, 0.97% higher misclassification rate compared to a full-precision implementation. Moreover, TaxoNN provides 2.1× power saving and 1.65× area reduction over the state-of-the-art DNN training accelerator.