# Efficient Non-linear Calculators

arXiv: https://arxiv.org/abs/2109.12686 · HTML: https://mlsanity.com/paper/2109.12686/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 20 references that are in the map.
- First posted on arXiv 26 Sep 2021.

## Builds on (most cited first, 4 of 4)

- [ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA](https://mlsanity.com/paper/1612.00694/) (2016)
- [C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs](https://mlsanity.com/paper/1803.06305/) (2018)
- [FINN-L: Library Extensions and Design Trade-off Analysis for Variable Precision LSTM Networks on FPGAs](https://mlsanity.com/paper/1807.04093/) (2018)
- [Design Space Exploration of Neural Network Activation Function Circuits](https://mlsanity.com/paper/1810.08650/) (2018)

## Abstract (arXiv)

A novel algorithm for producing smooth nonlinearities on digital hardware is presented. The non-linearities are inherently quadratic and have both symmetrical and asymmetrical variants. The integer (and fixed point) implementation is highly amenable for use with digital gates on an ASIC or FPGA. The implementations are multiplier-less. Scaling of the non-linear output, as required in an LSTM cell, is integrated into the implementation. This too does not require a multiplier. The non-linearities are useful as activation functions in a variety of ANN architectures. The floating point mappings have been compared with other non-linearities and have been benchmarked. Results show that these functions should be considered in the ANN design phase. The hardware resource usage of the implementations have been thoroughly investigated. Our results make a strong case for implementions in edge applications. This document summarizes the findings and serves to give a quick overview of the outcomes of our researchfootnoteThe authors peer-reviewed manuscripts (available at https://doi.org/10.1016/j.neucom.2021.02.030) offer more detail and may be better suited for a thorough consideration.