# Optimizing Neural Networks with Learnable Non-Linear Activation Functions via Lookup-Based FPGA Acceleration

arXiv: https://arxiv.org/abs/2508.17069 · HTML: https://mlsanity.com/paper/2508.17069/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 23 Aug 2025.

## Builds on (most cited first, 16 of 16)

- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [KAN 2.0: Kolmogorov-Arnold Networks Meet Science](https://mlsanity.com/paper/2408.10205/) (2024)
- [LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models](https://mlsanity.com/paper/2206.09557/) (2022)
- [A survey on modern trainable activation functions](https://mlsanity.com/paper/2005.00817/) (2020)
- [LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications](https://mlsanity.com/paper/2004.03021/) (2020)
- [PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference](https://mlsanity.com/paper/2309.02334/) (2023)
- [Fast Matrix Multiplications for Lookup Table-Quantized LLMs](https://mlsanity.com/paper/2407.10960/) (2024)
- [LUTNet: Rethinking Inference in FPGA Soft Logic](https://mlsanity.com/paper/1904.00938/) (2019)
- [F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization](https://mlsanity.com/paper/2202.05239/) (2022)
- [Deep Learning with S-shaped Rectified Linear Activation Units](https://mlsanity.com/paper/1512.07030/) (2015)
- [NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference](https://mlsanity.com/paper/2112.02191/) (2021)
- [Iteratively Training Look-Up Tables for Network Quantization](https://mlsanity.com/paper/1911.04951/) (2019)
- [Efficient Softmax Approximation for Deep Neural Networks with Attention Mechanism](https://mlsanity.com/paper/2111.10770/) (2021)
- [Table-Lookup MAC: Scalable Processing of Quantised Neural Networks in FPGA Soft Logic](https://mlsanity.com/paper/2403.11414/) (2024)
- [Logic Shrinkage: Learned FPGA Netlist Sparsity for Efficient Neural Network Inference](https://mlsanity.com/paper/2112.02346/) (2021)

## Built on by (most cited first, 1 of 1)

- [BiKA: Kolmogorov-Arnold-Network-inspired Ultra Lightweight Neural Network Hardware Accelerator](https://mlsanity.com/paper/2602.23455/) (2026)

## Abstract (arXiv)

Learned activation functions in models like Kolmogorov-Arnold Networks (KANs) outperform fixed-activation architectures in terms of accuracy and interpretability; however, their computational complexity poses critical challenges for energy-constrained edge AI deployments. Conventional CPUs/GPUs incur prohibitive latency and power costs when evaluating higher order activations, limiting deployability under ultra-tight energy budgets. We address this via a reconfigurable lookup architecture with edge FPGAs. By coupling fine-grained quantization with adaptive lookup tables, our design minimizes energy-intensive arithmetic operations while preserving activation fidelity. FPGA reconfigurability enables dynamic hardware specialization for learned functions, a key advantage for edge systems that require post-deployment adaptability. Evaluations using KANs - where unique activation functions play a critical role - demonstrate that our FPGA-based design achieves superior computational speed and over 104 times higher energy efficiency compared to edge CPUs and GPUs, while maintaining matching accuracy and minimal footprint overhead. This breakthrough positions our approach as a practical enabler for energy-critical edge AI, where computational intensity and power constraints traditionally preclude the use of adaptive activation networks.