# C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs

arXiv: https://arxiv.org/abs/1803.06305 · HTML: https://mlsanity.com/paper/1803.06305/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2018.

## Built on by (most cited first, 21 of 21)

- [Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design](https://mlsanity.com/paper/2209.09570/) (2022)
- [Spartus: A 9.4 TOp/s FPGA-based LSTM Accelerator Exploiting Spatio-Temporal Sparsity](https://mlsanity.com/paper/2108.02297/) (2021)
- [An Automatic and Efficient BERT Pruning for Edge AI Systems](https://mlsanity.com/paper/2206.10461/) (2022)
- [RecLight: A Recurrent Neural Network Accelerator with Integrated Silicon Photonics](https://mlsanity.com/paper/2209.00084/) (2022)
- [BlockGNN: Towards Efficient GNN Acceleration Using Block-Circulant Weight Matrices](https://mlsanity.com/paper/2104.06214/) (2021)
- [GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity](https://mlsanity.com/paper/2108.11033/) (2021)
- [A survey on FPGA-based accelerator for ML models](https://mlsanity.com/paper/2412.15666/) (2024)
- [A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques](https://mlsanity.com/paper/2302.08571/) (2023)
- [Vau da muntanialas: Energy-efficient multi-die scalable acceleration of RNN inference](https://mlsanity.com/paper/2202.07462/) (2022)
- [Parameter-Efficient Fine-Tuning with Circulant and Diagonal Vectors](https://mlsanity.com/paper/2505.00580/) (2025)
- [BRDS: An FPGA-based LSTM Accelerator with Row-Balanced Dual-Ratio Sparsification](https://mlsanity.com/paper/2101.02667/) (2021)
- [Smartphone Impostor Detection with Behavioral Data Privacy and Minimalist Hardware Support](https://mlsanity.com/paper/2103.06453/) (2021)
- [Algorithm and Hardware Co-Design of Energy-Efficient LSTM Networks for Video Recognition with Hierarchical Tucker Tensor Decomposition](https://mlsanity.com/paper/2212.02046/) (2022)
- [Robust Group Anomaly Detection for Quasi-Periodic Network Time Series](https://mlsanity.com/paper/2506.16815/) (2025)
- [MOHAQ: Multi-Objective Hardware-Aware Quantization of Recurrent Neural Networks](https://mlsanity.com/paper/2108.01192/) (2021)
- [Efficient Non-linear Calculators](https://mlsanity.com/paper/2109.12686/) (2021)
- [SenDaL: An Effective and Efficient Calibration Framework of Low-Cost Sensors for Daily Life](https://mlsanity.com/paper/2502.10460/) (2025)
- [Penrose Tiled Low-Rank Compression and Section-Wise Q&A Fine-Tuning: A General Framework for Domain-Specific Large Language Model Adaptation](https://mlsanity.com/paper/2503.22074/) (2025)
- [FINN-GL: Generalized Mixed-Precision Extensions for FPGA-Accelerated LSTMs](https://mlsanity.com/paper/2506.20810/) (2025)
- [FLToP CTC: Frame-Level Token Pruning via Relative Threshold for Efficient and Memory-Saving Decoding on Diverse Platforms](https://mlsanity.com/paper/2510.09085/) (2025)
- [Mugi: Value Level Parallelism For Efficient LLMs](https://mlsanity.com/paper/2601.10823/) (2026)

## Abstract (arXiv)

Recently, significant accuracy improvement has been achieved for acoustic recognition systems by increasing the model size of Long Short-Term Memory (LSTM) networks. Unfortunately, the ever-increasing size of LSTM model leads to inefficient designs on FPGAs due to the limited on-chip resources. The previous work proposes to use a pruning based compression technique to reduce the model size and thus speedups the inference on FPGAs. However, the random nature of the pruning technique transforms the dense matrices of the model to highly unstructured sparse ones, which leads to unbalanced computation and irregular memory accesses and thus hurts the overall performance and energy efficiency. In contrast, we propose to use a structured compression technique which could not only reduce the LSTM model size but also eliminate the irregularities of computation and memory accesses. This approach employs block-circulant instead of sparse matrices to compress weight matrices and reduces the storage requirement from O(k2) to O(k). Fast Fourier Transform algorithm is utilized to further accelerate the inference by reducing the computational complexity from O(k2) to O(klogk). The datapath and activation functions are quantized as 16-bit to improve the resource utilization. More importantly, we propose a comprehensive framework called C-LSTM to automatically optimize and implement a wide range of LSTM variants on FPGAs. According to the experimental results, C-LSTM achieves up to 18.8X and 33.5X gains for performance and energy efficiency compared with the state-of-the-art LSTM implementation under the same experimental setup, and the accuracy degradation is very small.