# Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN

arXiv: https://arxiv.org/abs/2106.12901 · HTML: https://mlsanity.com/paper/2106.12901/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 56 references that are in the map.
- First posted on arXiv 22 Jun 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Regularizing and Optimizing LSTM Language Models](https://mlsanity.com/paper/1708.02182/) (2017)
- [Exploring the Limits of Language Modeling](https://mlsanity.com/paper/1602.02410/) (2016)
- [A Convolutional Neural Network for Modelling Sentences](https://mlsanity.com/paper/1404.2188/) (2014)
- [A Simple Way to Initialize Recurrent Networks of Rectified Linear Units](https://mlsanity.com/paper/1504.00941/) (2015)
- [Full-Capacity Unitary Recurrent Neural Networks](https://mlsanity.com/paper/1611.00035/) (2016)
- [Quasi-Recurrent Neural Networks](https://mlsanity.com/paper/1611.01576/) (2016)
- [Dilated Recurrent Neural Networks](https://mlsanity.com/paper/1710.02224/) (2017)
- [Neural Machine Translation in Linear Time](https://mlsanity.com/paper/1610.10099/) (2016)
- [Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs](https://mlsanity.com/paper/1612.05231/) (2016)
- [How to Construct Deep Recurrent Neural Networks](https://mlsanity.com/paper/1312.6026/) (2013)
- [Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations](https://mlsanity.com/paper/1606.01305/) (2016)
- [Architectural Complexity Measures of Recurrent Neural Networks](https://mlsanity.com/paper/1602.08210/) (2016)
- [Factorization tricks for LSTM networks](https://mlsanity.com/paper/1703.10722/) (2017)
- [Regularizing RNNs by Stabilizing Activations](https://mlsanity.com/paper/1511.08400/) (2015)

## Built on by (most cited first, 16 of 16)

- [Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformers](https://mlsanity.com/paper/2406.12199/) (2024)
- [CU-Net: a U-Net architecture for efficient brain-tumor segmentation on BraTS 2019 dataset](https://mlsanity.com/paper/2406.13113/) (2024)
- [Research on Driver Facial Fatigue Detection Based on Yolov8 Model](https://mlsanity.com/paper/2406.18575/) (2024)
- [Exploration of Multi-Scale Image Fusion Systems in Intelligent Medical Image Analysis](https://mlsanity.com/paper/2406.18548/) (2024)
- [Advancements in Feature Extraction Recognition of Medical Imaging Systems Through Deep Learning Technique](https://mlsanity.com/paper/2406.18549/) (2024)
- [Deep Learning in Medical Image Classification from MRI-based Brain Tumor Images](https://mlsanity.com/paper/2408.00636/) (2024)
- [Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Training](https://mlsanity.com/paper/2409.04707/) (2024)
- [Stock Type Prediction Model Based on Hierarchical Graph Neural Network](https://mlsanity.com/paper/2412.06862/) (2024)
- [Real-time Video Target Tracking Algorithm Utilizing Convolutional Neural Networks (CNN)](https://mlsanity.com/paper/2411.18314/) (2024)
- [Parallelizable memory recurrent units](https://mlsanity.com/paper/2601.09495/) (2026)
- [Multicam-SLAM: Non-overlapping Multi-camera SLAM for Indirect Visual Localization and Navigation](https://mlsanity.com/paper/2406.06374/) (2024)
- [Electroencephalogram Emotion Recognition via AUC Maximization](https://mlsanity.com/paper/2408.08979/) (2024)
- [Instructional Prompt Optimization for Few-Shot LLM-Based Recommendations on Cold-Start Users](https://mlsanity.com/paper/2509.09066/) (2025)
- [An Improved Time Feedforward Connections Recurrent Neural Networks](https://mlsanity.com/paper/2211.02561/) (2022)
- [Research on Edge Detection of LiDAR Images Based on Artificial Intelligence Technology](https://mlsanity.com/paper/2406.09773/) (2024)
- [Unsupervised Graph Modeling for Anomaly Detection in Accounting Subject Relationships](https://mlsanity.com/paper/2604.26216/) (2026)

## Abstract (arXiv)

The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes the operation inefficient. The same problem was encountered in serial adder at the early stage of digital electronics. In this paper, we discuss the similarities between recurrent neural network (RNN) and serial adder. Inspired by carry-lookahead adder, we introduce carry-lookahead module to RNN, which makes it possible for RNN to run in parallel. Then, we design the method of parallel RNN computation, and finally Carry-lookahead RNN (CL-RNN) is proposed. CL-RNN takes advantages in parallelism and flexible receptive field. Through a comprehensive set of tests, we verify that CL-RNN can perform better than existing typical RNNs in sequence modeling tasks which are specially designed for RNNs.