mlsanityData to 7 Oct 2026

Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN

arXiv 2106.12901 ↗First posted 22 Jun 2021cs.LG cs.NEcited by 16

Not on the map. Why?

Abstract (arXiv)

The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes the operation inefficient. The same problem was encountered in serial adder at the early stage of digital electronics. In this paper, we discuss the similarities between recurrent neural network (RNN) and serial adder. Inspired by carry-lookahead adder, we introduce carry-lookahead module to RNN, which makes it possible for RNN to run in parallel. Then, we design the method of parallel RNN computation, and finally Carry-lookahead RNN (CL-RNN) is proposed. CL-RNN takes advantages in parallelism and flexible receptive field. Through a comprehensive set of tests, we verify that CL-RNN can perform better than existing typical RNNs in sequence modeling tasks which are specially designed for RNNs.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 32 of its 56 references that are in the map. First posted on arXiv 22 Jun 2021.

Builds on

32 of 56 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2014Going Deeper with Convolutionscited by 4,231
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  8. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  9. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  10. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  11. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  12. 2014Distributed Representations of Sentences and Documentscited by 677
20 more
  1. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  2. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  3. 2017Residual Attention Network for Image Classificationcited by 385
  4. 2015LSTM: A Search Space Odysseycited by 315
  5. 2017Regularizing and Optimizing LSTM Language Modelscited by 205
  6. 2016Exploring the Limits of Language Modelingcited by 172
  7. 2014A Convolutional Neural Network for Modelling Sentencescited by 163
  8. 2015A Simple Way to Initialize Recurrent Networks of Rectified Linear Unitscited by 130
  9. 2016Full-Capacity Unitary Recurrent Neural Networkscited by 90
  10. 2016Quasi-Recurrent Neural Networkscited by 87
  11. 2017Dilated Recurrent Neural Networkscited by 83
  12. 2016Neural Machine Translation in Linear Timecited by 65
  13. 2013How to Construct Deep Recurrent Neural Networkscited by 64
  14. 2016Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNscited by 62
  15. 2016Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activationscited by 54
  16. 2016Architectural Complexity Measures of Recurrent Neural Networkscited by 17
  17. 2017Factorization tricks for LSTM networkscited by 13
  18. 2015Regularizing RNNs by Stabilizing Activationscited by 6
  19. 2016Gated Word-Character Recurrent Language Modelcited by 5
  20. 2019ERSFQ 8-bit Parallel Arithmetic Logic Unitcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 0; 2024: 12; 2025: 1; 2026: 2.

16 papers.

Sort:

  1. 2024Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformerscited by 35
  2. 2024CU-Net: a U-Net architecture for efficient brain-tumor segmentation on BraTS 2019 datasetcited by 24
  3. 2024Research on Driver Facial Fatigue Detection Based on Yolov8 Modelcited by 11
  4. 2024Exploration of Multi-Scale Image Fusion Systems in Intelligent Medical Image Analysiscited by 10
  5. 2024Advancements in Feature Extraction Recognition of Medical Imaging Systems Through Deep Learning Techniquecited by 7
  6. 2024Deep Learning in Medical Image Classification from MRI-based Brain Tumor Imagescited by 7
  7. 2024Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Trainingcited by 7
  8. 2024Stock Type Prediction Model Based on Hierarchical Graph Neural Networkcited by 5
  9. 2024Real-time Video Target Tracking Algorithm Utilizing Convolutional Neural Networks (CNN)cited by 4
  10. 2024Multicam-SLAM: Non-overlapping Multi-camera SLAM for Indirect Visual Localization and Navigationcited by 2
  11. 2024Electroencephalogram Emotion Recognition via AUC Maximizationcited by 2
  12. 2026Parallelizable memory recurrent unitscited by 2
4 more
  1. 2025Instructional Prompt Optimization for Few-Shot LLM-Based Recommendations on Cold-Start Userscited by 1
  2. 2022An Improved Time Feedforward Connections Recurrent Neural Networkscited by 0
  3. 2024Research on Edge Detection of LiDAR Images Based on Artificial Intelligence Technologycited by 0
  4. 2026Unsupervised Graph Modeling for Anomaly Detection in Accounting Subject Relationshipscited by 0