mlsanityData to 7 Oct 2026

RecycleGPT: An Autoregressive Language Model with Recyclable Module

arXiv 2308.03421 ↗First posted 7 Aug 2023cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Existing large language models have to run K times to generate a sequence of K tokens. In this paper, we present RecycleGPT, a generative language model with fast decoding speed by recycling pre-generated model states without running the whole model in multiple steps. Our approach relies on the observation that adjacent tokens in a sequence usually have strong correlations and the next token in a sequence can be reasonably guessed or inferred based on the preceding ones. Experiments and analysis demonstrate the effectiveness of our approach in lowering inference latency, achieving up to 1.4x speedup while preserving high performance.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 47 of its 54 references that are in the map. First posted on arXiv 7 Aug 2023.

Builds on

47 of 54 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
35 more
  1. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  2. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  3. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  4. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  5. 2020GLU Variants Improve Transformercited by 1,724
  6. 2020Reformer: The Efficient Transformercited by 1,658
  7. 2020Rethinking Attention with Performerscited by 1,584
  8. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  9. 2019Root Mean Square Layer Normalizationcited by 1,425
  10. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  11. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  12. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  13. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  14. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  15. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  16. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  17. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  18. 2022Efficiently Scaling Transformer Inferencecited by 529
  19. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  20. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  21. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  22. 2019Learning Deep Transformer Models for Machine Translationcited by 309
  23. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  24. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  25. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 194
  26. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 177
  27. 2019Adaptive Attention Span in Transformerscited by 148
  28. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 125
  29. 2021Sparse is Enough in Scaling Transformerscited by 86
  30. 2019Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approachescited by 64
  31. 2022Datasheet for the Pilecited by 41
  32. 2020Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruningcited by 39
  33. 2019Sharing Attention Weights for Fast Transformercited by 34
  34. 2020Successfully Applying the Stabilized Lottery Ticket Hypothesis to the Transformer Architecturecited by 23
  35. 2022Lossless Acceleration for Seq2seq Generation with Aggressive Decodingcited by 18

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2.

3 papers.

  1. 2023SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Modelscited by 36
  2. 2024ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decodingcited by 12
  3. 2024Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decodingcited by 11