mlsanityData to 14 Oct 2026

Pointer Sentinel Mixture Models

arXiv 1609.07843 ↗First posted 26 Sep 2016cs.CL cs.AIcited by 3,096

Not on the map. Why?

Cited by 174 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent neural network sequence models with softmax classifiers have achieved their best language modeling performance only with very large hidden states and large vocabularies. Even then they struggle to predict rare or unseen words even if the context makes the prediction unambiguous. We introduce the pointer sentinel mixture architecture for neural sequence models which has the ability to either reproduce a word from the recent context or produce a word from a standard softmax classifier. Our pointer sentinel-LSTM model achieves state of the art language modeling performance on the Penn Treebank (70.9 perplexity) while using far fewer parameters than a standard softmax LSTM. In order to evaluate how well language models can exploit longer contexts and deal with more realistic vocabularies and larger corpora we also introduce the freely available WikiText corpus.

Key facts. Cited by 3,096 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 185; 2022: 214; 2023: 368; 2024: 630; 2025: 750; 2026: 949.

3,096 papers.

Sort:

By area:

  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  4. 2023A Survey of Large Language Modelscited by 2,576
  5. 2022Emergent Abilities of Large Language Modelscited by 2,512
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  9. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  10. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  11. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  12. 2022Editing Models with Task Arithmeticcited by 1,120
38 more
  1. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  2. 2021Membership Inference Attacks From First Principlescited by 878
  3. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  4. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  5. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  6. 2024Chronos: Learning the Language of Time Seriescited by 759
  7. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  8. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  9. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  10. 2023A Comprehensive Overview of Large Language Modelscited by 648
  11. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  12. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  13. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  14. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  15. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  16. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  17. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  18. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  19. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  20. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  21. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  22. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
  23. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  24. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  25. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
  26. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 429
  27. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 427
  28. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  29. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  30. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  31. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  32. 2023Augmented Language Models: a Surveycited by 403
  33. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  34. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
  35. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  36. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 362
  37. 2021Random Feature Attentioncited by 354
  38. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337

50 most cited of 3,096 All 3,096 (JSON)