mlsanityData to 7 Oct 2026

Unbounded cache model for online language modeling with open vocabulary

arXiv 1711.02604 ↗First posted 7 Nov 2017cs.LG cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

Recently, continuous cache models were proposed as extensions to recurrent neural network language models, to adapt their predictions to local changes in the data distribution. These models only capture the local context, of up to a few thousands tokens. In this paper, we propose an extension of continuous cache models, which can scale to larger contexts. In particular, we use a large scale non-parametric memory component that stores all the hidden activations seen in the past. We leverage recent advances in approximate nearest neighbor search and quantization algorithms to store millions of representations while searching them efficiently. We conduct extensive experiments showing that our approach significantly improves the perplexity of pre-trained language models on new distributions, and can scale efficiently to much larger contexts than previously proposed local cache models.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 10; 2023: 5; 2024: 6; 2025: 5; 2026: 3.

37 papers.

Sort:

By area:

  1. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  2. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  3. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
  4. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 355
  5. 2024Efficient Test-Time Adaptation of Vision-Language Modelscited by 138
  6. 2022Training Language Models with Memory Augmentationcited by 112
  7. 2021Adaptive Semiparametric Language Modelscited by 83
  8. 2022Tranception: protein fitness prediction with autoregressive transformers and inference-time retrievalcited by 78
  9. 2022Neuro-Symbolic Language Modeling with Automaton-augmented Retrievalcited by 60
  10. 2024Larimar: Large Language Models with Episodic Memory Controlcited by 53
  11. 2022Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learningcited by 36
  12. 2023Why do Nearest Neighbor Language Models Work?cited by 29
25 more
  1. 2021A Survey on Open Set Recognitioncited by 25
  2. 2024Efficient Prompt Caching via Embedding Similaritycited by 9
  3. 2021Capturing Structural Locality in Non-parametric Language Modelscited by 8
  4. 2021Rethinking Nearest Neighbors for Visual Classificationcited by 6
  5. 2023HistAlign: Improving Context Dependency in Language Generation by Aligning with Historycited by 6
  6. 2024RAP: Retrieval-Augmented Planner for Adaptive Procedure Planning in Instructional Videoscited by 6
  7. 2023Nearest Neighbors Meet Deep Neural Networks for Point Cloud Analysiscited by 5
  8. 2022Artefact Retrieval: Overview of NLP Models with Knowledge Base Accesscited by 4
  9. 2022kNN-Embed: Locally Smoothed Embedding Mixtures For Multi-interest Candidate Retrievalcited by 4
  10. 2025Point-Cache: Test-time Dynamic and Hierarchical Cache for Robust and Generalizable Point Cloud Analysiscited by 4
  11. 2025Protriever: End-to-End Differentiable Protein Homology Search for Fitness Predictioncited by 4
  12. 2025Unified modality separation: A vision-language framework for unsupervised domain adaptationcited by 4
  13. 2025ETTA: Efficient Test-Time Adaptation for Vision-Language Models through Dynamic Embedding Updatescited by 3
  14. 2022Nearest Neighbor Language Models for Stylistic Controllable Generationcited by 2
  15. 2021Towards Continual Entity Learning in Language Models for Conversational Agentscited by 1
  16. 2023N-Gram Nearest Neighbor Machine Translationcited by 1
  17. 2025Long-Tail Crisis in Nearest Neighbor Language Modelscited by 1
  18. 2021CNN with large memory layerscited by 0
  19. 2021Representation Memorization for Fast Learning New Knowledge without Forgettingcited by 0
  20. 2022Learning to Adapt Clinical Sequences with Residual Mixture of Expertscited by 0
  21. 2023Noise-Tolerant Few-Shot Unsupervised Adapter for Vision-Language Modelscited by 0
  22. 2024Learning to Adapt Category Consistent Meta-Feature of CLIP for Few-Shot Classificationcited by 0
  23. 2026DiCLIP: Diffusion Model Enhances CLIP's Dense Knowledge for Weakly Supervised Semantic Segmentationcited by 0
  24. 2026Plug-and-play Class-aware Knowledge Injection for Prompt Learning with Visual-Language Modelcited by 0
  25. 2026SkelHCC: A Hyperbolic CLIP-Driven Cache Adaptation Framework for Skeleton-based One-Shot Action Recognitioncited by 0