mlsanityData to 7 Oct 2026

Dynamic Evaluation of Neural Sequence Models

arXiv 1709.07432 ↗First posted 21 Sep 2017cs.NE cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

We present methodology for using dynamic evaluation to improve neural sequence models. Models are adapted to recent history via a gradient descent based mechanism, causing them to assign higher probabilities to re-occurring sequential patterns. Dynamic evaluation outperforms existing adaptation approaches in our comparisons. Dynamic evaluation improves the state-of-the-art word-level perplexities on the Penn Treebank and WikiText-2 datasets to 51.1 and 44.3 respectively, and the state-of-the-art character-level cross-entropies on the text8 and Hutter Prize datasets to 1.19 bits/char and 1.08 bits/char respectively.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 10; 2023: 6; 2024: 7; 2025: 10; 2026: 8.

53 papers.

Sort:

By area:

  1. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  2. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  3. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  4. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  5. 2023DyVal: Dynamic Evaluation of Large Language Models for Reasoning Taskscited by 100
  6. 2021Adaptive Semiparametric Language Modelscited by 83
  7. 2023Test-Time Training on Nearest Neighbors for Large Language Modelscited by 82
  8. 2025End-to-End Test-Time Training for Long Contextcited by 53
  9. 2021Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformerscited by 42
  10. 2022A Modern Self-Referential Weight Matrix That Learns to Modify Itselfcited by 40
  11. 2025MesaNet: Sequence Modeling by Locally Optimal Test-Time Trainingcited by 37
  12. 2021When Attention Meets Fast Recurrence: Training Language Models with Reduced Computecited by 34
38 more
  1. 2021Selfish Sparse RNN Trainingcited by 26
  2. 2022Efficient recurrent architectures through activity sparsity and sparse back-propagation through timecited by 26
  3. 2025Latent Thought Models with Variational Bayes Inference-Time Computationcited by 25
  4. 2022Relational Memory Augmented Language Modelscited by 24
  5. 2025Pre-training under infinite computecited by 24
  6. 2022Meta-Learning Fast Weight Language Modelscited by 20
  7. 2023Exploring the Promise and Limits of Real-Time Recurrent Learningcited by 19
  8. 2024Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testingcited by 17
  9. 2022A Dirichlet Process Mixture of Robust Task Models for Scalable Lifelong Reinforcement Learningcited by 15
  10. 2023kNN-Adapter: Efficient Domain Adaptation for Black-Box Language Modelscited by 13
  11. 2023The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Computecited by 10
  12. 2025Local Mixtures of Experts: Essentially Free Test-Time Training via Model Mergingcited by 9
  13. 2024Active Fine-Tuning of Multi-Task Policiescited by 8
  14. 2025Test-time Offline Reinforcement Learning on Goal-related Experiencecited by 8
  15. 2024Revisiting Dynamic Evaluation: Online Adaptation for Large Language Modelscited by 7
  16. 2022Construction Repetition Reduces Information Rate in Dialoguecited by 5
  17. 2022Training Integer-Only Deep Recurrent Neural Networkscited by 5
  18. 2024Collapse of Self-trained Language Modelscited by 5
  19. 2025Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Modelscited by 5
  20. 2024Towards interfacing large language models with ASR systems using confidence measures and promptingcited by 4
  21. 2021iRNN: Integer-only Recurrent Neural Networkcited by 3
  22. 2021Reconsidering the Past: Optimizing Hidden States in Language Modelscited by 3
  23. 2026Fast Spatial Memory with Elastic Test-Time Trainingcited by 3
  24. 2026MDN: Parallelizing Stepwise Momentum for Delta Linear Attentioncited by 3
  25. 2021Predictive Representation Learning for Language Modelingcited by 1
  26. 2025Attention is All You Need Until You Need Retentioncited by 1
  27. 2025ETT: Expanding the Long Context Understanding Capability of LLMs at Test-Timecited by 1
  28. 2026Modular TTT: Rethinking Test-Time Training as Composable Modulescited by 1
  29. 2021Recoding latent sentence representations -- Dynamic gradient-based activation modification in RNNscited by 0
  30. 2021Go Forth and Prosper: Language Modeling with Ancient Textual Historycited by 0
  31. 2021The Importance of the Current Input in Sequence Modelingcited by 0
  32. 2022Learning to Adapt Clinical Sequences with Residual Mixture of Expertscited by 0
  33. 2022Circling Back to Recurrent Models of Languagecited by 0
  34. 2023Local-to-Global Information Communication for Real-Time Semantic Segmentation Network Searchcited by 0
  35. 2024Self-Train Before You Transcribecited by 0
  36. 2025Reproducibility Report: Test-Time Training on Nearest Neighbors for Large Language Modelscited by 0
  37. 2026TABED: Test-Time Adaptive Ensemble Drafting for Robust Speculative Decoding in LVLMscited by 0
  38. 2026Learning Inference Concurrency in DynamicGate MLP Structural and Mathematical Justificationcited by 0

50 most cited of 53 All 53 (JSON)