mlsanityData to 7 Oct 2026

Dynamic Evaluation of Transformer Language Models

arXiv 1904.08378 ↗First posted 17 Apr 2019cs.LG cs.NE stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

This research note combines two methods that have recently improved the state of the art in language modeling: Transformers and dynamic evaluation. Transformers use stacked layers of self-attention that allow them to capture long range dependencies in sequential data. Dynamic evaluation fits models to the recent sequence history, allowing them to assign higher probabilities to re-occurring sequential patterns. By applying dynamic evaluation to Transformer-XL models, we improve the state of the art on enwik8 from 0.99 to 0.94 bits/char, text8 from 1.08 to 1.04 bits/char, and WikiText-103 from 18.3 to 16.4 perplexity points.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 4; 2023: 2; 2024: 3; 2025: 4; 2026: 3.

23 papers.

Sort:

By area:

  1. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  2. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  3. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  4. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 146
  5. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  6. 2021Adaptive Semiparametric Language Modelscited by 83
  7. 2023Test-Time Training on Nearest Neighbors for Large Language Modelscited by 82
  8. 2021Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Mediacited by 51
  9. 2021GNN-LM: Language Modeling based on Global Contexts via GNNcited by 27
  10. 2022Relational Memory Augmented Language Modelscited by 24
  11. 2022Meta-Learning Fast Weight Language Modelscited by 20
  12. 2023Trainable Transformer in Transformercited by 17
11 more
  1. 2025Local Mixtures of Experts: Essentially Free Test-Time Training via Model Mergingcited by 9
  2. 2025Test-time Offline Reinforcement Learning on Goal-related Experiencecited by 8
  3. 2024Revisiting Dynamic Evaluation: Online Adaptation for Large Language Modelscited by 7
  4. 2022Optimizing Test-Time Query Representations for Dense Retrievalcited by 6
  5. 2021Reconsidering the Past: Optimizing Hidden States in Language Modelscited by 3
  6. 2025Maximizing Prefix-Confidence at Test-Time Efficiently Improves Mathematical Reasoningcited by 3
  7. 2026Reflective Prompt Tuning through Language Model Function-Callingcited by 1
  8. 2021Effective Cross-Utterance Language Modeling for Conversational Speech Recognitioncited by 0
  9. 2024Survey: Transformer-based Models in Data Modality Conversioncited by 0
  10. 2026Unsupervised Layer-Wise Dynamic Test Time Adaptation for LLMscited by 0
  11. 2026DeepEdu-v1: Efficient and Scalable Agentic LLMs for Vietnamese Educationcited by 0