mlsanityData to 7 Oct 2026

The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives

arXiv 1909.01380 ↗First posted 3 Sep 2019cs.CLcited by 139

Not on the map. Why?

Abstract (arXiv)

We seek to understand how the representations of individual tokens and the structure of the learned feature space evolve between layers in deep neural networks under different learning objectives. We focus on the Transformers for our analysis as they have been shown effective on various tasks, including machine translation (MT), standard left-to-right language models (LM) and masked language modeling (MLM). Previous work used black-box probing tasks to show that the representations learned by the Transformer differ significantly depending on the objective. In this work, we use canonical correlation analysis and mutual information estimators to study how information flows across Transformer layers and how this process depends on the choice of learning objective. For example, as you go from bottom to top layers, information about the past in left-to-right language models gets vanished and predictions about the future get formed. In contrast, for MLM, representations initially acquire information about the context around the token, partially forgetting the token identity and producing a more generalized token representation. The token identity then gets recreated at the top MLM layers.

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 23; 2023: 27; 2024: 23; 2025: 20; 2026: 20.

139 papers.

Sort:

By area:

  1. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  2. 2021Editing Factual Knowledge in Language Modelscited by 548
  3. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  4. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  5. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  6. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
  7. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 293
  8. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  9. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 149
  10. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 148
  11. 2022Analyzing Transformers in Embedding Spacecited by 131
  12. 2023Large Language Models in Law: A Surveycited by 128
38 more
  1. 2022Should You Mask 15% in Masked Language Modeling?cited by 117
  2. 2021Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoTcited by 104
  3. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 101
  4. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 97
  5. 2022The Geometry of Multilingual Language Model Representationscited by 94
  6. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
  7. 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
  8. 2021Representation Topology Divergence: A Method for Comparing Neural Network Representationscited by 66
  9. 2023What Do Self-Supervised Speech Models Know About Words?cited by 62
  10. 2024Factual Confidence of LLMs: on Reliability and Robustness of Current Estimatorscited by 59
  11. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 54
  12. 2021Variational Information Bottleneck for Effective Low-Resource Fine-Tuningcited by 51
  13. 2024From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuningcited by 51
  14. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45
  15. 2023Privacy-Preserving Parameter-Efficient Fine-Tuning for Large Language Model Servicescited by 43
  16. 2024Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semanticscited by 43
  17. 2023Explaining How Transformers Use Context to Build Predictionscited by 42
  18. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  19. 2021Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Sensescited by 39
  20. 2021The Low-Dimensional Linear Geometry of Contextualized Word Representationscited by 39
  21. 2021Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Modelscited by 39
  22. 2024LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Taskscited by 39
  23. 2021Discourse Probing of Pretrained Language Modelscited by 38
  24. 2021Automatic Text Evaluation through the Lens of Wasserstein Barycenterscited by 38
  25. 2021On Compositional Generalization of Neural Machine Translationcited by 34
  26. 2023A Survey on Knowledge Editing of Neural Networkscited by 34
  27. 2021Exploring the Role of BERT Token Representations to Explain Sentence Probing Resultscited by 29
  28. 2021Fine-Tuned Transformers Show Clusters of Similar Representations Across Layerscited by 29
  29. 2022Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translationcited by 28
  30. 2021Translation Error Detection as Rationale Extractioncited by 22
  31. 2023Text Rendering Strategies for Pixel Language Modelscited by 22
  32. 2021CoBERL: Contrastive BERT for Reinforcement Learningcited by 20
  33. 2022How to Dissect a Muppet: The Structure of Transformer Embedding Spacescited by 19
  34. 2021LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyondcited by 17
  35. 2021Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representationscited by 16
  36. 2024The Hidden Space of Transformer Language Adapterscited by 16
  37. 2021Inspecting the concept knowledge graph encoded by modern language modelscited by 15
  38. 2022On Robust Prefix-Tuning for Text Classificationcited by 15

50 most cited of 139 All 139 (JSON)