mlsanityData to 7 Oct 2026

What Happens To BERT Embeddings During Fine-tuning?

arXiv 2004.14448 ↗First posted 29 Apr 2020cs.CLcited by 113

Not on the map. Why?

Abstract (arXiv)

While there has been much recent work studying how linguistic information is encoded in pre-trained sentence representations, comparatively little is understood about how these models change when adapted to solve downstream tasks. Using a suite of analysis techniques (probing classifiers, Representational Similarity Analysis, and model ablations), we investigate how fine-tuning affects the representations of the BERT model. We find that while fine-tuning necessarily makes significant changes, it does not lead to catastrophic forgetting of linguistic phenomena. We instead find that fine-tuning primarily affects the top layers of BERT, but with noteworthy variation across tasks. In particular, dependency parsing reconfigures most of the model, whereas SQuAD and MNLI appear to involve much shallower processing. Finally, we also find that fine-tuning has a weaker effect on representations of out-of-domain sentences, suggesting room for improvement in model generalization.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 20; 2023: 25; 2024: 20; 2025: 18; 2026: 7.

113 papers.

Sort:

By area:

  1. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  2. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  3. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
  4. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  5. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 140
  6. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  7. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  8. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  9. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  10. 2021First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERTcited by 64
  11. 2021Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Taskscited by 46
  12. 2021A Closer Look at How Fine-tuning Changes BERTcited by 42
38 more
  1. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  2. 2021How transfer learning impacts linguistic knowledge in deep NLP models?cited by 32
  3. 2025Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuningcited by 31
  4. 2021Fine-Tuned Transformers Show Clusters of Similar Representations Across Layerscited by 29
  5. 2021CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Taskscited by 29
  6. 2022Discovering Language-neutral Sub-networks in Multilingual Language Modelscited by 28
  7. 2021DirectProbe: Studying Representations without Classifierscited by 25
  8. 2023Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Modelscited by 25
  9. 2025Narrow Finetuning Leaves Clearly Readable Traces in Activation Differencescited by 24
  10. 2021How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropycited by 23
  11. 2023Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyondcited by 20
  12. 2022Acceptability Judgements via Examining the Topology of Attention Mapscited by 19
  13. 2022On the Transformation of Latent Space in Fine-Tuned NLP Modelscited by 19
  14. 2023Investigating Forgetting in Pre-Trained Representations Through Continual Learningcited by 19
  15. 2024Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Modelscited by 18
  16. 2024ReSi: A Comprehensive Benchmark for Representational Similarity Measurescited by 18
  17. 2021Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representationscited by 16
  18. 2022On Isotropy, Contextualization and Learning Dynamics of Contrastive-based Sentence Representation Learningcited by 15
  19. 2024Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Modelscited by 15
  20. 2024Vision-Language Model Fine-Tuning via Simple Parameter-Efficient Modificationcited by 14
  21. 2022Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learningcited by 13
  22. 2024Exploring Alignment in Shared Cross-lingual Spacescited by 13
  23. 2022On the Importance of Data Size in Probing Fine-tuned Modelscited by 12
  24. 2023Identifying the Correlation Between Language Distance and Cross-Lingual Transfer in a Multilingual Representation Spacecited by 12
  25. 2021T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLPcited by 11
  26. 2022On the Effectiveness of Compact Biomedical Transformerscited by 11
  27. 2022Representation Projection Invariance Mitigates Representation Collapsecited by 10
  28. 2022Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer?cited by 10
  29. 2023Scaling up Discovery of Latent Concepts in Deep NLP Modelscited by 9
  30. 2023Vision-Language Integration in Multimodal Video Transformers (Partially) Aligns with the Braincited by 9
  31. 2025Learning to Interpret Weight Differences in Language Modelscited by 9
  32. 2023On Robustness of Finetuned Transformer-based NLP Modelscited by 8
  33. 2023On Surgical Fine-tuning for Language Encoderscited by 8
  34. 2025Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMscited by 8
  35. 2022StorSeismic: A new paradigm in deep learning for seismic processingcited by 7
  36. 2022On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learningcited by 7
  37. 2021On the Evolution of Syntactic Information Encoded by BERT's Contextualized Representationscited by 6
  38. 2023All Roads Lead to Rome? Exploring the Invariance of Transformers' Representationscited by 6

50 most cited of 113 All 113 (JSON)