# How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representations

arXiv: https://arxiv.org/abs/1909.04925 · HTML: https://mlsanity.com/paper/1909.04925/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2019.

## Built on by (most cited first, 30 of 63)

- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [LLaMA Pro: Progressive LLaMA with Block Expansion](https://mlsanity.com/paper/2401.02415/) (2024)
- [StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts](https://mlsanity.com/paper/2204.08292/) (2022)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration](https://mlsanity.com/paper/2102.04110/) (2021)
- [BERTnesia: Investigating the capture and forgetting of knowledge in BERT](https://mlsanity.com/paper/2106.02902/) (2021)
- [The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers](https://mlsanity.com/paper/2305.01628/) (2023)
- [FedAdapter: Efficient Federated Learning for Modern NLP](https://mlsanity.com/paper/2205.10162/) (2022)
- [Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layers](https://mlsanity.com/paper/2409.03621/) (2024)
- [Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processing](https://mlsanity.com/paper/2103.02800/) (2021)
- [Contrastive Domain Adaptation for Question Answering using Limited Text Corpora](https://mlsanity.com/paper/2108.13854/) (2021)
- [Examining Forgetting in Continual Pre-training of Aligned Large Language Models](https://mlsanity.com/paper/2401.03129/) (2024)
- [Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models](https://mlsanity.com/paper/2402.19465/) (2024)
- [Attention-based Clinical Note Summarization](https://mlsanity.com/paper/2104.08942/) (2021)
- [Federated Few-Shot Learning for Mobile NLP](https://mlsanity.com/paper/2212.05974/) (2022)
- [MAS-SAM: Segment Any Marine Animal with Aggregated Features](https://mlsanity.com/paper/2404.15700/) (2024)
- [An Automatic and Efficient BERT Pruning for Edge AI Systems](https://mlsanity.com/paper/2206.10461/) (2022)
- [A context-aware knowledge transferring strategy for CTC-based ASR](https://mlsanity.com/paper/2210.06244/) (2022)
- [LLM Safety From Within: Detecting Harmful Content with Internal Representations](https://mlsanity.com/paper/2604.18519/) (2026)
- [Transformer Block Coupling and its Correlation with Generalization in LLMs](https://mlsanity.com/paper/2407.07810/) (2024)
- [EigenTrack: Spectral Activation Feature Tracking for Hallucination and Out-of-Distribution Detection in LLMs and VLMs](https://mlsanity.com/paper/2509.15735/) (2025)
- [Modeling Context in Answer Sentence Selection Systems on a Latency Budget](https://mlsanity.com/paper/2101.12093/) (2021)
- [What's in your Head? Emergent Behaviour in Multi-Task Transformer Models](https://mlsanity.com/paper/2104.06129/) (2021)
- [QRelScore: Better Evaluating Generated Questions with Deeper Understanding of Context-aware Relevance](https://mlsanity.com/paper/2204.13921/) (2022)
- [Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities](https://mlsanity.com/paper/2212.10017/) (2022)
- [Unifying Two-Stream Encoders with Transformers for Cross-Modal Retrieval](https://mlsanity.com/paper/2308.04343/) (2023)
- [C2A: Client-Customized Adaptation for Parameter-Efficient Federated Learning](https://mlsanity.com/paper/2411.00311/) (2024)
- [Interactive Question Answering Systems: Literature Review](https://mlsanity.com/paper/2209.01621/) (2022)
- [Domain Adaptation for Question Answering via Question Classification](https://mlsanity.com/paper/2209.04998/) (2022)
- [LYNX: Learning Dynamic Exits for Confidence-Controlled Reasoning](https://mlsanity.com/paper/2512.05325/) (2025)

## Abstract (arXiv)

Bidirectional Encoder Representations from Transformers (BERT) reach state-of-the-art results in a variety of Natural Language Processing tasks. However, understanding of their internal functioning is still insufficient and unsatisfactory. In order to better understand BERT and other Transformer-based models, we present a layer-wise analysis of BERT's hidden states. Unlike previous research, which mainly focuses on explaining Transformer models by their attention weights, we argue that hidden states contain equally valuable information. Specifically, our analysis focuses on models fine-tuned on the task of Question Answering (QA) as an example of a complex downstream task. We inspect how QA models transform token vectors in order to find the correct answer. To this end, we apply a set of general and QA-specific probing tasks that reveal the information stored in each representation layer. Our qualitative analysis of hidden state visualizations provides additional insights into BERT's reasoning process. Our results show that the transformations within BERT go through phases that are related to traditional pipeline tasks. The system can therefore implicitly incorporate task-specific information into its token representations. Furthermore, our analysis reveals that fine-tuning has little impact on the models' semantic abilities and that prediction errors can be recognized in the vector representations of even early layers.