mlsanityData to 7 Oct 2026

Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks

arXiv 1608.04207 ↗First posted 15 Aug 2016cs.CLcited by 216

Not on the map. Why?

Abstract (arXiv)

There is a lot of research interest in encoding variable length sentences into fixed length vectors, in a way that preserves the sentence meanings. Two common methods include representations based on averaging word vectors, and representations based on the hidden states of recurrent neural networks such as LSTMs. The sentence vectors are used as features for subsequent machine learning tasks or for pre-training in the context of deep learning. However, not much is known about the properties that are encoded in these sentence representations and about the language information they capture. We propose a framework that facilitates better understanding of the encoded representations. We define prediction tasks around isolated aspects of sentence structure (namely sentence length, word content, and word order), and score representations by the ability to train a classifier to solve each prediction task when using the representation as input. We demonstrate the potential contribution of the approach by analyzing different sentence representation mechanisms. The analysis sheds light on the relative strengths of different sentence embedding methods with respect to these low level prediction tasks, and on the effect of the encoded vector's dimensionality on the resulting representations.

Key facts. Cited by 216 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 50; 2022: 49; 2023: 42; 2024: 36; 2025: 26; 2026: 13.

216 papers.

Sort:

By area:

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  3. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  4. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  5. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  6. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  7. 2022Analyzing Transformers in Embedding Spacecited by 131
  8. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  9. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  10. 2023The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reductioncited by 120
  11. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
  12. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 97
38 more
  1. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
  2. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  3. 2021Chess as a Testbed for Language Model State Trackingcited by 69
  4. 2021First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERTcited by 64
  5. 2021Low-Complexity Probing via Finding Subnetworkscited by 63
  6. 2023Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIPcited by 62
  7. 2021Conditional probing: measuring usable information beyond a baselinecited by 61
  8. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 61
  9. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 58
  10. 2022Probing for the Usage of Grammatical Numbercited by 57
  11. 2021Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?cited by 55
  12. 2023The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Modelscited by 54
  13. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 45
  14. 2021Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Explorationcited by 44
  15. 2023SkinCon: A skin disease dataset densely annotated by domain experts for fine-grained model debugging and analysiscited by 44
  16. 2022Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Taskscited by 42
  17. 2022Probing Classifiers are Unreliable for Concept Removal and Detectioncited by 40
  18. 2021Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Sensescited by 39
  19. 2024Estimating Knowledge in Large Language Models Without Generating a Single Tokencited by 39
  20. 2021Evaluating Saliency Methods for Neural Language Modelscited by 37
  21. 2022Analyzing Encoded Concepts in Transformer Language Modelscited by 37
  22. 2022Joint processing of linguistic properties in brains and language modelscited by 37
  23. 2021What do pre-trained code models know about code?cited by 36
  24. 2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34
  25. 2021How transfer learning impacts linguistic knowledge in deep NLP models?cited by 32
  26. 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
  27. 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writingcited by 31
  28. 2022textless-lib: a Library for Textless Spoken Language Processingcited by 28
  29. 2023Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignmentcited by 27
  30. 2024Holmes: A Benchmark to Assess the Linguistic Competence of Language Modelscited by 26
  31. 2021Global Explainability of BERT-Based Evaluation Metrics by Disentangling along Linguistic Factorscited by 25
  32. 2024Mechanistic?cited by 25
  33. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  34. 2024Language Models as Models of Languagecited by 24
  35. 2025Mechanistic Interpretability of Emotion Inference in Large Language Modelscited by 24
  36. 2025SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reductioncited by 24
  37. 2022Unit Testing for Concepts in Neural Networkscited by 23
  38. 2023Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Trainingcited by 23

50 most cited of 216 All 216 (JSON)