mlsanityData to 13 Oct 2026

Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexica

arXiv 2109.02738 ↗First posted 6 Sep 2021cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

People convey their intention and attitude through linguistic styles of the text that they write. In this study, we investigate lexicon usages across styles throughout two lenses: human perception and machine word importance, since words differ in the strength of the stylistic cues that they provide. To collect labels of human perception, we curate a new dataset, Hummingbird, on top of benchmarking style datasets. We have crowd workers highlight the representative words in the text that makes them think the text has the following styles: politeness, sentiment, offensiveness, and five emotion types. We then compare these human word labels with word importance derived from a popular fine-tuned style classifier like BERT. Our results show that the BERT often finds content words not relevant to the target style as important words used in style prediction, but humans do not perceive the same way even though for some styles (e.g., positive sentiment and joy) human- and machine-identified words share significant overlap for some styles.

Key facts. Cited by 23 papers in the map (data through 13 Oct 2026). Builds on 9 of its 20 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

9 of 20 references.

References · 9

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,368
  4. 2020A Primer in BERTology: What we know about how BERT workscited by 809
  5. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 731
  6. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 546
  7. 2013A Computational Approach to Politeness with Application to Social Factorscited by 106
  8. 2018Did the Model Understand the Question?cited by 60
  9. 2021SelfExplain: A Self-Explaining Architecture for Neural Text Classifierscited by 44

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 5; 2023: 6; 2024: 6; 2025: 5; 2026: 1.

23 papers.

Sort:

By area:

  1. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 148
  2. 2023Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmarkcited by 62
  3. 2023The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Taskscited by 35
  4. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  5. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Datacited by 23
  6. 2023How Far Can We Extract Diverse Perspectives from Large Language Models?cited by 22
  7. 2022A Fine-grained Interpretability Evaluation Benchmark for Neural NLPcited by 15
  8. 2023Through the Lens of Core Competency: Survey on Evaluation of Large Language Modelscited by 12
  9. 2024SoMeLVLM: A Large Vision Language Model for Social Media Processingcited by 11
  10. 2024Building Knowledge-Guided Lexica to Model Cultural Variationcited by 11
  11. 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLPcited by 9
  12. 2024SOCIALITE-LLAMA: An Instruction-Tuned Model for Social Scientific Taskscited by 7
11 more
  1. 2022An Attention Matrix for Every Decision: Faithfulness-based Arbitration Among Multiple Attention-Based Interpretations of Transformers in Text Classificationcited by 6
  2. 2022StyLEx: Explaining Style Using Human Lexical Annotationscited by 3
  3. 2024Investigating the Impact of Model Instability on Explanations and Uncertaintycited by 3
  4. 2022A Comparative Study on Textual Saliency of Styles from Eye Tracking, Annotations, and Language Modelscited by 2
  5. 2023Comparing Styles across Languages: A Cross-Cultural Exploration of Politenesscited by 2
  6. 2025Learning Explainable Dense Reward Shapes via Bayesian Optimizationcited by 2
  7. 2025Main Predicate and Their Arguments as Explanation Signals For Intent Classificationcited by 1
  8. 2025Capturing Human Cognitive Styles with Language: Towards an Experimental Evaluation Paradigmcited by 1
  9. 2022RedPen: Region- and Reason-Annotated Dataset of Unnatural Speechcited by 0
  10. 2025Splits! Flexible Sociocultural Linguistic Investigation at Scalecited by 0
  11. 2026Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understandingcited by 0