mlsanityData to 14 Oct 2026

Revisiting subword tokenization: A case study on affixal negation in large language models

arXiv 2404.02421 ↗First posted 3 Apr 2024cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

In this work, we measure the impact of affixal negation on modern English large language models (LLMs). In affixal negation, the negated meaning is expressed through a negative morpheme, which is potentially challenging for LLMs as their tokenizers are often not morphologically plausible. We conduct extensive experiments using LLMs with different subword tokenization methods, which lead to several insights on the interaction between tokenization performance and negation sensitivity. Despite some interesting mismatches between tokenization accuracy and negation detection performance, we show that models can, on the whole, reliably recognize the meaning of affixal negation.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

23 of 41 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  8. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  9. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  10. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
  11. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  12. 2020Byte Pair Encoding is Suboptimal for Language Model Pretrainingcited by 192
11 more
  1. 2023Linear Representations of Sentiment in Large Language Modelscited by 163
  2. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  3. 2023Language models are not naysayers: An analysis of language models on negation benchmarkscited by 74
  4. 2022An Analysis of Negation in Natural Language Understanding Corporacited by 39
  5. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 39
  6. 2018How Much Does Tokenization Affect Neural Machine Translation?cited by 26
  7. 2022The SIGMORPHON 2022 Shared Task on Morpheme Segmentationcited by 25
  8. 2022Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negationcited by 25
  9. 2023ScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context Learningcited by 18
  10. 2021The Effectiveness of Morphology-aware Segmentation in Low-Resource Neural Machine Translationcited by 16
  11. 2020Morfessor EM+Prune: Improved Subword Segmentation with Expectation Maximization and Pruningcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Epistemological Fault Lines Between Human and Artificial Intelligencecited by 11
  2. 2026Not all Negation Cues are Equal: Affixal Negations Yield Better Negation Understandingcited by 0