# Revisiting subword tokenization: A case study on affixal negation in large language models

arXiv: https://arxiv.org/abs/2404.02421 · HTML: https://mlsanity.com/paper/2404.02421/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 3 Apr 2024.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly](https://mlsanity.com/paper/1911.03343/) (2019)
- [Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://mlsanity.com/paper/2004.03720/) (2020)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [Inseq: An Interpretability Toolkit for Sequence Generation Models](https://mlsanity.com/paper/2302.13942/) (2023)
- [Language models are not naysayers: An analysis of language models on negation benchmarks](https://mlsanity.com/paper/2306.08189/) (2023)
- [An Analysis of Negation in Natural Language Understanding Corpora](https://mlsanity.com/paper/2203.08929/) (2022)
- [CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation](https://mlsanity.com/paper/2211.00295/) (2022)
- [How Much Does Tokenization Affect Neural Machine Translation?](https://mlsanity.com/paper/1812.08621/) (2018)
- [The SIGMORPHON 2022 Shared Task on Morpheme Segmentation](https://mlsanity.com/paper/2206.07615/) (2022)
- [Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation](https://mlsanity.com/paper/2210.03256/) (2022)
- [ScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context Learning](https://mlsanity.com/paper/2305.19426/) (2023)
- [The Effectiveness of Morphology-aware Segmentation in Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2103.11189/) (2021)
- [Morfessor EM+Prune: Improved Subword Segmentation with Expectation Maximization and Pruning](https://mlsanity.com/paper/2003.03131/) (2020)

## Built on by (most cited first, 2 of 2)

- [Epistemological Fault Lines Between Human and Artificial Intelligence](https://mlsanity.com/paper/2512.19466/) (2025)
- [Not all Negation Cues are Equal: Affixal Negations Yield Better Negation Understanding](https://mlsanity.com/paper/2609.13685/) (2026)

## Abstract (arXiv)

In this work, we measure the impact of affixal negation on modern English large language models (LLMs). In affixal negation, the negated meaning is expressed through a negative morpheme, which is potentially challenging for LLMs as their tokenizers are often not morphologically plausible. We conduct extensive experiments using LLMs with different subword tokenization methods, which lead to several insights on the interaction between tokenization performance and negation sensitivity. Despite some interesting mismatches between tokenization accuracy and negation detection performance, we show that models can, on the whole, reliably recognize the meaning of affixal negation.