# BERT Rankers are Brittle: a Study using Adversarial Document Perturbations

arXiv: https://arxiv.org/abs/2206.11724 · HTML: https://mlsanity.com/paper/2206.11724/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 18 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 9 of 9)

- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [Adversarial Training Methods for Semi-Supervised Text Classification](https://mlsanity.com/paper/1605.07725/) (2016)
- [Contextualized Perturbation for Textual Adversarial Attack](https://mlsanity.com/paper/2009.07502/) (2020)
- [One word at a time: adversarial attacks on retrieval models](https://mlsanity.com/paper/2008.02197/) (2020)
- [Ranking Robustness Under Adversarial Document Manipulations](https://mlsanity.com/paper/1806.04425/) (2018)
- [Adversarial Attacks and Defense on Texts: A Survey](https://mlsanity.com/paper/2005.14108/) (2020)

## Built on by (most cited first, 16 of 16)

- [Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Method](https://mlsanity.com/paper/2308.09861/) (2023)
- [Towards Imperceptible Document Manipulations against Neural Ranking Models](https://mlsanity.com/paper/2305.01860/) (2023)
- [Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworks](https://mlsanity.com/paper/2408.05025/) (2024)
- [Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence](https://mlsanity.com/paper/2503.05037/) (2025)
- [Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems](https://mlsanity.com/paper/2601.07072/) (2026)
- [Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2407.06992/) (2024)
- [Multi-granular Adversarial Attacks against Black-box Neural Ranking Models](https://mlsanity.com/paper/2404.01574/) (2024)
- [A Study on FGSM Adversarial Training for Neural Retrieval](https://mlsanity.com/paper/2301.10576/) (2023)
- [Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-box Neural Ranking Models](https://mlsanity.com/paper/2412.18770/) (2024)
- [Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain](https://mlsanity.com/paper/2509.03787/) (2025)
- [Towards Effective Paraphrasing for Information Disguise](https://mlsanity.com/paper/2311.05018/) (2023)
- [Adversarial Attacks against Neural Ranking Models via In-Context Learning](https://mlsanity.com/paper/2508.15283/) (2025)
- [Defense of Adversarial Ranking Attack in Text Retrieval: Benchmark and Baseline via Detection](https://mlsanity.com/paper/2307.16816/) (2023)
- [One Word is Enough: Minimal Adversarial Perturbations for Neural Text Ranking](https://mlsanity.com/paper/2601.20283/) (2026)
- [Led to Mislead: Adversarial Content Injection for Attacks on Neural Ranking Models](https://mlsanity.com/paper/2605.01591/) (2026)
- [VerTox: Verifiable Reward-Guided Corpus Poisoning Against Neural Ranking Models](https://mlsanity.com/paper/2609.01325/) (2026)

## Abstract (arXiv)

Contextual ranking models based on BERT are now well established for a wide range of passage and document ranking tasks. However, the robustness of BERT-based ranking models under adversarial inputs is under-explored. In this paper, we argue that BERT-rankers are not immune to adversarial attacks targeting retrieved documents given a query. Firstly, we propose algorithms for adversarial perturbation of both highly relevant and non-relevant documents using gradient-based optimization methods. The aim of our algorithms is to add/replace a small number of tokens to a highly relevant or non-relevant document to cause a large rank demotion or promotion. Our experiments show that a small number of tokens can already result in a large change in the rank of a document. Moreover, we find that BERT-rankers heavily rely on the document start/head for relevance prediction, making the initial part of the document more susceptible to adversarial attacks. More interestingly, we find a small set of recurring adversarial words that when added to documents result in successful rank demotion/promotion of any relevant/non-relevant document respectively. Finally, our adversarial tokens also show particular topic preferences within and across datasets, exposing potential biases from BERT pre-training or downstream datasets.