# One word at a time: adversarial attacks on retrieval models

arXiv: https://arxiv.org/abs/2008.02197 · HTML: https://mlsanity.com/paper/2008.02197/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Aug 2020.

## Built on by (most cited first, 23 of 23)

- [Poisoning Retrieval Corpora by Injecting Adversarial Passages](https://mlsanity.com/paper/2310.19156/) (2023)
- ["Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models](https://mlsanity.com/paper/2406.19417/) (2024)
- [Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models](https://mlsanity.com/paper/2209.06506/) (2022)
- [Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models](https://mlsanity.com/paper/2502.01386/) (2025)
- [Corpus Poisoning via Approximate Greedy Gradient Descent](https://mlsanity.com/paper/2406.05087/) (2024)
- [GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Search](https://mlsanity.com/paper/2412.20953/) (2024)
- [Towards Imperceptible Document Manipulations against Neural Ranking Models](https://mlsanity.com/paper/2305.01860/) (2023)
- [Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworks](https://mlsanity.com/paper/2408.05025/) (2024)
- [BERT Rankers are Brittle: a Study using Adversarial Document Perturbations](https://mlsanity.com/paper/2206.11724/) (2022)
- [Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems](https://mlsanity.com/paper/2601.07072/) (2026)
- [Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2407.06992/) (2024)
- [Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attack](https://mlsanity.com/paper/2503.21315/) (2025)
- [Reproducing HotFlip for Corpus Poisoning Attacks in Dense Retrieval](https://mlsanity.com/paper/2501.04802/) (2025)
- [Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-box Neural Ranking Models](https://mlsanity.com/paper/2412.18770/) (2024)
- [Subverting Fair Image Search with Generative Adversarial Perturbations](https://mlsanity.com/paper/2205.02414/) (2022)
- [Towards Effective Paraphrasing for Information Disguise](https://mlsanity.com/paper/2311.05018/) (2023)
- [RankSHAP: Shapley Value Based Feature Attributions for Learning to Rank](https://mlsanity.com/paper/2405.01848/) (2024)
- [Adversarial Attacks against Neural Ranking Models via In-Context Learning](https://mlsanity.com/paper/2508.15283/) (2025)
- [Defense of Adversarial Ranking Attack in Text Retrieval: Benchmark and Baseline via Detection](https://mlsanity.com/paper/2307.16816/) (2023)
- [The Ranking Blind Spot: Decision Hijacking in LLM-based Text Ranking](https://mlsanity.com/paper/2509.18575/) (2025)
- [CSP: A Simulator For Multi-Agent Ranking Competitions](https://mlsanity.com/paper/2502.11197/) (2025)
- [RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Masking](https://mlsanity.com/paper/2512.23307/) (2025)
- [TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness](https://mlsanity.com/paper/2512.01183/) (2025)

## Abstract (arXiv)

Adversarial examples, generated by applying small perturbations to input features, are widely used to fool classifiers and measure their robustness to noisy inputs. However, little work has been done to evaluate the robustness of ranking models through adversarial examples. In this work, we present a systematic approach of leveraging adversarial examples to measure the robustness of popular ranking models. We explore a simple method to generate adversarial examples that forces a ranker to incorrectly rank the documents. Using this approach, we analyze the robustness of various ranking models and the quality of perturbations generated by the adversarial attacker across two datasets. Our findings suggest that with very few token changes (1-3), the attacker can yield semantically similar perturbed documents that can fool different rankers into changing a document's score, lowering its rank by several positions.