# Ranking Robustness Under Adversarial Document Manipulations

arXiv: https://arxiv.org/abs/1806.04425 · HTML: https://mlsanity.com/paper/1806.04425/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2018.

## Built on by (most cited first, 15 of 15)

- [Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models](https://mlsanity.com/paper/2209.06506/) (2022)
- [Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models](https://mlsanity.com/paper/2502.01386/) (2025)
- [Towards Imperceptible Document Manipulations against Neural Ranking Models](https://mlsanity.com/paper/2305.01860/) (2023)
- [The King is Naked: on the Notion of Robustness for Natural Language Processing](https://mlsanity.com/paper/2112.07605/) (2021)
- [BERT Rankers are Brittle: a Study using Adversarial Document Perturbations](https://mlsanity.com/paper/2206.11724/) (2022)
- [Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2407.06992/) (2024)
- [Practical Relative Order Attack in Deep Ranking](https://mlsanity.com/paper/2103.05248/) (2021)
- [Adversarial Attack and Defense in Deep Ranking](https://mlsanity.com/paper/2106.03614/) (2021)
- [Isotropic Representation Can Improve Dense Retrieval](https://mlsanity.com/paper/2209.00218/) (2022)
- [Faithfulness and the Notion of Adversarial Sensitivity in NLP Explanations](https://mlsanity.com/paper/2409.17774/) (2024)
- [Provable Robust Saliency-based Explanations](https://mlsanity.com/paper/2212.14106/) (2022)
- [Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity](https://mlsanity.com/paper/2308.00177/) (2023)
- [RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Masking](https://mlsanity.com/paper/2512.23307/) (2025)
- [Robust Ranking Explanations](https://mlsanity.com/paper/2307.04024/) (2023)
- [Mechanism Design for Generative Engines: From Exploitation toward Win-Win Outcomes](https://mlsanity.com/paper/2608.11390/) (2026)

## Abstract (arXiv)

For many queries in the Web retrieval setting there is an on-going ranking competition: authors manipulate their documents so as to promote them in rankings. Such competitions can have unwarranted effects not only in terms of retrieval effectiveness, but also in terms of ranking robustness. A case in point, rankings can (rapidly) change due to small indiscernible perturbations of documents. While there has been a recent growing interest in analyzing the robustness of classifiers to adversarial manipulations, there has not yet been a study of the robustness of relevance-ranking functions. We address this challenge by formally analyzing different definitions and aspects of the robustness of learning-to-rank-based ranking functions. For example, we formally show that increased regularization of linear ranking functions increases ranking robustness. This finding leads us to conjecture that decreased variance of any ranking function results in increased robustness. We propose several measures for quantifying ranking robustness and use them to analyze ranking competitions between documents' authors. The empirical findings support our formal analysis and conjecture for both RankSVM and LambdaMART.