Adversarial Semantic Collisions
Not on the map. Why?
Abstract (arXiv)
We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and demonstrate that state-of-the-art models for many tasks which rely on analyzing the meaning and similarity of texts-- including paraphrase identification, document retrieval, response suggestion, and extractive summarization-- are vulnerable to semantic collisions. For example, given a target query, inserting a crafted collision into an irrelevant document can shift its retrieval rank from 1000 to top 3. We show how to generate semantic collisions that evade perplexity-based filtering and discuss other potential mitigations. Our code is available at https://github.com/csong27/collision-bert.
Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Nov 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 5; 2023: 11; 2024: 10; 2025: 6; 2026: 7.
42 papers.
Sort:
By area:
- 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
- 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
- 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
- 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMscited by 124
- 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 102
- 2021Hidden Backdoors in Human-Centric Language Modelscited by 90
- 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
- 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
- 2024"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Modelscited by 60
- 2022Describing Differences between Text Distributions with Natural Languagecited by 54
- 2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Modelscited by 37
- 2023Mass-Producing Failures of Multimodal Systems with Language Modelscited by 36
30 more
- 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Modelscited by 33
- 2023Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Methodcited by 29
- 2023Adversarial Illusions in Multi-Modal Embeddingscited by 29
- 2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Modelscited by 25
- 2024Corpus Poisoning via Approximate Greedy Gradient Descentcited by 21
- 2023A Survey on Out-of-Distribution Evaluation of Neural NLP Modelscited by 20
- 2024GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Searchcited by 20
- 2023Towards Imperceptible Document Manipulations against Neural Ranking Modelscited by 18
- 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworkscited by 17
- 2021Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluationcited by 16
- 2024Adversarial Decoding: Generating Readable Documents for Adversarial Objectivescited by 16
- 2026Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systemscited by 16
- 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
- 2024Multi-granular Adversarial Attacks against Black-box Neural Ranking Modelscited by 14
- 2022Paraphrase Identification with Deep Learning: A Review of Datasets and Methodscited by 13
- 2024Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 11
- 2025Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrievalcited by 11
- 2025Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attackcited by 10
- 2023MAWSEO: Adversarial Wiki Search Poisoning for Illicit Online Promotioncited by 7
- 2025Reproducing HotFlip for Corpus Poisoning Attacks in Dense Retrievalcited by 7
- 2022Redwood: Using Collision Detection to Grow a Large-Scale Intent Classification Datasetcited by 6
- 2024Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-box Neural Ranking Modelscited by 6
- 2026ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Modelscited by 5
- 2023Defense of Adversarial Ranking Attack in Text Retrieval: Benchmark and Baseline via Detectioncited by 3
- 2025RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Maskingcited by 1
- 2026SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoningcited by 1
- 2026Statistically Undetectable Backdoors in Deep Neural Networkscited by 1
- 2026TriShieldRAG: 3 Rings, One Blind Spot in Layered Defenses for Retrieval-Augmented Generationcited by 0
- 2026Coverage Is Not Containment: A Fundamental Limit of Admission-Time Defenses Against Coordinated Poisoning of Vector Retrievalcited by 0
- 2026VerTox: Verifiable Reward-Guided Corpus Poisoning Against Neural Ranking Modelscited by 0
- 2026VerTox: Verifiable Reward-Guided Corpus Poisoning Against Neural Ranking Models1 Sep 2026
- 2026Coverage Is Not Containment: A Fundamental Limit of Admission-Time Defenses Against Coordinated Poisoning of Vector Retrieval17 Aug 2026
- 2026TriShieldRAG: 3 Rings, One Blind Spot in Layered Defenses for Retrieval-Augmented Generation26 Jul 2026
- 2026Statistically Undetectable Backdoors in Deep Neural Networks10 Jul 2026
- 2026SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoning27 May 2026
- 2026ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models17 Feb 2026
- 2026Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems11 Jan 2026
- 2025RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Masking29 Dec 2025
- 2025Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrieval24 Apr 2025
- 2025Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attack27 Mar 2025
- 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models3 Feb 2025
- 2025Reproducing HotFlip for Corpus Poisoning Attacks in Dense Retrieval8 Jan 2025
30 more
- 2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Models6 Jan 2025
- 2024GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Search30 Dec 2024
- 2024Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-box Neural Ranking Models25 Dec 2024
- 2024Adversarial Decoding: Generating Readable Documents for Adversarial Objectives3 Oct 2024
- 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworks9 Aug 2024
- 2024Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Models18 Jul 2024
- 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective9 Jul 2024
- 2024"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models26 Jun 2024
- 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents9 Jun 2024
- 2024Corpus Poisoning via Approximate Greedy Gradient Descent7 Jun 2024
- 2024Multi-granular Adversarial Attacks against Black-box Neural Ranking Models2 Apr 2024
- 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov 2023
- 2023Poisoning Retrieval Corpora by Injecting Adversarial Passages29 Oct 2023
- 2023Text Embeddings Reveal (Almost) As Much As Text10 Oct 2023
- 2023Adversarial Illusions in Multi-Modal Embeddings22 Aug 2023
- 2023Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Method19 Aug 2023
- 2023Defense of Adversarial Ranking Attack in Text Retrieval: Benchmark and Baseline via Detection31 Jul 2023
- 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs19 Jul 2023
- 2023A Survey on Out-of-Distribution Evaluation of Neural NLP Models27 Jun 2023
- 2023Mass-Producing Failures of Multimodal Systems with Language Models21 Jun 2023
- 2023Towards Imperceptible Document Manipulations against Neural Ranking Models3 May 2023
- 2023MAWSEO: Adversarial Wiki Search Poisoning for Illicit Online Promotion22 Apr 2023
- 2022Paraphrase Identification with Deep Learning: A Review of Datasets and Methods13 Dec 2022
- 2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models14 Sep 2022
- 2022Gradient-Based Constrained Sampling from Language Models25 May 2022
- 2022Redwood: Using Collision Detection to Grow a Large-Scale Intent Classification Dataset12 Apr 2022
- 2022Describing Differences between Text Distributions with Natural Language28 Jan 2022
- 2021Controlled Text Generation as Continuous Optimization with Multiple Constraints4 Aug 2021
- 2021Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation10 Jun 2021
- 2021Hidden Backdoors in Human-Centric Language Models1 May 2021
- 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 102
- 2024"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Modelscited by 60
- 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Modelscited by 33
- 2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Modelscited by 25
- 2024Corpus Poisoning via Approximate Greedy Gradient Descentcited by 21
- 2024GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Searchcited by 20
- 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworkscited by 17
- 2024Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 11
- 2025Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attackcited by 10
- 2025RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Maskingcited by 1
- 2026SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoningcited by 1
- 2026TriShieldRAG: 3 Rings, One Blind Spot in Layered Defenses for Retrieval-Augmented Generationcited by 0
12 most cited of 14 All 14 (JSON)
- 2026VerTox: Verifiable Reward-Guided Corpus Poisoning Against Neural Ranking Models1 Sep 2026
- 2026Coverage Is Not Containment: A Fundamental Limit of Admission-Time Defenses Against Coordinated Poisoning of Vector Retrieval17 Aug 2026
- 2026TriShieldRAG: 3 Rings, One Blind Spot in Layered Defenses for Retrieval-Augmented Generation26 Jul 2026
- 2026SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoning27 May 2026
- 2025RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Masking29 Dec 2025
- 2025Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attack27 Mar 2025
- 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models3 Feb 2025
- 2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Models6 Jan 2025
- 2024GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Search30 Dec 2024
- 2024Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworks9 Aug 2024
- 2024Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Models18 Jul 2024
- 2024"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models26 Jun 2024
12 newest of 14 All 14 (JSON)
- 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
- 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
- 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
- 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
- 2022Describing Differences between Text Distributions with Natural Languagecited by 54
- 2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Modelscited by 37
- 2023Mass-Producing Failures of Multimodal Systems with Language Modelscited by 36
- 2023Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Methodcited by 29
- 2023Adversarial Illusions in Multi-Modal Embeddingscited by 29
- 2023A Survey on Out-of-Distribution Evaluation of Neural NLP Modelscited by 20
- 2023Towards Imperceptible Document Manipulations against Neural Ranking Modelscited by 18
- 2021Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluationcited by 16
12 most cited of 22 All 22 (JSON)
- 2026Statistically Undetectable Backdoors in Deep Neural Networks10 Jul 2026
- 2026ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models17 Feb 2026
- 2025Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrieval24 Apr 2025
- 2025Reproducing HotFlip for Corpus Poisoning Attacks in Dense Retrieval8 Jan 2025
- 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective9 Jul 2024
- 2024Multi-granular Adversarial Attacks against Black-box Neural Ranking Models2 Apr 2024
- 2023Poisoning Retrieval Corpora by Injecting Adversarial Passages29 Oct 2023
- 2023Text Embeddings Reveal (Almost) As Much As Text10 Oct 2023
- 2023Adversarial Illusions in Multi-Modal Embeddings22 Aug 2023
- 2023Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Method19 Aug 2023
- 2023Defense of Adversarial Ranking Attack in Text Retrieval: Benchmark and Baseline via Detection31 Jul 2023
- 2023A Survey on Out-of-Distribution Evaluation of Neural NLP Models27 Jun 2023
12 newest of 22 All 22 (JSON)