# Fall of Giants: How popular text-based MLaaS fall against a simple evasion attack

arXiv: https://arxiv.org/abs/2104.05996 · HTML: https://mlsanity.com/paper/2104.05996/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 60 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 26 of 26)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Generating Natural Language Adversarial Examples](https://mlsanity.com/paper/1804.07998/) (2018)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018)
- [Deep Reinforcement Learning framework for Autonomous Driving](https://mlsanity.com/paper/1704.02532/) (2017)
- [Brain Tumor Segmentation with Deep Neural Networks](https://mlsanity.com/paper/1505.03540/) (2015)
- [Robust Neural Machine Translation with Doubly Adversarial Inputs](https://mlsanity.com/paper/1906.02443/) (2019)
- [Deceiving Google's Perspective API Built for Detecting Toxic Comments](https://mlsanity.com/paper/1702.08138/) (2017)
- [An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks](https://mlsanity.com/paper/2006.08131/) (2020)
- [Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples](https://mlsanity.com/paper/1803.01128/) (2018)
- [All You Need is "Love": Evading Hate-speech Detection](https://mlsanity.com/paper/1808.09115/) (2018)
- [Towards Crafting Text Adversarial Samples](https://mlsanity.com/paper/1707.02812/) (2017)
- [Addressing the Rare Word Problem in Neural Machine Translation](https://mlsanity.com/paper/1410.8206/) (2014)
- [Adversarial Texts with Gradient Methods](https://mlsanity.com/paper/1801.07175/) (2018)
- [Security Risks in Deep Learning Implementations](https://mlsanity.com/paper/1711.11008/) (2017)
- [How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection](https://mlsanity.com/paper/1902.10339/) (2019)

## Built on by (most cited first, 13 of 13)

- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Unbiased Watermark for Large Language Models](https://mlsanity.com/paper/2310.10669/) (2023)
- ["Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice](https://mlsanity.com/paper/2212.14315/) (2022)
- [The Hidden Threat in Plain Text: Attacking RAG Data Loaders](https://mlsanity.com/paper/2507.05093/) (2025)
- [When Vision Fails: Text Attacks Against ViT and OCR](https://mlsanity.com/paper/2306.07033/) (2023)
- [Your Attack Is Too DUMB: Formalizing Attacker Scenarios for Adversarial Transferability](https://mlsanity.com/paper/2306.15363/) (2023)
- [DeBackdoor: A Deductive Framework for Detecting Backdoor Attacks on Deep Models with Limited Data](https://mlsanity.com/paper/2503.21305/) (2025)
- [Adversarial News and Lost Profits: Manipulating Headlines in LLM-Driven Algorithmic Trading](https://mlsanity.com/paper/2601.13082/) (2026)
- [Privacy Implications of Explainable AI in Data-Driven Systems](https://mlsanity.com/paper/2406.15789/) (2024)
- [Captcha Attack: Turning Captchas Against Humanity](https://mlsanity.com/paper/2201.04014/) (2022)
- [Invisible Threats: Backdoor Attack in OCR Systems](https://mlsanity.com/paper/2310.08259/) (2023)
- [Unraveling Attacks in Machine Learning-based IoT Ecosystems: A Survey and the Open Libraries Behind Them](https://mlsanity.com/paper/2401.11723/) (2024)
- [Watermark Smoothing Attacks against Language Models](https://mlsanity.com/paper/2407.14206/) (2024)

## Abstract (arXiv)

The increased demand for machine learning applications made companies offer Machine-Learning-as-a-Service (MLaaS). In MLaaS (a market estimated 8000M USD by 2025), users pay for well-performing ML models without dealing with the complicated training procedure. Among MLaaS, text-based applications are the most popular ones (e.g., language translators). Given this popularity, MLaaS must provide resiliency to adversarial manipulations. For example, a wrong translation might lead to a misunderstanding between two parties. In the text domain, state-of-the-art attacks mainly focus on strategies that leverage ML models' weaknesses. Unfortunately, not much attention has been given to the other pipeline' stages, such as the indexing stage (i.e., when a sentence is converted from a textual to a numerical representation) that, if manipulated, can significantly affect the final performance of the application. In this paper, we propose a novel text evasion technique called "textitZero-Width attack" (ZeW) that leverages the injection of human non-readable characters, affecting indexing stage mechanisms. We demonstrate that our simple yet effective attack deceives MLaaS of "giants" such as Amazon, Google, IBM, and Microsoft. Our case study, based on the manipulation of hateful tweets, shows that out of 12 analyzed services, only one is resistant to our injection strategy. We finally introduce and test a simple textitinput validation defense that can prevent our proposed attack.