# PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Models

arXiv: https://arxiv.org/abs/2109.06777 · HTML: https://mlsanity.com/paper/2109.06777/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 14 Sep 2021.

## Builds on (most cited first, 16 of 16)

- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [Combating Adversarial Misspellings with Robust Word Recognition](https://mlsanity.com/paper/1905.11268/) (2019)
- [Hierarchical Transformers for Multi-Document Summarization](https://mlsanity.com/paper/1905.13164/) (2019)
- [Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks](https://mlsanity.com/paper/1603.03827/) (2016)
- [MALCOM: Generating Malicious Comments to Attack Neural Fake News Detection Models](https://mlsanity.com/paper/2009.01048/) (2020)
- [An Army of Me: Sockpuppets in Online Discussion Communities](https://mlsanity.com/paper/1703.07355/) (2017)
- [Robust Spammer Detection by Nash Reinforcement Learning](https://mlsanity.com/paper/2006.06069/) (2020)
- [VEWS: A Wikipedia Vandal Early Warning System](https://mlsanity.com/paper/1507.01272/) (2015)
- [TIES: Temporal Interaction Embeddings For Enhancing Social Media Integrity At Facebook](https://mlsanity.com/paper/2002.07917/) (2020)

## Built on by (most cited first, 17 of 17)

- [Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks](https://mlsanity.com/paper/2310.10830/) (2023)
- [Reinforcement Learning-based Counter-Misinformation Response Generation: A Case Study of COVID-19 Vaccine Misinformation](https://mlsanity.com/paper/2303.06433/) (2023)
- [Rank List Sensitivity of Recommender Systems to Interaction Perturbations](https://mlsanity.com/paper/2201.12686/) (2022)
- [Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomes](https://mlsanity.com/paper/2404.04392/) (2024)
- [SAGE-RT: Synthetic Alignment data Generation for Safety Evaluation and Red Teaming](https://mlsanity.com/paper/2408.11851/) (2024)
- [ID-Free Not Risk-Free: LLM-Powered Agents Unveil Risks in ID-Free Recommender Systems](https://mlsanity.com/paper/2409.11690/) (2024)
- [Shilling Black-box Review-based Recommender Systems through Fake Review Generation](https://mlsanity.com/paper/2306.16526/) (2023)
- [Robust Fake News Detection using Large Language Models under Adversarial Sentiment Attacks](https://mlsanity.com/paper/2601.15277/) (2026)
- [COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation](https://mlsanity.com/paper/2210.15500/) (2022)
- [Robustness of Fusion-based Multimodal Classifiers to Cross-Modal Content Dilutions](https://mlsanity.com/paper/2211.02646/) (2022)
- [Adversarial machine learning for protecting against online manipulation](https://mlsanity.com/paper/2111.12034/) (2021)
- [Auditing and Robustifying COVID-19 Misinformation Datasets via Anticontent Sampling](https://mlsanity.com/paper/2310.07078/) (2023)
- [Hierarchical Query Classification in E-commerce Search](https://mlsanity.com/paper/2403.06021/) (2024)
- [A Self-Paced Mixed Distillation Method for Non-Autoregressive Generation](https://mlsanity.com/paper/2205.11162/) (2022)
- [Adversarial Text Rewriting for Text-aware Recommender Systems](https://mlsanity.com/paper/2408.00312/) (2024)
- [Navigating the Black Box: Leveraging LLMs for Effective Text-Level Graph Injection Attacks](https://mlsanity.com/paper/2506.13276/) (2025)
- [ROBAD: Robust Adversary-aware Local-Global Attended Bad Actor Detection Sequential Model](https://mlsanity.com/paper/2507.15067/) (2025)

## Abstract (arXiv)

What should a malicious user write next to fool a detection model? Identifying malicious users is critical to ensure the safety and integrity of internet platforms. Several deep learning-based detection models have been created. However, malicious users can evade deep detection models by manipulating their behavior, rendering these models of little use. The vulnerability of such deep detection models against adversarial attacks is unknown. Here we create a novel adversarial attack model against deep user sequence embedding based classification models, which use the sequence of user posts to generate user embeddings and detect malicious users. In the attack, the adversary generates a new post to fool the classifier. We propose a novel end-to-end Personalized Text Generation Attack model, called PETGEN, that simultaneously reduces the efficacy of the detection model and generates posts that have several key desirable properties. Specifically, PETGEN generates posts that are personalized to the user's writing style, have knowledge about a given target context, are aware of the user's historical posts on the target context, and encapsulate the user's recent topical interests. We conduct extensive experiments on two real-world datasets (Yelp and Wikipedia, both with ground-truth of malicious users) to show that PETGEN significantly reduces the performance of popular deep user sequence embedding-based classification models. PETGEN outperforms five attack baselines in terms of text quality and attack efficacy in both white-box and black-box classifier settings. Overall, this work paves the path towards the next generation of adversary-aware sequence classification models.