mlsanityData to 14 Oct 2026

Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness

arXiv 2503.01345 ↗First posted 3 Mar 2025cs.CL cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Insensitivity to semantically-preserving variations of prompts (paraphrases) is crucial for reliable behavior and real-world deployment of large language models. However, language models exhibit significant performance degradation when faced with semantically equivalent but differently phrased prompts, and existing solutions either depend on trial-and-error prompt engineering or require computationally expensive inference-time algorithms. In this study, built on the key insight that worst-case prompts exhibit a drift in embedding space, we present Latent Adversarial Paraphrasing (LAP), a dual-loop adversarial framework: the inner loop trains a learnable perturbation to serve as a "latent continuous paraphrase" while preserving semantics through Lagrangian regulation, and the outer loop optimizes the language model parameters on these perturbations. We conduct extensive experiments to demonstrate the effectiveness of LAP across multiple LLM architectures on the RobustAlpaca benchmark with a 0.5%-4% absolution improvement on worst-case win-rate compared with vanilla supervised fine-tuning.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 49 of its 61 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

49 of 61 references.

References · 49

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  9. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  10. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  11. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  12. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
37 more
  1. 2023Large Language Models as Optimizerscited by 864
  2. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  3. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 438
  4. 2023Llemma: An Open Language Model For Mathematicscited by 388
  5. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  6. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 377
  7. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 376
  8. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 310
  9. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 310
  10. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 229
  11. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 216
  12. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 173
  13. 2023Platypus: Quick, Cheap, and Powerful Refinement of LLMscited by 150
  14. 2024Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewardscited by 148
  15. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  16. 2023Black-Box Prompt Optimization: Aligning Large Language Models without Model Trainingcited by 122
  17. 2024Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 115
  18. 2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 113
  19. 2024The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performancecited by 100
  20. 2024Attacking Large Language Models with Projected Gradient Descentcited by 94
  21. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 88
  22. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 83
  23. 2024Efficient multi-prompt evaluation of LLMscited by 73
  24. 2022Prompt Consistency for Zero-Shot Task Generalizationcited by 67
  25. 2023Evaluating the Zero-shot Robustness of Instruction-tuned Language Modelscited by 66
  26. 2024BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Modelscited by 58
  27. 2025Open Problems in Machine Unlearning for AI Safetycited by 51
  28. 2022Toward Human Readable Prompt Tuning: Kubrick's The Shining is a good movie, and a good prompt too?cited by 49
  29. 2023Adversarial Attacks and Defenses in Large Language Models: Old and New Threatscited by 49
  30. 2024On the Worst Prompt Performance of Large Language Modelscited by 38
  31. 2024Are Large Language Models Good Prompt Optimizers?cited by 37
  32. 2022Robustness of Learning from Task Instructionscited by 35
  33. 2023Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learningcited by 31
  34. 2023Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignmentcited by 30
  35. 2017Regularizing deep networks using efficient layerwise adversarial trainingcited by 24
  36. 2023Scaling Laws for Adversarial Attacks on Language Model Activationscited by 19
  37. 2024Monotonic Paraphrasing Improves Generalization of Language Model Promptingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sinkcited by 4
  2. 2025Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMscited by 3
  3. 2026Evaluating Prompt Robustness in Text-to-Audio Systems for Adaptive Virtual Agents and Game Soundtrackscited by 0
  4. 2026Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignmentcited by 0
  5. 2026Hallucination-R1: Robustness-Oriented Paraphrase Generation for Factual Consistencycited by 0