mlsanityData to 7 Oct 2026

Language Model Inversion

arXiv 2311.13647 ↗First posted 22 Nov 2023cs.CL cs.LGcited by 66

Not on the map. Why?

Abstract (arXiv)

Language models produce a distribution over the next token; can we use this information to recover the prompt tokens? We consider the problem of language model inversion and show that next-token probabilities contain a surprising amount of information about the preceding text. Often we can recover the text in cases where it is hidden from the user, motivating a method for recovering unknown prompts given only the model's current distribution output. We consider a variety of model access scenarios, and show how even without predictions for every token in the vocabulary we can recover the probability vector through search. On Llama-2 7b, our inversion method reconstructs prompts with a BLEU of 59 and token-level F1 of 78 and recovers 27% of prompts exactly. Code for reproducing all experiments is available at http://github.com/jxmorris12/vec2text.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 23 of its 44 references that are in the map. First posted on arXiv 22 Nov 2023.

Builds on

23 of 44 references.

References · 23

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2016Pointer Sentinel Mixture Modelscited by 2,992
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  7. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  8. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  9. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  10. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  11. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  12. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
11 more
  1. 2020Information Leakage in Embedding Modelscited by 199
  2. 2015Inverting Visual Representations with Convolutional Networkscited by 168
  3. 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 136
  4. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  5. 2023Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentencecited by 81
  6. 2020Imitation Attacks and Defenses for Black-box Machine Translation Systemscited by 78
  7. 2021High Fidelity Visualization of What Your Self-Supervised Representation Knows Aboutcited by 64
  8. 2022Text Revealer: Private Text Reconstruction via Model Inversion Attacks against Transformerscited by 28
  9. 2019A framework for the extraction of Deep Neural Networks by leveraging public datacited by 22
  10. 2021Understanding Invariance via Feedforward Inversion of Discriminatively Trained Classifierscited by 11
  11. 2023Breaching FedMD: Image Recovery via Paired-Logits Inversion Attackcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 34; 2026: 17.

66 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  3. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  4. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  5. 2024Query-Based Adversarial Prompt Generationcited by 55
  6. 2025Harnessing the Universal Geometry of Embeddingscited by 54
  7. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 46
  8. 2024Extracting Prompts by Inverting LLM Outputscited by 40
  9. 2024Logits of API-Protected LLMs Leak Proprietary Informationcited by 38
  10. 2024Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defensescited by 30
  11. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30
  12. 2024Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Modelscited by 28
38 more
  1. 2024Privacy Preserving Prompt Engineering: A Surveycited by 24
  2. 2024LLM-based Multi-Agent Systems: Techniques and Business Perspectivescited by 22
  3. 2025Privacy Auditing of Large Language Modelscited by 22
  4. 2025Eliciting Language Model Behaviors with Investigator Agentscited by 21
  5. 2024PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMscited by 17
  6. 2026RAGEN-2: Reasoning Collapse in Agentic RLcited by 16
  7. 2025Depth Gives a False Sense of Privacy: LLM Internal States Inversioncited by 14
  8. 2024Black-Box Adversarial Attacks on LLM-Based Code Completioncited by 13
  9. 2025Single-pass Detection of Jailbreaking Input in Large Language Modelscited by 10
  10. 2024PRSA: Prompt Stealing Attacks against Real-World Prompt Servicescited by 9
  11. 2025Better Language Model Inversion by Compactly Representing Next-Token Distributionscited by 9
  12. 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8
  13. 2025Conditional Data Synthesis Augmentationcited by 7
  14. 2025Diffusion LLMs are Natural Adversaries for any LLMcited by 7
  15. 2025Privacy Risks of LLM-Empowered Recommender Systems: An Inversion Attack Perspectivecited by 6
  16. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  17. 2025Privacy-Preserving LLM Embedding Transmission for End-Cloud Collaborationcited by 5
  18. 2025DP-Fusion: Token-Level Differentially Private Inference for Large Language Modelscited by 5
  19. 2025An Investigation of Memorization Risk in Healthcare Foundation Modelscited by 5
  20. 2025What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requestscited by 4
  21. 2025You Can't Steal Nothing: Mitigating Prompt Leakages in LLMs via System Vectorscited by 4
  22. 2025Has My System Prompt Been Used? Large Language Model Prompt Membership Inferencecited by 3
  23. 2025Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Promptscited by 3
  24. 2025Every Language Model Has a Forgery-Resistant Signaturecited by 3
  25. 2024The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcarecited by 2
  26. 2025Initial Findings on Sensor based Open Vocabulary Activity Recognition via Text Embedding Inversioncited by 2
  27. 2025IPAD: Inverse Prompt for AI Detection - A Robust and Interpretable LLM-Generated Text Detectorcited by 2
  28. 2025StyleRec: A Benchmark Dataset for Prompt Recovery in Writing Style Transformationcited by 2
  29. 2025Robustness in Both Domains: CLIP Needs a Robust Text Encodercited by 2
  30. 2025GPT, But Backwards: Exactly Inverting Language Model Outputscited by 2
  31. 2025Model Inversion Attacks on Llama 3: Extracting PII from Large Language Modelscited by 2
  32. 2025LogitTrace: Detecting Benchmark Contamination via Layerwise Logit Trajectoriescited by 2
  33. 2025Generative Distribution Prediction: A Unified Approach to Multimodal Learningcited by 1
  34. 2025Distribution Prompting: Understanding the Expressivity of Language Models Through the Next-Token Distributions They Can Producecited by 1
  35. 2025Don't Change My View: Ideological Bias Auditing in Large Language Modelscited by 1
  36. 2025Know Your Limits: Entropy Estimation Modeling for Compression and Generalizationcited by 1
  37. 2026Buy versus Build an LLM: A Decision Framework for Governmentscited by 1
  38. 2026The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decodingcited by 1

50 most cited of 66 All 66 (JSON)