mlsanityData to 7 Oct 2026

Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations

arXiv 2301.04246 ↗First posted 10 Jan 2023cs.CYcited by 157

Not on the map. Why?

Cited by 36 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generative language models have improved drastically, and can now produce realistic text outputs that are difficult to distinguish from human-written content. For malicious actors, these language models bring the promise of automating the creation of convincing and misleading text for use in influence operations. This report assesses how language models might change influence operations in the future, and what steps can be taken to mitigate this threat. We lay out possible changes to the actors, behaviors, and content of online influence operations, and provide a framework for stages of the language model-to-influence operations pipeline that mitigations could target (model construction, model access, content dissemination, and belief formation). While no reasonable mitigation can be expected to fully prevent the threat of AI-enabled influence operations, a combination of multiple mitigations may make an important difference.

Key facts. Cited by 157 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jan 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 49; 2024: 56; 2025: 27; 2026: 25.

157 papers.

Sort:

By area:

  1. 2024GPT-4o System Cardcited by 4,467
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  4. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  5. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  6. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  7. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  8. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  9. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
  10. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  11. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  12. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
38 more
  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2023Regulating ChatGPT and other Large Generative AI Modelscited by 102
  3. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  4. 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 95
  5. 2023Art and the science of generative AI: A deeper divecited by 93
  6. 2023Evaluating the Social Impact of Generative AI Systems in Systems and Societycited by 91
  7. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  8. 2023Robust Multi-bit Natural Language Watermarking through Invariant Featurescited by 76
  9. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  10. 2023The Gradient of Generative AI Release: Methods and Considerationscited by 63
  11. 2025Authenticated Delegation and Authorized AI Agentscited by 63
  12. 2023Mark My Words: Analyzing and Evaluating Language Model Watermarkscited by 62
  13. 2024Building Guardrails for Large Language Modelscited by 61
  14. 2023Unveiling Security, Privacy, and Ethical Concerns of ChatGPTcited by 53
  15. 2024Language Models Resist Alignment: Evidence From Data Compressioncited by 50
  16. 2023Supporting Human-AI Collaboration in Auditing LLMs with LLMscited by 46
  17. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspectivecited by 44
  18. 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Modelscited by 44
  19. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  20. 2024DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detectioncited by 41
  21. 2023Honesty Is the Best Policy: Defining and Mitigating AI Deceptioncited by 40
  22. 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 40
  23. 2023Fake Alignment: Are LLMs Really Aligned Well?cited by 39
  24. 2024SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanationscited by 37
  25. 2023Anatomy of an AI-powered malicious social botnetcited by 35
  26. 2023Flames: Benchmarking Value Alignment of LLMs in Chinesecited by 35
  27. 2023All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Modelscited by 33
  28. 2024Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settingscited by 33
  29. 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectivescited by 31
  30. 2023Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attackscited by 30
  31. 2025Emergence of polarization in networks of large language model agentscited by 25
  32. 2023Disinformation Capabilities of Large Language Modelscited by 23
  33. 2024CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responsescited by 23
  34. 2023Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreakcited by 22
  35. 2023Optimizing watermarks for large language modelscited by 22
  36. 2024The Rise of AI-Generated Content in Wikipediacited by 22
  37. 2024Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecuritycited by 21
  38. 2024Can Language Models Recognize Convincing Arguments?cited by 21

50 most cited of 157 All 157 (JSON)