mlsanityData to 14 Oct 2026

Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defense

arXiv 2303.12132 ↗First posted 21 Mar 2023cs.CL cs.CR cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Generative Language Models gained significant attention in late 2022 / early 2023, notably with the introduction of models refined to act consistently with users' expectations of interactions with AI (conversational models). Arguably the focal point of public attention has been such a refinement of the GPT3 model -- the ChatGPT and its subsequent integration with auxiliary capabilities, including search as part of Microsoft Bing. Despite extensive prior research invested in their development, their performance and applicability to a range of daily tasks remained unclear and niche. However, their wider utilization without a requirement for technical expertise, made in large part possible through conversational fine-tuning, revealed the extent of their true capabilities in a real-world environment. This has garnered both public excitement for their potential applications and concerns about their capabilities and potential malicious uses. This review aims to provide a brief overview of the history, state of the art, and implications of Generative Language Models in terms of their principles, abilities, limitations, and future prospects -- especially in the context of cyber-defense, with a focus on the Swiss operational environment.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 60 of its 107 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

60 of 107 references.

References · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  10. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  11. 2020Scaling Laws for Neural Language Modelscited by 6,586
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
38 more
  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  13. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  14. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  15. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  16. 2018Deep contextualized word representationscited by 2,290
  17. 2020Extracting Training Data from Large Language Modelscited by 2,189
  18. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  19. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  20. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  21. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  22. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  23. 2022Red Teaming Language Models with Language Modelscited by 1,026
  24. 2022Galactica: A Large Language Model for Sciencecited by 744
  25. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  26. 2015Generating Sentences from a Continuous Spacecited by 612
  27. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  28. 2019Defending Against Neural Fake Newscited by 559
  29. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  30. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  31. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  32. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  33. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 364
  34. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  35. 2022Predictability and Surprise in Large Generative Modelscited by 210
  36. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  37. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  38. 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 177

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 3; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social mediacited by 45
  2. 2023Anatomy of an AI-powered malicious social botnetcited by 36
  3. 2024Large Language Models in Cybersecurity: State-of-the-Artcited by 34
  4. 2025Digital Forensics in the Age of Large Language Modelscited by 10
  5. 2024Transfer Learning in Pre-Trained Large Language Models for Malware Detection Based on System Callscited by 8
  6. 2024AttacKG+:Boosting Attack Knowledge Graph Construction with Large Language Modelscited by 4
  7. 2023LLMs Perform Poorly at Concept Extraction in Cyber-security Research Literaturecited by 3
  8. 2024Foundation Models for Autonomous Robots in Unstructured Environmentscited by 3
  9. 2024Novel-WD: Exploring acquisition of Novel World Knowledge in LLMs Using Prefix-Tuningcited by 2
  10. 2024Synthetic Image Generation in Cyber Influence Operations: An Emergent Threat?cited by 1
  11. 2025Generative AI for Internet of Things Security: Challenges and Opportunitiescited by 1
  12. 2025Monitoring Transformative Technological Convergence Through LLM-Extracted Semantic Entity Triple Graphscited by 0
1 more
  1. 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Modelscited by 0