mlsanityData to 14 Oct 2026

On Surjectivity of Neural Networks: Can you elicit any behavior from your model?

arXiv 2508.19445 ↗First posted 26 Aug 2025cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Given a trained neural network, can any specified output be generated by some input? Equivalently, does the network correspond to a function that is surjective? In generative models, surjectivity implies that any output, including harmful or undesirable content, can in principle be generated by the networks, raising concerns about model safety and jailbreak vulnerabilities. In this paper, we prove that many fundamental building blocks of modern neural architectures, such as networks with pre-layer normalization and linear-attention modules, are almost always surjective. As corollaries, widely used generative frameworks, including GPT-style transformers and diffusion models with deterministic ODE solvers, admit inverse mappings for arbitrary outputs. By studying surjectivity of these modern and commonly used neural architectures, we contribute a formalism that sheds light on their unavoidable vulnerability to a broad class of adversarial attacks.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 98 of its 110 references that are in the map. First posted on arXiv 26 Aug 2025.

Builds on

98 of 110 references.

References · 98

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  11. 2022Scalable Diffusion Models with Transformerscited by 6,054
  12. 2016Layer Normalizationcited by 5,669
38 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  6. 2015Variational Inference with Normalizing Flowscited by 2,170
  7. 2016Density estimation using Real NVPcited by 2,042
  8. 2020Reformer: The Efficient Transformercited by 1,668
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  10. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  11. 2018Group Normalizationcited by 1,524
  12. 2019Root Mean Square Layer Normalizationcited by 1,480
  13. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  14. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  15. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  16. 2020On Layer Normalization in the Transformer Architecturecited by 922
  17. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  18. 2017Masked Autoregressive Flow for Density Estimationcited by 684
  19. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  20. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  21. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  22. 2019Neural Spline Flowscited by 596
  23. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
  24. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
  25. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  26. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  27. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  28. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  29. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  30. 2019Implicit Regularization in Deep Matrix Factorizationcited by 405
  31. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
  32. 2018Invertible Residual Networkscited by 365
  33. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  34. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  35. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 328
  36. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 284
  37. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
  38. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 273

50 most cited of 98 All 98 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Language Models are Injective and Hence Invertiblecited by 35
  2. 2026NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agentscited by 1
  3. 2026Prompting Complexity: Shortest Prompts for Texts and Behaviors in LLMscited by 1
  4. 2025Transformer Injectivity & Geometric Robustness - Analytic Margins and Bi-Lipschitz Uniformity of Sequence-Level Hidden Statescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Threat Model-Agnostic Adversarial Defense using Diffusion Models
  2. 2025Information-Geometric Inverse Distillation for Enhancing Adversarial Transferability
  3. 2024Model Mimic Attack: Knowledge Distillation for Provably Transferable Adversarial Examples
  4. 2025Data-Free Model-Related Attacks: Unleashing the Potential of Generative AI
  5. 2024Exploring Adversarial Attacks against Latent Diffusion Model from the Perspective of Adversarial Transferability
  6. 2024SIDE: Surrogate Conditional Data Extraction from Diffusion Models
  7. 2025Embedding Hidden Adversarial Capabilities in Pre-Trained Diffusion Models
  8. 2026Representational Simplicity and Circuit Size Dissociate in a Threshold-Dependent Way: A Controlled Test via Adversarial Training