mlsanityData to 7 Oct 2026

NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Models

arXiv 2203.15859 ↗First posted 29 Mar 2022cs.CV cs.AI cs.LGcited by 47

Not on the map. Why?

Abstract (arXiv)

Neural image caption generation (NICG) models have received massive attention from the research community due to their excellent performance in visual understanding. Existing work focuses on improving NICG model accuracy while efficiency is less explored. However, many real-world applications require real-time feedback, which highly relies on the efficiency of NICG models. Recent research observed that the efficiency of NICG models could vary for different inputs. This observation brings in a new attack surface of NICG models, i.e., An adversary might be able to slightly change inputs to cause the NICG models to consume more computational resources. To further understand such efficiency-oriented threats, we propose a new attack approach, NICGSlowDown, to evaluate the efficiency robustness of NICG models. Our experimental results show that NICGSlowDown can generate images with human-unnoticeable perturbations that will increase the NICG model latency up to 483.86%. We hope this research could raise the community's concern about the efficiency robustness of NICG models.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

32 of 43 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  11. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  12. 2016Adversarial examples in the physical worldcited by 1,762
20 more
  1. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  2. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  3. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  4. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  5. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  6. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  7. 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
  8. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  9. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  10. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  11. 2019Attention on Attention for Image Captioningcited by 289
  12. 2020X-Linear Attention Networks for Image Captioningcited by 198
  13. 2017EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examplescited by 154
  14. 2016Spatially Adaptive Computation Time for Residual Networkscited by 118
  15. 2018Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defensescited by 109
  16. 2018Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examplescited by 98
  17. 2020A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inferencecited by 61
  18. 2016Semantic Compositional Networks for Visual Captioningcited by 50
  19. 2020RATT: Recurrent Attention to Transient Tasks for Continual Image Captioningcited by 32
  20. 2022EREBA: Black-box Energy Testing of Adaptive Neural Networkscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 10; 2024: 9; 2025: 15; 2026: 9.

47 papers.

Sort:

By area:

  1. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
  2. 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Imagescited by 69
  3. 2024Denial-of-Service Poisoning Attacks against Large Language Modelscited by 34
  4. 2023Sibling-Attack: Rethinking Transferable Adversarial Attacks against Face Recognitioncited by 32
  5. 2024FMint: Bridging Human Designed and Data Pretrained Models for Differential Equation Foundation Modelcited by 18
  6. 2024Poison-splat: Computation Cost Attack on 3D Gaussian Splattingcited by 17
  7. 2022LLMEffiChecker: Understanding and Testing Efficiency Degradation of Large Language Modelscited by 16
  8. 2022DeepPerform: An Efficient Approach for Performance Testing of Resource-Constrained Neural Networkscited by 16
  9. 2024On Efficient Variants of Segment Anything Model: A Surveycited by 16
  10. 2024Energy-Latency Manipulation of Multi-modal Large Language Models via Verbose Samplescited by 14
  11. 2023Dynamic Transformers Provide a False Sense of Efficiencycited by 11
  12. 2023SlothSpeech: Denial-of-service Attack Against Speech Recognition Modelscited by 9
35 more
  1. 2023RT-LM: Uncertainty-Aware Resource Management for Real-Time Inference of Language Modelscited by 9
  2. 2025LingoLoop Attack: Trapping MLLMs via Linguistic Context and State Entrapment into Endless Loopscited by 9
  3. 2024PPM: Automated Generation of Diverse Programming Problems for Benchmarking Code Generation Modelscited by 7
  4. 2024The Impact of Uniform Inputs on Activation Sparsity and Energy-Latency Attacks in Computer Visioncited by 7
  5. 2025RemedyGS: Defend 3D Gaussian Splatting against Computation Cost Attackscited by 7
  6. 2025Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilerscited by 6
  7. 2022Dynamics-aware Adversarial Attack of Adaptive Neural Networkscited by 5
  8. 2025Energy-Latency Attacks: A New Adversarial Threat to Deep Learningcited by 5
  9. 2025Protecting Your Video Content: Disrupting Automated Video-based LLM Annotationscited by 5
  10. 2023R^3: On-device Real-Time Deep Reinforcement Learning for Autonomous Roboticscited by 4
  11. 2023RED: A Systematic Real-Time Scheduling Approach for Robotic Environmental Dynamicscited by 4
  12. 2025On Surjectivity of Neural Networks: Can you elicit any behavior from your model?cited by 4
  13. 2023SlowFormer: Universal Adversarial Patch for Attack on Compute and Energy Efficiency of Inference Efficient Vision Transformerscited by 3
  14. 2025VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Servicecited by 3
  15. 2026DRAINCODE: Stealthy Energy Consumption Attacks on Retrieval-Augmented Code Generation via Context Poisoningcited by 3
  16. 2026SlowBA: An efficiency backdoor attack towards VLM-based GUI agentscited by 3
  17. 2022CorrGAN: Input Transformation Technique Against Natural Corruptionscited by 2
  18. 2025BitHydra: Towards Bit-flip Inference Cost Attack against Large Language Modelscited by 2
  19. 2025Resource Consumption Red-Teaming for Large Vision-Language Modelscited by 2
  20. 2026Rethinking Latency Denial-of-Service: Attacking the LLM Serving Framework, Not the Modelcited by 2
  21. 2026Resource Consumption Threats in Large Language Modelscited by 2
  22. 2023Dynamic Neural Network is All You Need: Understanding the Robustness of Dynamic Mechanisms in Neural Networkscited by 1
  23. 2023No-Skim: Towards Efficiency Robustness Evaluation on Skimming-based Language Modelscited by 1
  24. 2024Resilience of Entropy Model in Distributed Neural Networkscited by 1
  25. 2025Energy Backdoor Attack to Deep Neural Networkscited by 1
  26. 2025Efficiency Robustness of Dynamic Deep Learning Systemscited by 1
  27. 2025Exploiting Efficiency Vulnerabilities in Dynamic Deep Learning Systemscited by 1
  28. 2023Trainwreck: A damaging adversarial attack on image classifierscited by 0
  29. 2025Impact Analysis of Inference Time Attack of Perception Sensors on Autonomous Vehiclescited by 0
  30. 2025An Image Is Worth Ten Thousand Words: Verbose-Text Induction Attacks on VLMscited by 0
  31. 2026MORE: Multi-Objective Adversarial Attacks on Speech Recognitioncited by 0
  32. 2026VidDoS: Universal Denial-of-Service Attack on Video-based Large Language Modelscited by 0
  33. 2026AESOP: Adversarial Execution-path Selection to Overload Deep Learning Pipelinescited by 0
  34. 2026MOAT: Model-Agnostic Randomized Transformations for preventing Efficiency Degradation Attacks on ViTscited by 0
  35. 2026A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimizationcited by 0