mlsanityData to 7 Oct 2026

ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation

arXiv 2601.12983 ↗First posted 19 Jan 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal large language models (MLLMs) are increasingly used to automate chart generation from data tables, improving efficiency but introducing new misuse risks. We present ChartAttack, a framework for evaluating how MLLMs use design misleaders to generate charts that induce incorrect interpretations. We also introduce AttackViz, a chart question-answering (QA) dataset labeled with effective misleaders and their induced incorrect answers. ChartAttack reduces MLLM QA accuracy by 17.2 points in-domain and 11.9 points cross-domain. Conditional deception rates show targeted effects: correct answers shift to attacker-intended answers 11.2% of the time in-domain and 11.7-14.9% cross-domain, while originally incorrect answers rarely change. A controlled human study shows that ChartAttack-generated charts also reduce human QA performance. Finally, fine-tuning on AttackViz improves in-domain MLLM robustness to misleading charts. Our findings highlight the need for secure, robust MLLM chart generation. Code and data are publicly available on the project website.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 95 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

51 of 95 references.

References · 51

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  8. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  11. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  12. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
38 more
  1. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  2. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  3. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  4. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  5. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  6. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 228
  7. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 166
  8. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  9. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoningcited by 127
  10. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  11. 2025ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generationcited by 88
  12. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  13. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysiscited by 61
  14. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 52
  15. 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 48
  16. 2024GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuningcited by 40
  17. 2024How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?cited by 27
  18. 2024Advancing Multimodal Large Language Models in Chart Question Answering with Visualization-Referenced Instruction Tuningcited by 27
  19. 2023Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errorscited by 22
  20. 2024Evaluation of LLM Vulnerabilities to Being Misused for Personalized Disinformation Generationcited by 20
  21. 2025Benchmarking Visual Language Models on Standardized Visualization Literacy Testscited by 20
  22. 2024HumanEval-V: Benchmarking High-Level Visual Reasoning with Complex Diagrams in Coding Taskscited by 19
  23. 2025COVE: COntext and VEracity prediction for out-of-context imagescited by 14
  24. 2025Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answeringcited by 14
  25. 2024From Deception to Detection: The Dual Roles of Large Language Models in Fake Newscited by 11
  26. 2024"I Came Across a Junk": Understanding Design Flaws of Data Visualization from the Public's Perspectivecited by 9
  27. 2025Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversioncited by 9
  28. 2023Explaining with Contrastive Phrasal Highlighting: A Case Study in Assisting Humans to Detect Translation Differencescited by 8
  29. 2023BiasX: "Thinking Slow" in Toxic Content Moderation with Explanations of Implied Social Biasescited by 7
  30. 2025What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMscited by 7
  31. 2024Have LLMs Reopened the Pandora's Box of AI-Generated Fake News?cited by 6
  32. 2025Protecting multimodal large language models against misleading visualizationscited by 6
  33. 2023Why Change My Design: Explaining Poorly Constructed Visualization Designs with Explorable Explanationscited by 5
  34. 2024Debugging Defective Visualizations: Empirical Insights Informing a Human-AI Co-Debugging Systemcited by 5
  35. 2025Understanding Why ChatGPT Outperforms Humans in Visualization Design Advicecited by 5
  36. 2025The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Modelscited by 5
  37. 2021Mapping the Landscape of COVID-19 Crisis Visualizationscited by 4
  38. 2025Is this chart lying to me? Automating the detection of misleading visualizationscited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Losing the Plot: How VLM responses degrade on imperfect charts
  2. 2026Evaluating Prompting Strategies for Chart Question Answering with Large Language Models
  3. 2026Generating Statistical Charts with Validation-Driven LLM Workflows
  4. 2026ChartBmkAgent: Harness-Governed Multi-Agent Construction of Chart QA Benchmarks from Sparse Error-Taxonomy Specifications
  5. 2025Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs
  6. 2026Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models
  7. 2025Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question Answering
  8. 2025Chart-HQA: A Benchmark for Hypothetical Question Answering in Charts