mlsanityData to 14 Oct 2026

From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text

arXiv 2508.09450 ↗First posted 13 Aug 2025cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Charts are very common for exploring data and communicating insights, but extracting key takeaways from charts and articulating them in natural language can be challenging. The chart-to-text task aims to automate this process by generating textual summaries of charts. While with the rapid advancement of large Vision-Language Models (VLMs), we have witnessed great progress in this domain, little to no attention has been given to potential biases in their outputs. This paper investigates how VLMs can amplify geo-economic biases when generating chart summaries, potentially causing societal harm. Specifically, we conduct a large-scale evaluation of geo-economic biases in VLM-generated chart summaries across 6,000 chart-country pairs from six widely used proprietary and open-source models to understand how a country's economic status influences the sentiment of generated summaries. Our analysis reveals that existing VLMs tend to produce more positive descriptions for high-income countries compared to middle- or low-income countries, even when country attribution is the only variable changed. We also find that models such as GPT-4o-mini, Gemini-1.5-Flash, and Phi-3.5 exhibit varying degrees of bias. We further explore inference-time prompt-based debiasing techniques using positive distractors but find them only partially effective, underscoring the complexity of the issue and the need for more robust debiasing strategies. Our code and dataset are publicly available here.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 47 of its 68 references that are in the map. First posted on arXiv 13 Aug 2025.

Builds on

47 of 68 references.

References · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  5. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  7. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  8. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  9. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  10. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  11. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  12. 2019Measuring Bias in Contextualized Word Representationscited by 276
35 more
  1. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 268
  2. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  3. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  4. 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 209
  5. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  6. 2020Towards Debiasing Sentence Representationscited by 171
  7. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 169
  8. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  9. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  10. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 147
  11. 2023Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challengescited by 131
  12. 2021Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implicationscited by 128
  13. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 114
  14. 2020Towards Controllable Biases in Language Generationcited by 112
  15. 2021Mitigating Language-Dependent Ethnic Bias in BERTcited by 87
  16. 2020Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Modelcited by 78
  17. 2022OpenCQA: Open-ended Question Answering with Chartscited by 61
  18. 2019Understanding Undesirable Word Embedding Associationscited by 59
  19. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 53
  20. 2020Mitigating Gender Bias in Captioning Systemscited by 50
  21. 2023Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioningcited by 49
  22. 2020Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learningcited by 44
  23. 2020Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognitioncited by 43
  24. 2019Exposing and Correcting the Gender Bias in Image Captioning Datasets and Modelscited by 36
  25. 2023Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checkingcited by 35
  26. 2021Uncovering Implicit Gender Bias in Narratives through Commonsense Inferencecited by 32
  27. 2023ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summariescited by 32
  28. 2024Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMscited by 30
  29. 2020Automatically Identifying Gender Issues in Machine Translation using Perturbationscited by 28
  30. 2019Towards Understanding Gender Bias in Relation Extractioncited by 23
  31. 2020Defining and Evaluating Fair Natural Language Generationcited by 17
  32. 2022Striking a Balance: Reader Takeaways and Preferences when Integrating Text and Chartscited by 17
  33. 2024"Global is Good, Local is Bad?": Understanding Brand Bias in LLMscited by 15
  34. 2023Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Modelscited by 11
  35. 2024Narrative Player: Reviving Data Narratives with Visualscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Chart Deception in Vision-Language Models: From Vulnerability to Mitigationcited by 0
  2. 2026Same Chart, Different Story: Bias in Vision-Language Chart Interpretationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ChartLens: Fine-grained Visual Attribution in Charts
  2. 2024Large Language Models are Geographically Biased
  3. 2025FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language Models
  4. 2025Losing the Plot: How VLM responses degrade on imperfect charts
  5. 2024Uplifting Lower-Income Data: Strategies for Socioeconomic Perspective Shifts in Large Multi-modal Models
  6. 2026Assessing Y-Axis Influence: Bias in Multimodal Language Models on Chart-to-Table Translation
  7. 2026Ideological Bias in LLMs' Economic Causal Reasoning
  8. 2025EconCausal: A Context-Aware Economic Reasoning Benchmark for Large Language Models