mlsanityData to 14 Oct 2026

Testing the Depth of ChatGPT's Comprehension via Cross-Modal Tasks Based on ASCII-Art: GPT3.5's Abilities in Regard to Recognizing and Generating ASCII-Art Are Not Totally Lacking

arXiv 2307.16806 ↗First posted 28 Jul 2023cs.CL cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Over the eight months since its release, ChatGPT and its underlying model, GPT3.5, have garnered massive attention, due to their potent mix of capability and accessibility. While a niche-industry of papers have emerged examining the scope of capabilities these models possess, the information fed to and extracted from these networks has been either natural language text or stylized, code-like language. Drawing inspiration from the prowess we expect a truly human-level intelligent agent to have across multiple signal modalities, in this work we examine GPT3.5's aptitude for visual tasks, where the inputs feature content provided as ASCII-art without overt distillation into a lingual summary. We conduct experiments analyzing the model's performance on image recognition tasks after various transforms typical in visual settings, trials investigating knowledge of image parts, and tasks covering image generation.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 40 of its 96 references that are in the map. First posted on arXiv 28 Jul 2023.

Builds on

40 of 96 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  6. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  7. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  8. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  9. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
  10. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
  11. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 301
  12. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 279
28 more
  1. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  2. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 210
  3. 2023ChatGPT: Jack of all trades, master of nonecited by 204
  4. 2023A Categorical Archive of ChatGPT Failurescited by 168
  5. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 116
  6. 2022Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Trainingcited by 88
  7. 2023Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPTcited by 80
  8. 2023Level Generation Through Large Language Modelscited by 68
  9. 2023Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex and GPT-3 Large Language Modelscited by 59
  10. 2023One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Eracited by 55
  11. 2023InstructVid2Vid: Controllable Video Editing with Natural Language Instructionscited by 44
  12. 2017ShapeWorld - A new test methodology for multimodal language understandingcited by 37
  13. 2023Introspective Tips: Large Language Model for In-Context Decision Makingcited by 28
  14. 2023ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphscited by 27
  15. 2023LMEye: An Interactive Perception Network for Large Language Modelscited by 27
  16. 2023Bot or Human? Detecting ChatGPT Imposters with A Single Questioncited by 26
  17. 2023Improved Trust in Human-Robot Collaboration with ChatGPTcited by 18
  18. 2023Learning to Reason over Scene Graphs: A Case Study of Finetuning GPT-2 into a Robot Language Model for Grounded Task Planningcited by 16
  19. 2023Analysis of ChatGPT on Source Codecited by 12
  20. 2023Combo of Thinking and Observing for Outside-Knowledge VQAcited by 9
  21. 2022Is GPT-3 all you need for Visual Question Answering in Cultural Heritage?cited by 6
  22. 2023Ask and You Shall Receive (a Graph Drawing): Testing ChatGPT's Potential to Apply Graph Layout Algorithmscited by 6
  23. 2023A Glimpse in ChatGPT Capabilities and its impact for AI researchcited by 6
  24. 2023Analyzing ChatGPT's Aptitude in an Introductory Computer Engineering Coursecited by 5
  25. 2023Instance-Level Semantic Maps for Vision Language Navigationcited by 5
  26. 2023Large language models and (non-)linguistic recursioncited by 5
  27. 2021Exploiting Vulnerabilities in Deep Neural Networks: Adversarial and Fault-Injection Attackscited by 4
  28. 2023Generating Language Corrections for Teaching Physical Control Taskscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 1.

7 papers.

  1. 2024Large Language Models for Page Stream Segmentationcited by 5
  2. 2025Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Modelscited by 5
  3. 2024ASCIIEval: Benchmarking Models' Visual Perception in Text Strings via ASCII Artcited by 4
  4. 2024Large Language Models Understand Layoutcited by 3
  5. 2025ArtPerception: ASCII Art-based Jailbreak on LLMs with Recognition Pre-testcited by 2
  6. 2025From Building Blocks to Planning: Multi-Step Spatial Reasoning in LLMs with Reinforcement Learningcited by 1
  7. 2026Resolution Thresholds in VLM Detection of Harmful ASCII Art Across Construction Modes and Languagescited by 1