mlsanityData to 7 Oct 2026

Towards High-Level Semantic Intelligence

arXiv 2607.24082 ↗First posted 27 Jul 2026cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Recent advances in AI have substantially expanded its cognitive and reasoning capabilities. From the perspective of semantic complexity, the development of AI reveals a clear trajectory from simple to complex semantic processing. While early AI systems mainly addressed tasks involving direct and literal semantic perception or expression, contemporary systems are increasingly expected to perform more sophisticated cognitive reasoning, enabling the understanding and generation of High-Level Semantics (HLS). A similar trajectory can also be observed in human cognitive development. We define this transition as the shift from Basic-Level Semantic Intelligence (BLSI) to High-Level Semantic Intelligence (HLSI). However, this issue has not yet been systematically and comprehensively examined in prior work. Motivated by this gap, this survey reviews the development of AI semantic intelligence from the perspective of semantic complexity. We systematically survey existing research on HLS tasks, including humor, sarcasm, metaphor, empathy, persuasion, narrative, and other general HLS phenomena, across text, speech, vision, and multimodal scenarios. Specifically, we summarize data construction methods, modeling and optimization strategies, and evaluation methodologies for both understanding and generation. HLS is essential for advancing AI toward genuinely human-like intelligence. By synthesizing existing methods and insights from the perspective of semantic intelligence, this survey aims to support the continued development of AI toward HLSI.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 279 of its 430 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

279 of 430 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Semantic-Aware Logical Reasoning via a Semiotic Framework29 Sep
  2. 2025PRELUDE: A Benchmark Designed to Require Global Comprehension and Reasoning over Long Contexts13 Aug
  3. 2025NovelHopQA: Diagnosing Multi-Hop Reasoning Failures in Long Narrative Contexts20 May
  4. 2025R^3-VQA: "Read the Room" by Video Social Reasoning7 May
  5. 2025Finding Flawed Fictions: Evaluating Complex Reasoning in Language Models via Plot Hole Detection16 Apr
  6. 2025Pragmatic Inference Chain (PIC) Improving LLMs' Reasoning of Authentic Implicit Toxic Language3 Mar
  7. 2025WHODUNIT: Evaluation benchmark for culprit detection in mystery stories11 Feb
  8. 2025A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models25 Jan
  9. 2024DRT: Deep Reasoning Translation via Long Chain-of-Thought23 Dec
  10. 2024Automatic Extraction of Metaphoric Analogies from Literary Texts: Task Formulation, Dataset Construction, and Evaluation19 Dec
  11. 2024DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels4 Sep
  12. 2024Do LLMs Exhibit Human-Like Reasoning? Evaluating Theory of Mind in LLMs for Open-Ended Responses9 Jun
4 more
  1. 2024Metaphor Understanding Challenge Dataset for LLMs18 Mar
  2. 2023Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation5 Dec
  3. 2023Think Before You Speak: Cultivating Communication Skills of Large Language Models via Inner Monologue13 Nov
  4. 2023DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning15 Jun

Other references · 263

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2017Mask R-CNNcited by 7,259
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  9. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  10. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
38 more
  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  2. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  5. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  6. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  7. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  8. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  9. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  10. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  11. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  12. 2015Listen, Attend and Spellcited by 542
  13. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  14. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  15. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  16. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  17. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  18. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 394
  19. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  20. 2019Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)cited by 173
  21. 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 142
  22. 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mixcited by 113
  23. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  24. 2018Event2Mind: Commonsense Inference on Events, Intents, and Reactionscited by 82
  25. 2024One Thousand and One Pairs: A "novel" challenge for long-context language modelscited by 80
  26. 2022Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehensioncited by 79
  27. 2022FLUTE: Figurative Language Understanding through Textual Explanationscited by 78
  28. 2022Testing the Ability of Language Models to Interpret Figurative Languagecited by 67
  29. 2023Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmarkcited by 62
  30. 2018Modeling Naive Psychology of Characters in Simple Commonsense Storiescited by 59
  31. 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
  32. 2018Modeling Empathy and Distress in Reaction to News Storiescited by 52
  33. 2024GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abusecited by 50
  34. 2024NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surroundingcited by 49
  35. 2021"Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative Understandingcited by 48
  36. 2023Multi-lingual and Multi-cultural Figurative Language Understandingcited by 47
  37. 2023MemeCap: A Dataset for Captioning and Interpreting Memescited by 46
  38. 2023The Persuasive Power of Large Language Modelscited by 46

50 most cited of 263 All 263 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks
  2. 2025The Philosophical Foundations of Growing AI Like A Child
  3. 2025MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models
  4. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
  5. 2022Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks
  6. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineering
  7. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
  8. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Models