mlsanityData to 14 Oct 2026

Commonsense Reasoning in Computer Vision: Foundations, Recent Advancements, and Future Directions

arXiv 2609.05257 ↗First posted 4 Sep 2026cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Commonsense reasoning in computer vision encompasses integrating visual data and contextual knowledge, crucial for enhancing AI's understanding of everyday scenarios. This understanding not only improves machine learning models but also enhances their ability to interact meaningfully with humans and the environment. Unlike CNN-based conventional vision models, which are designed to identify objects within a specific image, incorporating commonsense knowledge enables models to interpret scenes in a more holistic manner, thereby improving their spatial ability to reason about relationships among objects and actions. This integration not only enhances object recognition but also facilitates a deeper understanding of the contextual factors, ultimately leading to more precise predictions and interactions in real-world applications. This paper presents a comprehensive survey of recent developments that integrate commonsense knowledge into computer vision tasks. We systematically review approaches based on knowledge graphs, scene graphs, neuro-symbolic models, and commonsense-augmented transformers. We also outline current limitations related to dataset bias, knowledge incompleteness, and integration challenges. Finally, we highlight prospective research trajectories in cross-modal reasoning, scalable commonsense knowledge injection, and neuro-symbolic hybrid architectures to develop truly intelligent visual systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 35 of its 53 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

35 of 53 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Visual-Linguistic Agent: Towards Collaborative Contextual Object Reasoning15 Nov

Other references · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  9. 2015VQA: Visual Question Answeringcited by 2,506
  10. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  11. 2019Language Models as Knowledge Bases?cited by 1,936
  12. 2016Concrete Problems in AI Safetycited by 1,642
22 more
  1. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  2. 2019Natural Adversarial Examplescited by 1,408
  3. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  4. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  5. 2018Panoptic Segmentationcited by 802
  6. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 556
  7. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  8. 2018nocaps: novel object captioning at scalecited by 444
  9. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  10. 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 372
  11. 2017A simple neural network module for relational reasoningcited by 346
  12. 2018Graph R-CNN for Scene Graph Generationcited by 298
  13. 2020Neurosymbolic AI: The 3rd Wavecited by 255
  14. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  15. 2019Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcited by 216
  16. 2018To Trust Or Not To Trust A Classifiercited by 207
  17. 2020The Next Decade in AI: Four Steps Towards Robust Artificial Intelligencecited by 195
  18. 2021SGTR: End-to-end Scene Graph Generation with Transformercited by 103
  19. 2021What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpuscited by 80
  20. 2017Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Carcited by 66
  21. 2016Visual Question Answering: Datasets, Algorithms, and Future Challengescited by 56
  22. 2021Not All Relations are Equal: Mining Informative Labels for Scene Graph Generationcited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies
  2. 2022Combining Commonsense Reasoning and Knowledge Acquisition to Guide Deep Learning in Robotics
  3. 2025Generative Visual Commonsense Answering and Explaining with Generative Scene Graph Constructing
  4. 2023ROME: Evaluating Pre-trained Vision-Language Models on Reasoning beyond Visual Common Sense
  5. 2023Improving Vision-and-Language Reasoning via Spatial Relations Modeling
  6. 2025A Study of Commonsense Reasoning over Visual Object Properties
  7. 2023Benchmarks for Automated Commonsense Reasoning: A Survey
  8. 2022Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles