mlsanityData to 14 Oct 2026

Interpretable Visual Understanding with Cognitive Attention Network

arXiv 2108.02924 ↗First posted 6 Aug 2021cs.CV cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

While image understanding on recognition-level has achieved remarkable advancements, reliable visual scene understanding requires comprehensive image understanding on recognition-level but also cognition-level, which calls for exploiting the multi-source information as well as learning different levels of understanding and extensive commonsense knowledge. In this paper, we propose a novel Cognitive Attention Network (CAN) for visual commonsense reasoning to achieve interpretable visual understanding. Specifically, we first introduce an image-text fusion module to fuse information from images and text collectively. Second, a novel inference module is designed to encode commonsense among image, query and response. Extensive experiments on large-scale Visual Commonsense Reasoning (VCR) benchmark dataset demonstrate the effectiveness of our approach. The implementation is publicly available at https://github.com/tanjatang/CAN

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 47 of its 95 references that are in the map. First posted on arXiv 6 Aug 2021.

Builds on

47 of 95 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  8. 2015VQA: Visual Question Answeringcited by 2,506
  9. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  10. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  11. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  12. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
35 more
  1. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  2. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 556
  3. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 546
  4. 2018Rethinking ImageNet Pre-trainingcited by 377
  5. 2015Neural Module Networkscited by 376
  6. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 339
  7. 2016Visual Relationship Detection with Language Priorscited by 334
  8. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 302
  9. 2019Attention on Attention for Image Captioningcited by 289
  10. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 255
  11. 2017Gate-Variants of Gated Recurrent Unit (GRU) Neural Networkscited by 242
  12. 2015Bilinear CNNs for Fine-grained Visual Recognitioncited by 228
  13. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 228
  14. 2020ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graphcited by 211
  15. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 191
  16. 2014Recurrent Neural Network Regularizationcited by 189
  17. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  18. 2017MUTAN: Multimodal Tucker Fusion for Visual Question Answeringcited by 137
  19. 2016Visual Question Answering: A Survey of Methods and Datasetscited by 123
  20. 2016Learning to Compose Neural Networks for Question Answeringcited by 85
  21. 2019FAHT: An Adaptive Fairness-aware Decision Tree Classifiercited by 63
  22. 2016Dynamic Memory Networks for Visual and Textual Question Answeringcited by 62
  23. 2015Ask Your Neurons: A Neural-based Approach to Answering Questions about Imagescited by 51
  24. 2016Multimodal Residual Learning for Visual QAcited by 42
  25. 2020Disentangled Dynamic Graph Deep Generationcited by 27
  26. 2019Mask-RCNN and U-net Ensembled for Nuclei Segmentationcited by 23
  27. 2018A Deterministic Self-Organizing Map Approach and its Application on Satellite Data based Cloud Type Classificationcited by 22
  28. 2019Answer Them All! Toward Universal Visual Question Answering Modelscited by 17
  29. 2020Using Machine Learning to Automate Mammogram Images Analysiscited by 15
  30. 2021LSTM Based Sentiment Analysis for Cryptocurrency Predictioncited by 14
  31. 2021Cognitive Visual Commonsense Reasoning Using Dynamic Working Memorycited by 14
  32. 2016Training Recurrent Answering Units with Joint Loss Minimization for VQAcited by 9
  33. 2020A Data-driven Human Responsibility Management Systemcited by 8
  34. 2020Conditional-UNet: A Condition-aware Deep Model for Coherent Human Activity Recognition From Wearablescited by 7
  35. 2021Responsibility Management through Responsibility Networkscited by 3

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 4; 2025: 5.

12 papers.

  1. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 58
  2. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  3. 2024FairAIED: Navigating Fairness, Bias, and Ethics in Educational AI Applicationscited by 15
  4. 2022Incremental Mining of Frequent Serial Episodes Considering Multiple Occurrencescited by 7
  5. 2025Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applicationscited by 5
  6. 2025AMCR: A Framework for Assessing and Mitigating Copyright Risks in Generative Modelscited by 3
  7. 2025Towards Transparent AI: A Survey on Explainable Language Modelscited by 3
  8. 2023The Internet of Responsibilities-Connecting Human Responsibilities using Big Data and Blockchaincited by 1
  9. 2024Uncertain Boundaries: Multidisciplinary Approaches to Copyright Issues in Generative AIcited by 1
  10. 2025Fairness-Aware Graph Representation Learning with Limited Demographic Informationcited by 1
  11. 2022Attention Mechanism based Cognition-level Scene Understandingcited by 0
  12. 2025GEM: Generative Entropy-Guided Preference Modeling for Few-shot Alignment of LLMscited by 0