mlsanityData to 7 Oct 2026

Visual Knowledge in the Big Model Era: Retrospect and Prospect

arXiv 2404.04308 ↗First posted 5 Apr 2024cs.AI cs.CV cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Visual knowledge is a new form of knowledge representation that can encapsulate visual concepts and their relations in a succinct, comprehensive, and interpretable manner, with a deep root in cognitive psychology. As the knowledge about the visual world has been identified as an indispensable component of human cognition and intelligence, visual knowledge is poised to have a pivotal role in establishing machine intelligence. With the recent advance of Artificial Intelligence (AI) techniques, large AI models (or foundation models) have emerged as a potent tool capable of extracting versatile patterns from broad data as implicit knowledge, and abstracting them into an outrageous amount of numeric parameters. To pave the way for creating visual knowledge empowered AI machines in this coming wave, we present a timely review that investigates the origins and development of visual knowledge in the pre-big model era, and accentuates the opportunities and unique role of visual knowledge in the big model era.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 44 of its 91 references that are in the map. First posted on arXiv 5 Apr 2024.

Builds on

44 of 91 references.

References · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  8. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  10. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
32 more
  1. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  2. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  3. 2016Generative Adversarial Text to Image Synthesiscited by 735
  4. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  5. 2017Dynamic Routing Between Capsulescited by 689
  6. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  7. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  8. 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 470
  9. 2015Neural Module Networkscited by 375
  10. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  11. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  12. 2024A Survey on 3D Gaussian Splattingcited by 221
  13. 2019Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoningcited by 149
  14. 2022Rethinking Semantic Segmentation: A Prototype Viewcited by 140
  15. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  16. 2023Make-Your-Video: Customized Video Generation Using Textual and Structural Guidancecited by 129
  17. 2020Visual Commonsense R-CNNcited by 106
  18. 2021Counterfactual Zero-Shot and Open-Set Visual Recognitioncited by 95
  19. 2017Visual Interaction Networkscited by 94
  20. 2022GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Modelscited by 87
  21. 2020Causal Discovery in Physical Systems from Videoscited by 77
  22. 2020A causal framework for distribution generalizationcited by 68
  23. 2022Visual Recognition with Deep Nearest Centroidscited by 59
  24. 2018Anticipating Traffic Accidents with Adaptive Loss and Large-scale Incident DBcited by 51
  25. 2019Probabilistic Neural-symbolic Models for Interpretable Visual Question Answeringcited by 38
  26. 2020Hierarchical Human Parsing with Typed Part-Relation Reasoningcited by 38
  27. 2020Learning Compositional Neural Information Fusion for Human Parsingcited by 36
  28. 2020Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning"cited by 35
  29. 2021Relational World Knowledge Representation in Contextual Language Models: A Reviewcited by 31
  30. 2023LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoningcited by 17
  31. 2022Beyond Object Recognition: A New Benchmark towards Object Concept Learningcited by 11
  32. 2021Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 5; 2026: 2.

16 papers.

Sort:

By area:

  1. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
  2. 2024Imp: Highly Capable Large Multimodal Models for Mobile Devicescited by 19
  3. 2024Scene Graph Generation with Role-Playing Large Language Modelscited by 16
  4. 2024Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Modelscited by 15
  5. 2025Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Modelcited by 15
  6. 2024Nonverbal Interaction Detectioncited by 14
  7. 2025Learning Clustering-based Prototypes for Compositional Zero-shot Learningcited by 13
  8. 2024Hydra-SGG: Hybrid Relation Assignment for One-stage Scene Graph Generationcited by 10
  9. 2025DIFFVSGG: Diffusion-Driven Online Video Scene Graph Generationcited by 10
  10. 2024General and Task-Oriented Video Segmentationcited by 6
  11. 2026Spatio-temporal Decoupled Knowledge Compensator for Few-Shot Action Recognitioncited by 4
  12. 2024Fine-Grained Domain Generalization with Feature Structuralizationcited by 3
4 more
  1. 2024Tran-GCN: A Transformer-Enhanced Graph Convolutional Network for Person Re-Identification in Monitoring Videoscited by 2
  2. 2025A Conditional Probability Framework for Compositional Zero-shot Learningcited by 2
  3. 2024PiPa++: Towards Unification of Domain Adaptive Semantic Segmentation via Self-supervised Learningcited by 1
  4. 2026Learning 3D Representations for Spatial Intelligence from Unposed Multi-View Imagescited by 1