mlsanityData to 7 Oct 2026

Beyond Object Recognition: A New Benchmark towards Object Concept Learning

arXiv 2212.02710 ↗First posted 6 Dec 2022cs.CV cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Understanding objects is a central building block of artificial intelligence, especially for embodied AI. Even though object recognition excels with deep learning, current machines still struggle to learn higher-level knowledge, e.g., what attributes an object has, and what can we do with an object. In this work, we propose a challenging Object Concept Learning (OCL) task to push the envelope of object understanding. It requires machines to reason out object affordances and simultaneously give the reason: what attributes make an object possesses these affordances. To support OCL, we build a densely annotated knowledge base including extensive labels for three levels of object concept (category, attribute, affordance), and the causal relations of three levels. By analyzing the causal structure of OCL, we present a baseline, Object Concept Reasoning Network (OCRN). It leverages causal intervention and concept instantiation to infer the three levels following their causal relations. In experiments, OCRN effectively infers the object knowledge while following the causalities well. Our data and code are available at https://mvig-rhos.com/ocl.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 52 of its 111 references that are in the map. First posted on arXiv 6 Dec 2022.

Builds on

52 of 111 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  10. 2016Layer Normalizationcited by 5,579
  11. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  12. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
38 more
  1. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  2. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  3. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
  4. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  5. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  6. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
  7. 2020Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effectcited by 256
  8. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  9. 2017AttGAN: Facial Attribute Editing by Only Changing What You Wantcited by 241
  10. 2017Learning to Detect Human-Object Interactionscited by 238
  11. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  12. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 199
  13. 2017Neural-Symbolic Learning and Reasoning: A Survey and Interpretationcited by 195
  14. 2015Visual Semantic Role Labelingcited by 188
  15. 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 126
  16. 2018iCAN: Instance-Centric Attention Network for Human-Object Interaction Detectioncited by 126
  17. 2020Visual Commonsense R-CNNcited by 106
  18. 2020Detailed 2D-3D Joint Representation for Human-Object Interactioncited by 93
  19. 2020Symmetry and Group in Attribute-Object Compositionscited by 89
  20. 2020HOI Analysis: Integrating and Decomposing Human-Object Interactioncited by 81
  21. 2020PaStaNet: Toward Human Activity Knowledge Enginecited by 79
  22. 2020A causal view of compositional zero-shot recognitioncited by 78
  23. 2017Learning to Act Properly: Predicting and Explaining Affordances from Imagescited by 74
  24. 2014Visual Causal Feature Learningcited by 59
  25. 2019Causal Reasoning from Meta-reinforcement Learningcited by 53
  26. 2018Binge Watching: Scaling Affordance Learning from Sitcomscited by 41
  27. 2022Interactiveness Field in Human-Object Interactionscited by 40
  28. 2016The Curious Robot: Learning Visual Representations via Physical Interactionscited by 37
  29. 2022Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detectioncited by 34
  30. 2019HAKE: Human Activity Knowledge Enginecited by 32
  31. 2019Causal Induction from Visual Observations for Goal Directed Taskscited by 31
  32. 2022HAKE: A Knowledge Engine Foundation for Human Activity Understandingcited by 28
  33. 2022Constructing Balance from Imbalance for Long-tailed Image Recognitioncited by 25
  34. 2017Deep Affordance-grounded Sensorimotor Object Recognitioncited by 17
  35. 2015In Defense of the Direct Perception of Affordancescited by 9
  36. 2022Highlighting Object Category Immunity for the Generalization of Human-Object Interaction Detectioncited by 9
  37. 2018DOCK: Detecting Objects by transferring Common-sense Knowledgecited by 8
  38. 2021Learning Single/Multi-Attribute of Object with Symmetry and Groupcited by 8

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 1; 2026: 4.

11 papers.

  1. 2024Visual Knowledge in the Big Model Era: Retrospect and Prospectcited by 16
  2. 2024WorldAfford: Affordance Grounding based on Natural Language Instructionscited by 14
  3. 2026The Great March 100: 100 Detail-oriented Tasks for Evaluating Embodied AI Agentscited by 5
  4. 2024DISCO: Embodied Navigation and Interaction via Differentiable Scene Semantics and Dual-level Controlcited by 4
  5. 2024Take A Step Back: Rethinking the Two Stages in Visual Reasoningcited by 4
  6. 2024The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMscited by 4
  7. 2026Tac-DINO: Learning Vision-Tactile Features with Patch Alignmentcited by 1
  8. 2024M3-VOS: Multi-Phase, Multi-Transition, and Multi-Scenery Video Object Segmentationcited by 0
  9. 2025Contrastive Learning with Enhanced Abstract Representations using Grouped Loss of Abstract Semantic Supervisioncited by 0
  10. 2026Beyond Static Vision: Scene Dynamic Field Unlocks Intuitive Physics Understanding in Multi-modal Large Language Modelscited by 0
  11. 2026HAT-4D: Lifting Monocular Video for 4D Multi-Object Interactions via Human-Agent Collaborationcited by 0