# Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense

arXiv: https://arxiv.org/abs/2004.09044 · HTML: https://mlsanity.com/paper/2004.09044/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Apr 2020.

## Built on by (most cited first, 30 of 62)

- [An Embodied Generalist Agent in 3D World](https://mlsanity.com/paper/2311.12871/) (2023)
- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding](https://mlsanity.com/paper/2401.09340/) (2024)
- [Evaluating and Inducing Personality in Pre-trained Language Models](https://mlsanity.com/paper/2206.07550/) (2022)
- [PhyRecon: Physically Plausible Neural Scene Reconstruction](https://mlsanity.com/paper/2404.16666/) (2024)
- [Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution](https://mlsanity.com/paper/2103.14230/) (2021)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices](https://mlsanity.com/paper/2201.12382/) (2022)
- [PartAfford: Part-level Affordance Discovery from 3D Objects](https://mlsanity.com/paper/2202.13519/) (2022)
- [YouRefIt: Embodied Reference Understanding with Language and Gesture](https://mlsanity.com/paper/2109.03413/) (2021)
- [A Review of Emerging Research Directions in Abstract Visual Reasoning](https://mlsanity.com/paper/2202.10284/) (2022)
- [ACRE: Abstract Causal REasoning Beyond Covariation](https://mlsanity.com/paper/2103.14232/) (2021)
- [Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning](https://mlsanity.com/paper/2111.12990/) (2021)
- [PQ-Transformer: Jointly Parsing 3D Objects and Layouts from Point Clouds](https://mlsanity.com/paper/2109.05566/) (2021)
- [HALSIE: Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalities](https://mlsanity.com/paper/2211.10754/) (2022)
- [Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representations](https://mlsanity.com/paper/2404.17521/) (2024)
- [Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysis](https://mlsanity.com/paper/2503.22420/) (2025)
- [SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation](https://mlsanity.com/paper/2310.16838/) (2023)
- [Congestion-aware Multi-agent Trajectory Prediction for Collision Avoidance](https://mlsanity.com/paper/2103.14231/) (2021)
- [CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models](https://mlsanity.com/paper/2109.01401/) (2021)
- [Learning Triadic Belief Dynamics in Nonverbal Communication from Videos](https://mlsanity.com/paper/2104.02841/) (2021)
- [Learning 6-DoF Fine-grained Grasp Detection Based on Part Affordance Grounding](https://mlsanity.com/paper/2301.11564/) (2023)
- [MEWL: Few-shot multimodal word learning with referential uncertainty](https://mlsanity.com/paper/2306.00503/) (2023)
- [HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving](https://mlsanity.com/paper/2102.11344/) (2021)
- [I-PHYRE: Interactive Physical Reasoning](https://mlsanity.com/paper/2312.03009/) (2023)
- [Gathering Strength, Gathering Storms: The One Hundred Year Study on Artificial Intelligence (AI100) 2021 Study Panel Report](https://mlsanity.com/paper/2210.15767/) (2022)
- [On the Learning Mechanisms in Physical Reasoning](https://mlsanity.com/paper/2210.02075/) (2022)
- [Instance-wise or Class-wise? A Tale of Neighbor Shapley for Concept-based Explanation](https://mlsanity.com/paper/2109.01369/) (2021)
- [IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes](https://mlsanity.com/paper/2112.05298/) (2021)
- [Toward a `Standard Model' of Machine Learning](https://mlsanity.com/paper/2108.07783/) (2021)

## Abstract (arXiv)

Recent progress in deep learning is essentially based on a "big data for small tasks" paradigm, under which massive amounts of data are used to train a classifier for a single narrow task. In this paper, we call for a shift that flips this paradigm upside down. Specifically, we propose a "small data for big tasks" paradigm, wherein a single artificial intelligence (AI) system is challenged to develop "common sense", enabling it to solve a wide range of tasks with little training data. We illustrate the potential power of this new paradigm by reviewing models of common sense that synthesize recent breakthroughs in both machine and human vision. We identify functionality, physics, intent, causality, and utility (FPICU) as the five core domains of cognitive AI with humanlike common sense. When taken as a unified concept, FPICU is concerned with the questions of "why" and "how", beyond the dominant "what" and "where" framework for understanding vision. They are invisible in terms of pixels but nevertheless drive the creation, maintenance, and development of visual scenes. We therefore coin them the "dark matter" of vision. Just as our universe cannot be understood by merely studying observable matter, we argue that vision cannot be understood without studying FPICU. We demonstrate the power of this perspective to develop cognitive AI systems with humanlike common sense by showing how to observe and apply FPICU with little training data to solve a wide range of challenging tasks, including tool use, planning, utility inference, and social learning. In summary, we argue that the next generation of AI must embrace "dark" humanlike common sense for solving novel tasks.