# Hybrid Primal Sketch: Combining Analogy, Qualitative Representations, and Computer Vision for Scene Understanding

arXiv: https://arxiv.org/abs/2407.04859 · HTML: https://mlsanity.com/paper/2407.04859/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 66 references that are in the map.
- First posted on arXiv 5 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [A Diagram Is Worth A Dozen Images](https://mlsanity.com/paper/1603.07396/) (2016)
- [CogVLM: Visual Expert for Pretrained Language Models](https://mlsanity.com/paper/2311.03079/) (2023)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models](https://mlsanity.com/paper/2406.14852/) (2024)
- [Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models](https://mlsanity.com/paper/2404.07973/) (2024)
- [Visual Hallucinations of Multi-modal Large Language Models](https://mlsanity.com/paper/2402.14683/) (2024)
- [Tiny and Efficient Model for the Edge Detection Generalization](https://mlsanity.com/paper/2308.06468/) (2023)
- [Towards Unified Scene Text Spotting based on Sequence Generation](https://mlsanity.com/paper/2304.03435/) (2023)
- [Evaluation and Enhancement of Semantic Grounding in Large Vision-Language Models](https://mlsanity.com/paper/2309.04041/) (2023)
- [Semantic Parsing to Probabilistic Programs for Situated Question Answering](https://mlsanity.com/paper/1606.07046/) (2016)

## Abstract (arXiv)

One of the purposes of perception is to bridge between sensors and conceptual understanding. Marr's Primal Sketch combined initial edge-finding with multiple downstream processes to capture aspects of visual perception such as grouping and stereopsis. Given the progress made in multiple areas of AI since then, we have developed a new framework inspired by Marr's work, the Hybrid Primal Sketch, which combines computer vision components into an ensemble to produce sketch-like entities which are then further processed by CogSketch, our model of high-level human vision, to produce both more detailed shape representations and scene representations which can be used for data-efficient learning via analogical generalization. This paper describes our theoretical framework, summarizes several previous experiments, and outlines a new experiment in progress on diagram understanding.