Learning to Infer Generative Template Programs for Visual Concepts
Not on the map. Why?
Abstract (arXiv)
People grasp flexible visual concepts from a few examples. We explore a neurosymbolic system that learns how to infer programs that capture visual concepts in a domain-general fashion. We introduce Template Programs: programmatic expressions from a domain-specific language that specify structural and parametric patterns common to an input concept. Our framework supports multiple concept-related tasks, including few-shot generation and co-segmentation through parsing. We develop a learning paradigm that allows us to train networks that infer Template Programs directly from visual datasets that contain concept groupings. We run experiments across multiple visual domains: 2D layouts, Omniglot characters, and 3D shapes. We find that our method outperforms task-specific alternatives, and performs competitively against domain-specific approaches for the limited domains where they exist.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 20 Mar 2024.
Builds on
29 of 54 references.
References · 29
- 2017Attention Is All You Needcited by 59,932
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2017Neural Discrete Representation Learningcited by 4,650
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2017Prototypical Networks for Few-shot Learningcited by 3,001
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
- 2022SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebookscited by 121
- 2016Towards a Neural Statisticiancited by 119
- 2017CSGNet: Neural Shape Parser for Constructive Solid Geometrycited by 106
17 more
- 2019Write, Execute, Assess: Program Synthesis with a REPLcited by 92
- 2019BAE-NET: Branched Autoencoder for Shape Co-Segmentationcited by 89
- 2017Learning to Infer Graphics Programs from Hand-Drawn Imagescited by 73
- 2018Synthesizing Programs for Images using Reinforced Adversarial Learningcited by 72
- 2019Learning to Infer and Execute 3D Shape Programscited by 68
- 2020Simple and Effective VAE Training with Calibrated Decoderscited by 57
- 2019The Omniglot challenge: a 3-year progress reportcited by 54
- 2021Inferring CAD Modeling Sequences Using Zone Graphscited by 50
- 2016One-Shot Generalization in Deep Generative Modelscited by 44
- 2023Neurosymbolic Models for Computer Graphicscited by 27
- 2018The Variational Homoencoder: Learning to learn high capacity generative models from few examplescited by 23
- 2020PLAD: Learning to Infer Shape Programs with Pseudo-Labels and Approximate Distributionscited by 19
- 2020Learning to learn generative programs with Memoised Wake-Sleepcited by 15
- 2020Learning Task-General Representations with Generative Neuro-Symbolic Modelingcited by 14
- 2023Improving Unsupervised Visual Program Inference with Code Rewriting Familiescited by 10
- 2023PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views with Learnt Shape Programscited by 8
- 2022Drawing out of Distribution with Neuro-Symbolic Generative Modelscited by 5
Built on by
Citing papers per year: 2024: 2.
2 papers.