# Program-Guided Image Manipulators

arXiv: https://arxiv.org/abs/1909.02116 · HTML: https://mlsanity.com/paper/1909.02116/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2019.

## Built on by (most cited first, 12 of 12)

- [HAKE: A Knowledge Engine Foundation for Human Activity Understanding](https://mlsanity.com/paper/2202.06851/) (2022)
- [Is Programming by Example solved by LLMs?](https://mlsanity.com/paper/2406.08316/) (2024)
- [ProTo: Program-Guided Transformer for Program-Guided Tasks](https://mlsanity.com/paper/2110.00804/) (2021)
- [IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as Agentic Inverse Rendering](https://mlsanity.com/paper/2506.23329/) (2025)
- [Hierarchical Motion Understanding via Motion Programs](https://mlsanity.com/paper/2104.11216/) (2021)
- [Neurosymbolic Grounding for Compositional World Models](https://mlsanity.com/paper/2310.12690/) (2023)
- [Programmatic Concept Learning for Human Motion Description and Synthesis](https://mlsanity.com/paper/2206.13502/) (2022)
- [Learning to Infer 3D Shape Programs with Differentiable Renderer](https://mlsanity.com/paper/2206.12675/) (2022)
- [Tree-Structured Shading Decomposition](https://mlsanity.com/paper/2309.07122/) (2023)
- [DeltaGAN: Towards Diverse Few-shot Image Generation with Sample-Specific Delta](https://mlsanity.com/paper/2207.10271/) (2022)
- [Pattern Analogies: Learning to Perform Programmatic Image Edits by Analogy](https://mlsanity.com/paper/2412.12463/) (2024)
- [Grounding vs. Compositionality: On the Non-Complementarity of Reasoning in Neuro-Symbolic Systems](https://mlsanity.com/paper/2604.26521/) (2026)

## Abstract (arXiv)

Humans are capable of building holistic representations for images at various levels, from local objects, to pairwise relations, to global structures. The interpretation of structures involves reasoning over repetition and symmetry of the objects in the image. In this paper, we present the Program-Guided Image Manipulator (PG-IM), inducing neuro-symbolic program-like representations to represent and manipulate images. Given an image, PG-IM detects repeated patterns, induces symbolic programs, and manipulates the image using a neural network that is guided by the program. PG-IM learns from a single image, exploiting its internal statistics. Despite trained only on image inpainting, PG-IM is directly capable of extrapolation and regularity editing in a unified framework. Extensive experiments show that PG-IM achieves superior performance on all the tasks.