# Chop & Learn: Recognizing and Generating Object-State Compositions

arXiv: https://arxiv.org/abs/2309.14339 · HTML: https://mlsanity.com/paper/2309.14339/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 76 references that are in the map.
- First posted on arXiv 25 Sep 2023.

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [End-to-End Learning of Visual Representations from Uncurated Instructional Videos](https://mlsanity.com/paper/1912.06430/) (2019)
- [COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysis](https://mlsanity.com/paper/1903.02874/) (2019)
- [RealFusion: 360deg Reconstruction of Any Object from a Single Image](https://mlsanity.com/paper/2302.10663/) (2023)
- [Videos as Space-Time Region Graphs](https://mlsanity.com/paper/1806.01810/) (2018)
- [Cross-task weakly supervised learning from instructional videos](https://mlsanity.com/paper/1903.08225/) (2019)
- [Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition](https://mlsanity.com/paper/1708.07632/) (2017)
- [EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations](https://mlsanity.com/paper/2209.13064/) (2022)
- [Cross-view Action Modeling, Learning and Recognition](https://mlsanity.com/paper/1405.2941/) (2014)
- [Task-Driven Modular Networks for Zero-Shot Compositional Learning](https://mlsanity.com/paper/1905.05908/) (2019)
- [Something-Else: Compositional Action Recognition with Spatial-Temporal Interaction Networks](https://mlsanity.com/paper/1912.09930/) (2019)
- [EGO-TOPO: Environment Affordances from Egocentric Video](https://mlsanity.com/paper/2001.04583/) (2020)

## Built on by (most cited first, 13 of 13)

- [GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videos](https://mlsanity.com/paper/2312.07322/) (2023)
- [Learning Object State Changes in Videos: An Open-World Perspective](https://mlsanity.com/paper/2312.11782/) (2023)
- [SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videos](https://mlsanity.com/paper/2403.01599/) (2024)
- [OSCaR: Object State Captioning and State Change Representation](https://mlsanity.com/paper/2402.17128/) (2024)
- [PICTURE: PhotorealistIC virtual Try-on from UnconstRained dEsigns](https://mlsanity.com/paper/2312.04534/) (2023)
- [Active Object Detection with Knowledge Aggregation and Distillation from Large Models](https://mlsanity.com/paper/2405.12509/) (2024)
- [What Changed and What Could Have Changed? State-Change Counterfactuals for Procedure-Aware Video Representation Learning](https://mlsanity.com/paper/2503.21055/) (2025)
- [STATUS Bench: A Rigorous Benchmark for Evaluating Object State Understanding in Vision-Language Models](https://mlsanity.com/paper/2510.22571/) (2025)
- [Anticipating Object State Changes in Long Procedural Videos](https://mlsanity.com/paper/2405.12789/) (2024)
- [Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL](https://mlsanity.com/paper/2410.02874/) (2024)
- [Vision-Based Mistake Analysis in Procedural Activities: A Review of Advances and Challenges](https://mlsanity.com/paper/2510.19292/) (2025)
- [Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?](https://mlsanity.com/paper/2607.08514/) (2026)
- [Cooking beyond Frames: A Stereo Event Camera Dataset in the Kitchen](https://mlsanity.com/paper/2608.04865/) (2026)

## Abstract (arXiv)

Recognizing and generating object-state compositions has been a challenging task, especially when generalizing to unseen compositions. In this paper, we study the task of cutting objects in different styles and the resulting object state changes. We propose a new benchmark suite Chop & Learn, to accommodate the needs of learning objects and different cut styles using multiple viewpoints. We also propose a new task of Compositional Image Generation, which can transfer learned cut styles to different objects, by generating novel object-state images. Moreover, we also use the videos for Compositional Action Recognition, and show valuable uses of this dataset for multiple video tasks. Project website: https://chopnlearn.github.io.