# Coarse or Fine? Recognising Action End States without Labels

arXiv: https://arxiv.org/abs/2405.07723 · HTML: https://mlsanity.com/paper/2405.07723/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018)
- [Learning Graph Embeddings for Compositional Zero-shot Learning](https://mlsanity.com/paper/2102.01987/) (2021)
- [Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2101.12609/) (2021)
- [Symmetry and Group in Attribute-Object Compositions](https://mlsanity.com/paper/2004.00587/) (2020)
- [Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding](https://mlsanity.com/paper/2203.00867/) (2022)
- [Disentangling Visual Embeddings for Attributes and Objects](https://mlsanity.com/paper/2205.08536/) (2022)
- [Breaking the "Object" in Video Object Segmentation](https://mlsanity.com/paper/2212.06200/) (2022)
- [Learning Conditional Attributes for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2305.17940/) (2023)
- [Action Modifiers: Learning from Adverbs in Instructional Videos](https://mlsanity.com/paper/1912.06617/) (2019)
- [Learning Action Changes by Measuring Verb-Adverb Textual Relationships](https://mlsanity.com/paper/2303.15086/) (2023)
- [Video-adverb retrieval with compositional adverb-action embeddings](https://mlsanity.com/paper/2309.15086/) (2023)

## Abstract (arXiv)

We focus on the problem of recognising the end state of an action in an image, which is critical for understanding what action is performed and in which manner. We study this focusing on the task of predicting the coarseness of a cut, i.e., deciding whether an object was cut "coarsely" or "finely". No dataset with these annotated end states is available, so we propose an augmentation method to synthesise training data. We apply this method to cutting actions extracted from an existing action recognition dataset. Our method is object agnostic, i.e., it presupposes the location of the object but not its identity. Starting from less than a hundred images of a whole object, we can generate several thousands images simulating visually diverse cuts of different coarseness. We use our synthetic data to train a model based on UNet and test it on real images showing coarsely/finely cut objects. Results demonstrate that the model successfully recognises the end state of the cutting action despite the domain gap between training and testing, and that the model generalises well to unseen objects.