# STRAP: Structured Object Affordance Segmentation with Point Supervision

arXiv: https://arxiv.org/abs/2304.08492 · HTML: https://mlsanity.com/paper/2304.08492/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 69 references that are in the map.
- First posted on arXiv 17 Apr 2023.

## Builds on (most cited first, 30 of 46)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [Learning Human-Object Interactions by Graph Parsing Neural Networks](https://mlsanity.com/paper/1808.07962/) (2018)
- [Hand-Object Contact Consistency Reasoning for Human Grasps Generation](https://mlsanity.com/paper/2104.03304/) (2021)
- [HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes](https://mlsanity.com/paper/2210.09729/) (2022)
- [Deep Extreme Cut: From Extreme Points to Object Segmentation](https://mlsanity.com/paper/1711.09081/) (2017)
- [On Regularized Losses for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1803.09569/) (2018)
- [EgoTaskQA: Understanding Human Tasks in Egocentric Videos](https://mlsanity.com/paper/2210.03929/) (2022)
- [EGO-TOPO: Environment Affordances from Egocentric Video](https://mlsanity.com/paper/2001.04583/) (2020)
- [3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding](https://mlsanity.com/paper/2103.16397/) (2021)
- [ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact](https://mlsanity.com/paper/1904.03754/) (2019)
- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [Neural Cognitive Diagnosis for Intelligent Education Systems](https://mlsanity.com/paper/1908.08733/) (2019)
- [Human-centric Indoor Scene Synthesis Using Stochastic Grammar](https://mlsanity.com/paper/1808.08473/) (2018)
- [Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments](https://mlsanity.com/paper/1903.05690/) (2019)
- [Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense](https://mlsanity.com/paper/1909.01507/) (2019)
- [Synthesizing Diverse and Physically Stable Grasps with Arbitrary Hand Structures using Differentiable Force Closure Estimator](https://mlsanity.com/paper/2104.09194/) (2021)
- [Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image](https://mlsanity.com/paper/1808.02201/) (2018)
- [GenDexGrasp: Generalizable Dexterous Grasping](https://mlsanity.com/paper/2210.00722/) (2022)
- [LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activities](https://mlsanity.com/paper/2007.15781/) (2020)
- [O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning](https://mlsanity.com/paper/2106.15087/) (2021)
- [Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery](https://mlsanity.com/paper/2105.01047/) (2021)
- [Binge Watching: Scaling Affordance Learning from Sitcoms](https://mlsanity.com/paper/1804.03080/) (2018)

## Built on by (most cited first, 10 of 10)

- [One-Shot Open Affordance Learning with Foundation Models](https://mlsanity.com/paper/2311.17776/) (2023)
- [Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects](https://mlsanity.com/paper/2309.07473/) (2023)
- [GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistency](https://mlsanity.com/paper/2412.09511/) (2024)
- [PreAfford: Universal Affordance-Based Pre-Grasping for Diverse Objects and Environments](https://mlsanity.com/paper/2404.03634/) (2024)
- [Segmenting Object Affordances: Reproducibility and Sensitivity to Scale](https://mlsanity.com/paper/2409.01814/) (2024)
- [Visual Affordance Prediction: Survey and Reproducibility](https://mlsanity.com/paper/2505.05074/) (2025)
- [Informative Text-Image Alignment for Visual Affordance Learning with Foundation Models](https://mlsanity.com/paper/2509.17074/) (2025)
- [Probing and Bridging Geometry-Interaction Cues for Affordance Reasoning in Vision Foundation Models](https://mlsanity.com/paper/2602.20501/) (2026)
- [Zero-shot 2D Grounding with Novel Affordance Types](https://mlsanity.com/paper/2608.08929/) (2026)
- [Reproducible Multimodal Affordance Prediction](https://mlsanity.com/paper/2608.18317/) (2026)

## Abstract (arXiv)

With significant annotation savings, point supervision has been proven effective for numerous 2D and 3D scene understanding problems. This success is primarily attributed to the structured output space; i.e., samples with high spatial affinity tend to share the same labels. Sharing this spirit, we study affordance segmentation with point supervision, wherein the setting inherits an unexplored dual affinity-spatial affinity and label affinity. By label affinity, we refer to affordance segmentation as a multi-label prediction problem: A plate can be both holdable and containable. By spatial affinity, we refer to a universal prior that nearby pixels with similar visual features should share the same point annotation. To tackle label affinity, we devise a dense prediction network that enhances label relations by effectively densifying labels in a new domain (i.e., label co-occurrence). To address spatial affinity, we exploit a Transformer backbone for global patch interaction and a regularization loss. In experiments, we benchmark our method on the challenging CAD120 dataset, showing significant performance gains over prior methods.