mlsanityData to 14 Oct 2026

What Object Should I Use? - Task Driven Object Detection

arXiv 1904.03000 ↗First posted 5 Apr 2019cs.CV cs.LG cs.ROcited by 22

Not on the map. Why?

Abstract (arXiv)

When humans have to solve everyday tasks, they simply pick the objects that are most suitable. While the question which object should one use for a specific task sounds trivial for humans, it is very difficult to answer for robots or other autonomous systems. This issue, however, is not addressed by current benchmarks for object detection that focus on detecting object categories. We therefore introduce the COCO-Tasks dataset which comprises about 40,000 images where the most suitable objects for 14 tasks have been annotated. We furthermore propose an approach that detects the most suitable objects for a given task. The approach builds on a Gated Graph Neural Network to exploit the appearance of each object as well as the global context of all present objects in the scene. In our experiments, we show that the proposed approach outperforms other approaches that are evaluated on the dataset like classification or ranking approaches.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 2; 2024: 7; 2025: 9; 2026: 2.

22 papers.

Sort:

By area:

  1. 2023CoTDet: Affordance Knowledge Prompting for Task Driven Object Detectioncited by 36
  2. 2022TOIST: Task Oriented Instance Segmentation Transformer with Noun-Pronoun Distillationcited by 25
  3. 2024KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Modelscited by 24
  4. 2021Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsingcited by 22
  5. 2024TaskCLIP: Extend Large Vision-Language Model for Task Oriented Object Detectioncited by 21
  6. 2024Intent3D: 3D Object Detection in RGB-D Scans Based on Human Intentioncited by 16
  7. 2023RIO: A Benchmark for Reasoning Intention-Oriented Objects in Open Environmentscited by 15
  8. 2024VLTP: Vision-Language Guided Token Pruning for Task-Oriented Segmentationcited by 11
  9. 2025Visual Intention Grounding for Egocentric Assistantscited by 11
  10. 2025Generating 6DoF Object Manipulation Trajectories from Action Description in Egocentric Visioncited by 11
  11. 2024Beyond Literal Descriptions: Understanding and Locating Open-World Objects Aligned with Human Intentionscited by 8
  12. 2025LLaPa: A Vision-Language Model Framework for Counterfactual-Aware Procedural Planningcited by 6
10 more
  1. 2025Afford-X: Generalizable and Slim Affordance Reasoning for Task-oriented Manipulationcited by 3
  2. 2025Beyond Object Categories: Multi-Attribute Reference Understanding for Visual Groundingcited by 3
  3. 2025Back to Fundamentals: Low-Level Visual Features Guided Progressive Token Pruningcited by 1
  4. 2025Visual Affordance Prediction: Survey and Reproducibilitycited by 1
  5. 2025CRAFT: A Neuro-Symbolic Framework for Visual Functional Affordance Groundingcited by 1
  6. 2024Affordance Perception by a Knowledge-Guided Vision-Language Model with Efficient Error Correctioncited by 0
  7. 2024Leverage Task Context for Object Affordance Rankingcited by 0
  8. 2025CRAFT-E: A Neuro-Symbolic Framework for Embodied Affordance Groundingcited by 0
  9. 2026FUSE: Active Functional Affordance Grounding through Adaptive Semantic-Geometric Evidence Acquisitioncited by 0
  10. 2026Vague2Detect: Handling Ambiguous Prompts in Knowledge-Based Open-World Detectioncited by 0