Detector Confidence Signals Presence Rather Than Occlusion in Cluttered Manipulation
Not on the map. Why?
Abstract (arXiv)
Occlude a named object until about an eighth of it remains visible, and an open-vocabulary detector's confidence that the object is present barely changes; as the clutter around it grows the confidence can even rise. On real video the detector still reports the object present in 99% of occluded frames, on another instance of the same category. This matters because that confidence is widely read as a visibility signal, used to threshold detections, evaluate open-vocabulary detectors, ground language, retrieve instances, and gate active perception. We audit whether it reflects occlusion by pairing every view with a geometry-segmentation oracle that gives detector-free ground-truth visibility. As true visibility falls from every scene to one in eight, the confidence stays nearly constant and uncorrelated with visibility, and the detector reports the target present in about nine of ten scenes, firing on same-category distractors: it signals that the category is present somewhere, not that the specific target is visible. The failure holds across three detectors (Grounding DINO, OWLv2, and Segment Anything Model 3), nine object categories, two simulators with different renderers and object sets, built and natural occlusion, and real video. Two consequences follow: a confidence-based metric understates the value of resolving occlusion by about ten times (8 against 88 points in our active-perception setting), and a confidence-based gate fires exactly when the object is hidden. No single-view signal we tried, including a realizable localization check, flags the occlusion, because the occluders sit where the target is. We connect the effect to detector miscalibration and object hallucination, release the controlled benchmark, and recommend target-grounded signals for gating and evaluation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 21 references that are in the map. First posted on arXiv 15 Jul 2026.
Builds on
20 of 21 references.
References · 20
- 2023Segment Anythingcited by 8,234
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
- 2021Grounded Language-Image Pre-trainingcited by 1,156
- 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
- 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
- 2023Scaling Open-Vocabulary Object Detectioncited by 343
- 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 305
- 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
8 more
- 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
- 2022Closed-Loop Next-Best-View Planning for Target-Driven Graspingcited by 19
- 2025VISO-Grasp: Vision-Language Informed Spatial Object-centric 6-DoF Active View Planning and Grasping in Clutter and Invisibilitycited by 11
- 2025Score the Steps, Not Just the Goal: VLM-Based Subgoal Evaluation for Robotic Manipulationcited by 8
- 2025ActiveGrasp: Information-Guided Active Grasping with Calibrated Energy-based Modelcited by 4
- 2025From Grounding to Manipulation: Case Studies of Foundation Model Integration in Embodied Robotic Systemscited by 3
- 2026Overcoming Visual Clutter in Vision Language Action Models via Concept-Gated Visual Distillationcited by 2
- 2026Sample-efficient Low-level Motion Planning for Robotic Manipulation Tasks via Zero-shot Transfer Learningcited by 1
Built on by
Not cited yet.