mlsanityData to 14 Oct 2026

Venice-H1: Failure-Aware Query Re-Ranking with Multi-Scale Grid Signatures for Referring Image Segmentation

arXiv 2606.22546 ↗First posted 21 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern Referring Image Segmentation (RIS) systems generate multiple candidate masks per expression but rely on a simple heuristic--typically the argmax detection score--to select the final output. We identify query selection as a failure-case bottleneck: although heuristic selection succeeds on 82-93% of samples, the residual 7-18% of failures dominate the error budget, leaving a best-query selection gap of 3-11% mIoU. We introduce Venice-H1, a lightweight, backbone-decoupled post-hoc re-ranking module that encodes each candidate through multi-scale grid signatures--compact spatial descriptors pooled onto 4x4, 8x8, and 16x16 grids--and feeds them to a Transformer-based re-ranker with a Failure Gate (ROCAUC 0.78-0.82) that intervenes only when the default choice is likely suboptimal. Instantiated on DeRIS-L and DeRIS-B, Venice-H1 achieves delta_fail of +1.40 and +0.89 mIoU with strictly positive 95% CIs on all 16/16 (split, backbone) pairs and harmful-switch rates below 0.53%. Zero-shot transfer to medical referring segmentation (MS-CXR, M3D-RefSeg-2D) yields +1.16 and +0.51 mIoU without RIS-backbone fine-tuning. The module adds approximately 11.3M parameters and under 1 ms latency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 34 references that are in the map. First posted on arXiv 21 Jun 2026.

Builds on

26 of 34 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2023Segment Anythingcited by 8,234
  4. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  5. 2016Modeling Context in Referring Expressionscited by 1,063
  6. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  7. 2017Selective Classification for Deep Neural Networkscited by 852
  8. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 720
  9. 2017Soft-NMS -- Improving Object Detection With One Line of Codecited by 382
  10. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  11. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  12. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
14 more
  1. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
  2. 2019Mask Scoring R-CNNcited by 266
  3. 2018Acquisition of Localization Confidence for Accurate Object Detectioncited by 176
  4. 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 162
  5. 2020Referring Image Segmentation via Cross-Modal Progressive Comprehensioncited by 131
  6. 2017Learning non-maximum suppressioncited by 113
  7. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  8. 2020Linguistic Structure Guided Context Modeling for Referring Image Segmentationcited by 102
  9. 2020Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cellscited by 90
  10. 2024OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modelingcited by 28
  11. 2025Multi-task Visual Grounding with Coarse-to-Fine Consistency Constraintscited by 17
  12. 2025DeRIS: Decoupling Perception and Cognition for Enhanced Referring Image Segmentation through Loopback Synergycited by 13
  13. 2023Self-Supervised Learning of Representations for Space Generates Multi-Modular Grid Cellscited by 8
  14. 2023Segment, Select, Correct: A Framework for Weakly-Supervised Referring Segmentationcited by 3

Built on by

Not cited yet.