# Mitigating Query Selection Bias in Referring Video Object Segmentation

arXiv: https://arxiv.org/abs/2509.13722 · HTML: https://mlsanity.com/paper/2509.13722/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 17 Sep 2025.

## Builds on (most cited first, 23 of 23)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [Causal Intervention for Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2009.12547/) (2020)
- [MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions](https://mlsanity.com/paper/2308.08544/) (2023)
- [Video Object Segmentation with Language Referring Expressions](https://mlsanity.com/paper/1803.08006/) (2018)
- [VLT: Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2210.15871/) (2022)
- [Actor and Action Video Segmentation from a Sentence](https://mlsanity.com/paper/1803.07485/) (2018)
- [Cross-Modal Progressive Comprehension for Referring Segmentation](https://mlsanity.com/paper/2105.07175/) (2021)
- [Feature Pyramid Transformer](https://mlsanity.com/paper/2007.09451/) (2020)
- [VITA: Video Instance Segmentation via Object Token Association](https://mlsanity.com/paper/2206.04403/) (2022)
- [Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentation](https://mlsanity.com/paper/2404.03645/) (2024)
- [Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation](https://mlsanity.com/paper/2305.16318/) (2023)
- [Towards Robust Referring Video Object Segmentation with Cyclic Relational Consensus](https://mlsanity.com/paper/2207.01203/) (2022)
- [Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation](https://mlsanity.com/paper/2106.01061/) (2021)
- [Multi-Attention Network for Compressed Video Referring Object Segmentation](https://mlsanity.com/paper/2207.12622/) (2022)
- [UniVS: Unified and Universal Video Segmentation with Prompts as Queries](https://mlsanity.com/paper/2402.18115/) (2024)
- [UniRef++: Segment Every Reference Object in Spatial and Temporal Spaces](https://mlsanity.com/paper/2312.15715/) (2023)
- [ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentation](https://mlsanity.com/paper/2103.10702/) (2021)
- [Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation](https://mlsanity.com/paper/2204.02547/) (2022)
- [Boosting Weakly-Supervised Image Segmentation via Representation, Transform, and Compensator](https://mlsanity.com/paper/2309.00871/) (2023)

## Abstract (arXiv)

Recently, query-based methods have achieved remarkable performance in Referring Video Object Segmentation (RVOS) by using textual static object queries to drive cross-modal alignment. However, these static queries are easily misled by distractors with similar appearance or motion, resulting in query selection bias. To address this issue, we propose Triple Query Former (TQF), which factorizes the referring query into three specialized components: an appearance query for static attributes, an intra-frame interaction query for spatial relations, and an inter-frame motion query for temporal association. Instead of relying solely on textual embeddings, our queries are dynamically constructed by integrating both linguistic cues and visual guidance. Furthermore, we introduce two motion-aware aggregation modules that enhance object token representations: Intra-frame Interaction Aggregation incorporates position-aware interactions among objects within a single frame, while Inter-frame Motion Aggregation leverages trajectory-guided alignment across frames to ensure temporal coherence. Extensive experiments on multiple RVOS benchmarks demonstrate the advantages of TQF and the effectiveness of our structured query design and motion-aware aggregation modules.