mlsanityData to 14 Oct 2026

Saliency-Motion Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation

arXiv 2504.05904 ↗First posted 8 Apr 2025cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent mainstream unsupervised video object segmentation (UVOS) motion-appearance approaches use either the bi-encoder structure to separately encode motion and appearance features, or the uni-encoder structure for joint encoding. However, these methods fail to properly balance the motion-appearance relationship. Consequently, even with complex fusion modules for motion-appearance integration, the extracted suboptimal features degrade the models' overall performance. Moreover, the quality of optical flow varies across scenarios, making it insufficient to rely solely on optical flow to achieve high-quality segmentation results. To address these challenges, we propose the Saliency-Motion guided Trunk-Collateral Network (SMTC-Net), which better balances the motion-appearance relationship and incorporates model's intrinsic saliency information to enhance segmentation performance. Specifically, considering that optical flow maps are derived from RGB images, they share both commonalities and differences. Accordingly, we propose a novel Trunk-Collateral structure for motion-appearance UVOS. The shared trunk backbone captures the motion-appearance commonality, while the collateral branch learns the uniqueness of motion features. Furthermore, an Intrinsic Saliency guided Refinement Module (ISRM) is devised to efficiently leverage the model's intrinsic saliency information to refine high-level features, and provide pixel-level guidance for motion-appearance fusion, thereby enhancing performance without additional input. Experimental results show that SMTC-Net achieved state-of-the-art performance on three UVOS datasets ( 89.2% J&F on DAVIS-16, 76% J on YouTube-Objects, 86.4% J on FBMS ) and four standard video salient object detection (VSOD) benchmarks with the notable increase, demonstrating its effectiveness and superiority over previous methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 53 of its 87 references that are in the map. First posted on arXiv 8 Apr 2025.

Builds on

53 of 87 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  10. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  11. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  12. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
38 more
  1. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  2. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  3. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  4. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  5. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  6. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  7. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  8. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  9. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
  10. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  11. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 362
  12. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  13. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 320
  14. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 310
  15. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  16. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 232
  17. 2020AdapterDrop: On the Efficiency of Adapters in Transformerscited by 224
  18. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 203
  19. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  20. 2020See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networkscited by 138
  21. 2020Zero-Shot Video Object Segmentation via Attentive Graph Neural Networkscited by 84
  22. 2021Full-Duplex Strategy for Video Object Segmentationcited by 78
  23. 2023Lifelong Language Pretraining with Distribution-Specialized Expertscited by 66
  24. 2020Making a Case for 3D Convolutions for Object Segmentation in Videoscited by 56
  25. 2019Anchor Diffusion for Unsupervised Video Object Segmentationcited by 50
  26. 2023Dense Network Expansion for Class Incremental Learningcited by 45
  27. 2020Learning Discriminative Feature with CRF for Unsupervised Video Object Segmentationcited by 36
  28. 2020F2Net: Learning to Focus on the Foreground for Unsupervised Video Object Segmentationcited by 32
  29. 2022Unsupervised Video Object Segmentation via Prototype Memory Networkcited by 30
  30. 2018Video Object Segmentation using Teacher-Student Adaptation in a Human Robot Interaction (HRI) Settingcited by 27
  31. 2021Iteratively Selecting an Easy Reference Frame Makes Unsupervised Video Object Segmentation Easiercited by 27
  32. 2024Depth-aware Test-Time Training for Zero-shot Video Object Segmentationcited by 27
  33. 2021Efficient Test Time Adapter Ensembling for Low-resource Language Varietiescited by 26
  34. 2023Guided Slot Attention for Unsupervised Video Object Segmentationcited by 19
  35. 2022Dual Prototype Attention for Unsupervised Video Object Segmentationcited by 14
  36. 2023SimulFlow: Simultaneously Extracting Feature and Identifying Target for Unsupervised Video Object Segmentationcited by 12
  37. 2023Isomer: Isomerous Transformer for Zero-shot Video Object Segmentationcited by 10
  38. 2023Co-attention Propagation Network for Zero-Shot Video Object Segmentationcited by 9

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.