# SBNet: Segmentation-based Network for Natural Language-based Vehicle Search

arXiv: https://arxiv.org/abs/2104.11589 · HTML: https://mlsanity.com/paper/2104.11589/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 24 references that are in the map.
- First posted on arXiv 22 Apr 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Segmentation from Natural Language Expressions](https://mlsanity.com/paper/1603.06180/) (2016)
- [Person Search with Natural Language Description](https://mlsanity.com/paper/1702.05729/) (2017)
- [MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment](https://mlsanity.com/paper/1812.00087/) (2018)
- [Actor and Action Video Segmentation from a Sentence](https://mlsanity.com/paper/1803.07485/) (2018)
- [One-Shot Object Detection with Co-Attention and Co-Excitation](https://mlsanity.com/paper/1911.12529/) (2019)
- [A survey of advances in vision-based vehicle re-identification](https://mlsanity.com/paper/1905.13258/) (2019)
- [Real-time Visual Object Tracking with Natural Language Description](https://mlsanity.com/paper/1907.11751/) (2019)
- [CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptions](https://mlsanity.com/paper/2101.04741/) (2021)
- [Spatio-temporal Person Retrieval via Natural Language Queries](https://mlsanity.com/paper/1704.07945/) (2017)
- [StRDAN: Synthetic-to-Real Domain Adaptation Network for Vehicle Re-Identification](https://mlsanity.com/paper/2004.12032/) (2020)
- [Siamese Natural Language Tracker: Tracking by Natural Language Descriptions with Siamese Trackers](https://mlsanity.com/paper/1912.02048/) (2019)
- [Trends in Vehicle Re-identification Past, Present, and Future: A Comprehensive Review](https://mlsanity.com/paper/2102.09744/) (2021)

## Built on by (most cited first, 3 of 3)

- [OMG: Observe Multiple Granularities for Natural Language-Based Vehicle Retrieval](https://mlsanity.com/paper/2204.08209/) (2022)
- [CLIPVehicle: A Unified Framework for Vision-based Vehicle Search](https://mlsanity.com/paper/2508.04120/) (2025)
- [BEVTraj: Map-Free End-to-End Trajectory Prediction in Bird's-Eye View with Deformable Attention and Sparse Goal Proposals](https://mlsanity.com/paper/2509.10080/) (2025)

## Abstract (arXiv)

Natural language-based vehicle retrieval is a task to find a target vehicle within a given image based on a natural language description as a query. This technology can be applied to various areas including police searching for a suspect vehicle. However, it is challenging due to the ambiguity of language descriptions and the difficulty of processing multi-modal data. To tackle this problem, we propose a deep neural network called SBNet that performs natural language-based segmentation for vehicle retrieval. We also propose two task-specific modules to improve performance: a substitution module that helps features from different domains to be embedded in the same space and a future prediction module that learns temporal information. SBnet has been trained using the CityFlow-NL dataset that contains 2,498 tracks of vehicles with three unique natural language descriptions each and tested 530 unique vehicle tracks and their corresponding query sets. SBNet achieved a significant improvement over the baseline in the natural language-based vehicle tracking track in the AI City Challenge 2021.