# FindVehicle and VehicleFinder: A NER dataset for natural language-based vehicle retrieval and a keyword-based cross-modal vehicle retrieval system

arXiv: https://arxiv.org/abs/2304.10893 · HTML: https://mlsanity.com/paper/2304.10893/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 62 references that are in the map.
- First posted on arXiv 21 Apr 2023.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [Few-NERD: A Few-Shot Named Entity Recognition Dataset](https://mlsanity.com/paper/2105.07464/) (2021)
- [Talk2Car: Taking Control of Your Self-Driving Car](https://mlsanity.com/paper/1909.10838/) (2019)
- [CyCLIP: Cyclic Contrastive Language-Image Pretraining](https://mlsanity.com/paper/2205.14459/) (2022)
- [UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking](https://mlsanity.com/paper/1511.04136/) (2015)
- [Unified Named Entity Recognition as Word-Word Relation Classification](https://mlsanity.com/paper/2112.10070/) (2021)
- [Portuguese Named Entity Recognition using BERT-CRF](https://mlsanity.com/paper/1909.10649/) (2019)
- [FLAT: Chinese NER Using Flat-Lattice Transformer](https://mlsanity.com/paper/2004.11795/) (2020)
- [FLERT: Document-Level Features for Named Entity Recognition](https://mlsanity.com/paper/2011.06993/) (2020)
- [CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptions](https://mlsanity.com/paper/2101.04741/) (2021)
- [Connecting Language and Vision for Natural Language-Based Vehicle Retrieval](https://mlsanity.com/paper/2105.14897/) (2021)
- [All You Can Embed: Natural Language based Vehicle Retrieval with Spatio-Temporal Transformers](https://mlsanity.com/paper/2106.10153/) (2021)
- [Trigger-GNN: A Trigger-Based Graph Neural Network for Nested Named Entity Recognition](https://mlsanity.com/paper/2204.05518/) (2022)
- [Symmetric Network with Spatial Relationship Modeling for Natural Language-based Vehicle Retrieval](https://mlsanity.com/paper/2206.10879/) (2022)

## Built on by (most cited first, 15 of 15)

- [Large Language Models for Generative Information Extraction: A Survey](https://mlsanity.com/paper/2312.17617/) (2023)
- [GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer](https://mlsanity.com/paper/2311.08526/) (2023)
- [Rethinking Negative Instances for Generative Named Entity Recognition](https://mlsanity.com/paper/2402.16602/) (2024)
- [Mirror: A Universal Framework for Various Information Extraction Tasks](https://mlsanity.com/paper/2311.05419/) (2023)
- [JsonTuning: Towards Generalizable, Robust, and Controllable Instruction Tuning](https://mlsanity.com/paper/2310.02953/) (2023)
- [IEPile: Unearthing Large-Scale Schema-Based Information Extraction Corpus](https://mlsanity.com/paper/2402.14710/) (2024)
- [Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition](https://mlsanity.com/paper/2406.11192/) (2024)
- [RoadSceneVQA: Benchmarking Visual Question Answering in Roadside Perception Systems for Intelligent Transportation System](https://mlsanity.com/paper/2511.18286/) (2025)
- [RUIE: Retrieval-based Unified Information Extraction using Large Language Model](https://mlsanity.com/paper/2409.11673/) (2024)
- [KnowCoder-X: Boosting Multilingual Information Extraction via Code](https://mlsanity.com/paper/2411.04794/) (2024)
- [Selecting and Merging: Towards Adaptable and Scalable Named Entity Recognition with Large Language Models](https://mlsanity.com/paper/2506.22813/) (2025)
- [Veri-Car: Towards Open-world Vehicle Information Retrieval](https://mlsanity.com/paper/2411.06864/) (2024)
- [Improving Natural Language Understanding for LLMs via Large-Scale Instruction Synthesis](https://mlsanity.com/paper/2502.03843/) (2025)
- [KnowCoder-V2: Deep Knowledge Analysis](https://mlsanity.com/paper/2506.06881/) (2025)
- [AutoSpecNER: A Fine-Grained Named Entity Recognition Dataset for Vehicle Specification Extraction](https://mlsanity.com/paper/2606.24387/) (2026)

## Abstract (arXiv)

Natural language (NL) based vehicle retrieval is a task aiming to retrieve a vehicle that is most consistent with a given NL query from among all candidate vehicles. Because NL query can be easily obtained, such a task has a promising prospect in building an interactive intelligent traffic system (ITS). Current solutions mainly focus on extracting both text and image features and mapping them to the same latent space to compare the similarity. However, existing methods usually use dependency analysis or semantic role-labelling techniques to find keywords related to vehicle attributes. These techniques may require a lot of pre-processing and post-processing work, and also suffer from extracting the wrong keyword when the NL query is complex. To tackle these problems and simplify, we borrow the idea from named entity recognition (NER) and construct FindVehicle, a NER dataset in the traffic domain. It has 42.3k labelled NL descriptions of vehicle tracks, containing information such as the location, orientation, type and colour of the vehicle. FindVehicle also adopts both overlapping entities and fine-grained entities to meet further requirements. To verify its effectiveness, we propose a baseline NL-based vehicle retrieval model called VehicleFinder. Our experiment shows that by using text encoders pre-trained by FindVehicle, VehicleFinder achieves 87.7% precision and 89.4% recall when retrieving a target vehicle by text command on our homemade dataset based on UA-DETRAC. The time cost of VehicleFinder is 279.35 ms on one ARM v8.2 CPU and 93.72 ms on one RTX A4000 GPU, which is much faster than the Transformer-based system. The dataset is open-source via the link https://github.com/GuanRunwei/FindVehicle, and the implementation can be found via the link https://github.com/GuanRunwei/VehicleFinder-CTIM.