# CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptions

arXiv: https://arxiv.org/abs/2101.04741 · HTML: https://mlsanity.com/paper/2101.04741/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 21 references that are in the map.
- First posted on arXiv 12 Jan 2021.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [A Fast and Accurate One-Stage Approach to Visual Grounding](https://mlsanity.com/paper/1908.06354/) (2019)
- [DenseCap: Fully Convolutional Localization Networks for Dense Captioning](https://mlsanity.com/paper/1511.07571/) (2015)
- [MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment](https://mlsanity.com/paper/1812.00087/) (2018)
- [Natural Language Object Retrieval](https://mlsanity.com/paper/1511.04164/) (2015)
- [CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification](https://mlsanity.com/paper/1903.09254/) (2019)
- [Learning to Compose and Reason with Language Tree Structures for Visual Grounding](https://mlsanity.com/paper/1906.01784/) (2019)
- [Dialog-based Interactive Image Retrieval](https://mlsanity.com/paper/1805.00145/) (2018)
- [Real-time Visual Object Tracking with Natural Language Description](https://mlsanity.com/paper/1907.11751/) (2019)
- [Siamese Natural Language Tracker: Tracking by Natural Language Descriptions with Siamese Trackers](https://mlsanity.com/paper/1912.02048/) (2019)

## Built on by (most cited first, 26 of 26)

- [Multi-View Transformer for 3D Visual Grounding](https://mlsanity.com/paper/2204.02174/) (2022)
- [Vision Language Models in Autonomous Driving: A Survey and Outlook](https://mlsanity.com/paper/2310.14414/) (2023)
- [SAT: 2D Semantics Assisted Training for 3D Visual Grounding](https://mlsanity.com/paper/2105.11450/) (2021)
- [Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving](https://mlsanity.com/paper/2312.03661/) (2023)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Visual Programming for Zero-shot Open-Vocabulary 3D Visual Grounding](https://mlsanity.com/paper/2311.15383/) (2023)
- [The 5th AI City Challenge](https://mlsanity.com/paper/2104.12233/) (2021)
- [The 6th AI City Challenge](https://mlsanity.com/paper/2204.10380/) (2022)
- [The 7th AI City Challenge](https://mlsanity.com/paper/2304.07500/) (2023)
- [EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Driving](https://mlsanity.com/paper/2402.18302/) (2024)
- [Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding](https://mlsanity.com/paper/2307.09267/) (2023)
- [FindVehicle and VehicleFinder: A NER dataset for natural language-based vehicle retrieval and a keyword-based cross-modal vehicle retrieval system](https://mlsanity.com/paper/2304.10893/) (2023)
- [Connecting Language and Vision for Natural Language-Based Vehicle Retrieval](https://mlsanity.com/paper/2105.14897/) (2021)
- [Chain-of-Thought for Autonomous Driving: A Comprehensive Survey and Future Prospects](https://mlsanity.com/paper/2505.20223/) (2025)
- [Prospective Role of Foundation Models in Advancing Autonomous Vehicles](https://mlsanity.com/paper/2405.02288/) (2023)
- [ProxyTransformation: Preshaping Point Cloud Manifold With Proxy Attention For 3D Visual Grounding](https://mlsanity.com/paper/2502.19247/) (2025)
- [SBNet: Segmentation-based Network for Natural Language-based Vehicle Search](https://mlsanity.com/paper/2104.11589/) (2021)
- [All You Can Embed: Natural Language based Vehicle Retrieval with Spatio-Temporal Transformers](https://mlsanity.com/paper/2106.10153/) (2021)
- [OMG: Observe Multiple Granularities for Natural Language-Based Vehicle Retrieval](https://mlsanity.com/paper/2204.08209/) (2022)
- [Weakly-Supervised 3D Visual Grounding based on Visual Language Alignment](https://mlsanity.com/paper/2312.09625/) (2023)
- [Symmetric Network with Spatial Relationship Modeling for Natural Language-based Vehicle Retrieval](https://mlsanity.com/paper/2206.10879/) (2022)
- [Robust 2D/3D Vehicle Parsing in CVIS](https://mlsanity.com/paper/2103.06432/) (2021)
- [VQPy: An Object-Oriented Approach to Modern Video Analytics](https://mlsanity.com/paper/2311.01623/) (2023)
- [T2I-VeRW: Part-level Fine-grained Perception for Text-to-Image Vehicle Retrieval](https://mlsanity.com/paper/2605.06012/) (2026)
- [Deeply Interleaved Text-Image Contexts for Multimodal LLMs Assessment](https://mlsanity.com/paper/2609.02573/) (2026)
- [ReWorld-Track: A Recursive Event World Model for Language-Guided Multi-Camera Tracking](https://mlsanity.com/paper/2609.36677/) (2026)

## Abstract (arXiv)

Natural Language (NL) descriptions can be one of the most convenient or the only way to interact with systems built to understand and detect city scale traffic patterns and vehicle-related events. In this paper, we extend the widely adopted CityFlow Benchmark with NL descriptions for vehicle targets and introduce the CityFlow-NL Benchmark. The CityFlow-NL contains more than 5,000 unique and precise NL descriptions of vehicle targets, making it the first multi-target multi-camera tracking with NL descriptions dataset to our knowledge. Moreover, the dataset facilitates research at the intersection of multi-object tracking, retrieval by NL descriptions, and temporal localization of events. In this paper, we focus on two foundational tasks: the Vehicle Retrieval by NL task and the Vehicle Tracking by NL task, which take advantage of the proposed CityFlow-NL benchmark and provide a strong basis for future research on the multi-target multi-camera tracking by NL description task.