# Open-TransMind: A New Baseline and Benchmark for 1st Foundation Model Challenge of Intelligent Transportation

arXiv: https://arxiv.org/abs/2304.06051 · HTML: https://mlsanity.com/paper/2304.06051/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection](https://mlsanity.com/paper/2204.05575/) (2022)
- [HAT: Hardware-Aware Transformers for Efficient Natural Language Processing](https://mlsanity.com/paper/2005.14187/) (2020)
- [TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents](https://mlsanity.com/paper/1811.02146/) (2018)
- [HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis](https://mlsanity.com/paper/1709.09930/) (2017)
- [Rope3D: TheRoadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task](https://mlsanity.com/paper/2203.13608/) (2022)
- [Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classification](https://mlsanity.com/paper/2101.06635/) (2021)
- [UFO: Unified Feature Optimization](https://mlsanity.com/paper/2207.10341/) (2022)

## Built on by (most cited first, 14 of 14)

- [ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions](https://mlsanity.com/paper/2403.07392/) (2024)
- [C2Former: Calibrated and Complementary Transformer for RGB-Infrared Object Detection](https://mlsanity.com/paper/2306.16175/) (2023)
- [Visual Tuning](https://mlsanity.com/paper/2305.06061/) (2023)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [UniRGB-IR: A Unified Framework for Visible-Infrared Semantic Tasks via Adapter Tuning](https://mlsanity.com/paper/2404.17360/) (2024)
- [The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective](https://mlsanity.com/paper/2407.08583/) (2024)
- [Approaching Outside: Scaling Unsupervised 3D Object Detection from 2D Scene](https://mlsanity.com/paper/2407.08569/) (2024)
- [Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagery](https://mlsanity.com/paper/2511.11470/) (2025)
- [ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models](https://mlsanity.com/paper/2604.12251/) (2026)
- [You Only Gaussian Once: Controllable 3D Gaussian Splatting for Ultra-Densely Sampled Scenes](https://mlsanity.com/paper/2604.21400/) (2026)
- [OBSR: Open Benchmark for Spatial Representations](https://mlsanity.com/paper/2510.05879/) (2025)
- [EMDFNet: Efficient Multi-scale and Diverse Feature Network for Traffic Sign Detection](https://mlsanity.com/paper/2408.14189/) (2024)
- [SMDDFNet: State-space Modeling and Dynamic Dual Fusion Network for Traffic Sign Detection](https://mlsanity.com/paper/2505.05491/) (2025)
- [STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understanding](https://mlsanity.com/paper/2608.11699/) (2026)

## Abstract (arXiv)

With the continuous improvement of computing power and deep learning algorithms in recent years, the foundation model has grown in popularity. Because of its powerful capabilities and excellent performance, this technology is being adopted and applied by an increasing number of industries. In the intelligent transportation industry, artificial intelligence faces the following typical challenges: few shots, poor generalization, and a lack of multi-modal techniques. Foundation model technology can significantly alleviate the aforementioned issues. To address these, we designed the 1st Foundation Model Challenge, with the goal of increasing the popularity of foundation model technology in traffic scenarios and promoting the rapid development of the intelligent transportation industry. The challenge is divided into two tracks: all-in-one and cross-modal image retrieval. Furthermore, we provide a new baseline and benchmark for the two tracks, called Open-TransMind. According to our knowledge, Open-TransMind is the first open-source transportation foundation model with multi-task and multi-modal capabilities. Simultaneously, Open-TransMind can achieve state-of-the-art performance on detection, classification, and segmentation datasets of traffic scenarios. Our source code is available at https://github.com/Traffic-X/Open-TransMind.