# Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline

arXiv: https://arxiv.org/abs/2309.14611 · HTML: https://mlsanity.com/paper/2309.14611/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 35 of its 49 references that are in the map.
- First posted on arXiv 26 Sep 2023.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 30 of 35)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Learning Spatio-Temporal Transformer for Visual Tracking](https://mlsanity.com/paper/2103.17154/) (2021)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [Ocean: Object-aware Anchor-free Tracking](https://mlsanity.com/paper/2006.10721/) (2020)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [Siamese Box Adaptive Network for Visual Tracking](https://mlsanity.com/paper/2003.06761/) (2020)
- [Visual Prompt Multi-Modal Tracking](https://mlsanity.com/paper/2303.10826/) (2023)
- [AiATrack: Attention in Attention for Transformer Visual Tracking](https://mlsanity.com/paper/2207.09603/) (2022)
- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [Know Your Surroundings: Exploiting Scene Information for Object Tracking](https://mlsanity.com/paper/2003.11014/) (2020)
- [Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking](https://mlsanity.com/paper/2203.05328/) (2022)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [Transforming Model Prediction for Tracking](https://mlsanity.com/paper/2203.11192/) (2022)
- [Object Tracking by Jointly Exploiting Frame and Event Domain](https://mlsanity.com/paper/2109.09052/) (2021)
- [Learn to Match: Automatic Matching Network Design for Visual Tracking](https://mlsanity.com/paper/2108.00803/) (2021)
- [Deep Learning for Visual Tracking: A Comprehensive Survey](https://mlsanity.com/paper/1912.00535/) (2019)
- [Siamese Instance Search for Tracking](https://mlsanity.com/paper/1605.05863/) (2016)
- [Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric](https://mlsanity.com/paper/2211.11010/) (2022)
- [Frame-Event Alignment and Fusion Network for High Frame Rate Tracking](https://mlsanity.com/paper/2305.15688/) (2023)

## Built on by (most cited first, 30 of 73)

- [Global Structure-Aware Diffusion Process for Low-Light Image Enhancement](https://mlsanity.com/paper/2310.17577/) (2023)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [State Space Model for New-Generation Network Alternative to Transformers: A Survey](https://mlsanity.com/paper/2404.09516/) (2024)
- [Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset](https://mlsanity.com/paper/2412.06647/) (2024)
- [XTrack: Multimodal Training Boosts RGB-X Video Object Trackers](https://mlsanity.com/paper/2405.17773/) (2024)
- [CRSOT: Cross-Resolution Object Tracking using Unaligned Frame and Event Cameras](https://mlsanity.com/paper/2401.02826/) (2024)
- [FRED: The Florence RGB-Event Drone Dataset](https://mlsanity.com/paper/2506.05163/) (2025)
- [WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmark](https://mlsanity.com/paper/2405.19818/) (2024)
- [What You Have is What You Track: Adaptive and Robust Multimodal Tracking](https://mlsanity.com/paper/2507.05899/) (2025)
- [Neuromorphic Drone Detection: an Event-RGB Multimodal Approach](https://mlsanity.com/paper/2409.16099/) (2024)
- [Event-based Tiny Object Detection: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2506.23575/) (2025)
- [Event Stream based Human Action Recognition: A High-Definition Benchmark Dataset and Algorithms](https://mlsanity.com/paper/2408.09764/) (2024)
- [MambaEVT: Event Stream based Visual Object Tracking using State Space Model](https://mlsanity.com/paper/2408.10487/) (2024)
- [E-Motion: Future Motion Simulation via Event Sequence Diffusion](https://mlsanity.com/paper/2410.08649/) (2024)
- [SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks](https://mlsanity.com/paper/2503.08703/) (2025)
- [Adaptive Perception for Unified Visual Multi-modal Object Tracking](https://mlsanity.com/paper/2502.06583/) (2025)
- [EventFly: Event Camera Perception from Ground to the Sky](https://mlsanity.com/paper/2503.19916/) (2025)
- [Drone Detection with Event Cameras](https://mlsanity.com/paper/2508.04564/) (2025)
- [Long-Term Visual Object Tracking with Event Cameras: An Associative Memory Augmented Tracker and A Benchmark Dataset](https://mlsanity.com/paper/2403.05839/) (2024)
- [EMatch: A Unified Framework for Event-based Optical Flow and Stereo Matching](https://mlsanity.com/paper/2407.21735/) (2024)
- [Fully Spiking Neural Networks for Unified Frame-Event Object Tracking](https://mlsanity.com/paper/2505.20834/) (2025)
- [Talk2Event: Grounded Understanding of Dynamic Scenes from Event Cameras](https://mlsanity.com/paper/2507.17664/) (2025)
- [Unleashing the Power of CNN and Transformer for Balanced RGB-Event Video Recognition](https://mlsanity.com/paper/2312.11128/) (2023)
- [MouseSIS: A Frames-and-Events Dataset for Space-Time Instance Segmentation of Mice](https://mlsanity.com/paper/2409.03358/) (2024)
- [Modeling State Shifting via Local-Global Distillation for Event-Frame Gaze Tracking](https://mlsanity.com/paper/2404.00548/) (2024)
- [CM3AE: A Unified RGB Frame and Event-Voxel/-Frame Pre-training Framework](https://mlsanity.com/paper/2504.12576/) (2025)
- [UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification](https://mlsanity.com/paper/2507.04638/) (2025)
- [Learning Efficient Meshflow and Optical Flow from Event Cameras](https://mlsanity.com/paper/2510.04111/) (2025)
- [Retain, Blend, and Exchange: A Quality-aware Spatial-Stereo Fusion Approach for Event Stream Recognition](https://mlsanity.com/paper/2406.18845/) (2024)
- [Towards Low-Latency Event Stream-based Visual Object Tracking: A Slow-Fast Approach](https://mlsanity.com/paper/2505.12903/) (2025)

## Abstract (arXiv)

Tracking using bio-inspired event cameras has drawn more and more attention in recent years. Existing works either utilize aligned RGB and event data for accurate tracking or directly learn an event-based tracker. The first category needs more cost for inference and the second one may be easily influenced by noisy events or sparse spatial resolution. In this paper, we propose a novel hierarchical knowledge distillation framework that can fully utilize multi-modal / multi-view information during training to facilitate knowledge transfer, enabling us to achieve high-speed and low-latency visual tracking during testing by using only event signals. Specifically, a teacher Transformer-based multi-modal tracking framework is first trained by feeding the RGB frame and event stream simultaneously. Then, we design a new hierarchical knowledge distillation strategy which includes pairwise similarity, feature representation, and response maps-based knowledge distillation to guide the learning of the student Transformer network. Moreover, since existing event-based tracking datasets are all low-resolution (346 × 260), we propose the first large-scale high-resolution (1280 × 720) dataset named EventVOT. It contains 1141 videos and covers a wide range of categories such as pedestrians, vehicles, UAVs, ping pongs, etc. Extensive experiments on both low-resolution (FE240hz, VisEvent, COESOT), and our newly proposed high-resolution EventVOT dataset fully validated the effectiveness of our proposed method. The dataset, evaluation toolkit, and source code are available on urlhttps://github.com/Event-AHU/EventVOTBenchmark