mlsanityData to 14 Oct 2026

Talk2Event: Grounded Understanding of Dynamic Scenes from Event Cameras

arXiv 2507.17664 ↗First posted 23 Jul 2025cs.CV cs.ROcited by 6

Not on the map. Why?

Abstract (arXiv)

Event cameras offer microsecond-level latency and robustness to motion blur, making them ideal for understanding dynamic environments. Yet, connecting these asynchronous streams to human language remains an open challenge. We introduce Talk2Event, the first large-scale benchmark for language-driven object grounding in event-based perception. Built from real-world driving data, we provide over 30,000 validated referring expressions, each enriched with four grounding attributes -- appearance, status, relation to viewer, and relation to other objects -- bridging spatial, temporal, and relational reasoning. To fully exploit these cues, we propose EventRefer, an attribute-aware grounding framework that dynamically fuses multi-attribute representations through a Mixture of Event-Attribute Experts (MoEE). Our method adapts to different modalities and scene dynamics, achieving consistent gains over state-of-the-art baselines in event-only, frame-only, and event-frame fusion settings. We hope our dataset and approach will establish a foundation for advancing multimodal, temporally-aware, and language-driven perception in real-world robotics and autonomy.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 98 of its 119 references that are in the map. First posted on arXiv 23 Jul 2025.

Builds on

98 of 119 references.

References · 98

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2016Modeling Context in Referring Expressionscited by 1,063
  8. 2019Event-based Vision: A Surveycited by 1,016
  9. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  10. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  11. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  12. 2019High Speed and High Dynamic Range Video with an Event Cameracited by 345
38 more
  1. 2023Scaling Open-Vocabulary Object Detectioncited by 343
  2. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 328
  3. 2018Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotioncited by 307
  4. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 305
  5. 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 303
  6. 2018The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perceptioncited by 254
  7. 2018EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Camerascited by 224
  8. 2019End-to-End Learning of Representations for Asynchronous Event-Based Datacited by 217
  9. 2018Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Carscited by 200
  10. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 195
  11. 2020Learning to Detect Objects with a 1 Megapixel Event Cameracited by 186
  12. 2023Towards Open Vocabulary Learning: A Surveycited by 166
  13. 2019Video to Events: Recycling Video Datasets for Event Camerascited by 165
  14. 2022Recurrent Vision Transformers for Object Detection with Event Camerascited by 159
  15. 2021E-RAFT: Dense Optical Flow from Event Camerascited by 152
  16. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  17. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 141
  18. 2021Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Cloudscited by 128
  19. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  20. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 123
  21. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 122
  22. 2017Learning Two-Branch Neural Networks for Image-Text Matching Taskscited by 119
  23. 20223D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selectioncited by 110
  24. 2021Context-aware Biaffine Localizing Network for Temporal Sentence Groundingcited by 107
  25. 2018Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networkscited by 102
  26. 2022TubeDETR: Spatio-Temporal Video Grounding with Transformerscited by 98
  27. 2018EV-SegNet: Semantic Segmentation for Event-based Camerascited by 93
  28. 2023Referring Multi-Object Trackingcited by 93
  29. 2021N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Camerascited by 92
  30. 2022Event-aided Direct Sparse Odometrycited by 92
  31. 2023Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarkscited by 90
  32. 2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
  33. 2020Learning Monocular Dense Depth from Eventscited by 85
  34. 2017DDD17: End-To-End DAVIS Driving Datasetcited by 84
  35. 2022ESS: Learning Event-based Semantic Segmentation from Still Imagescited by 82
  36. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  37. 2019Zero-Shot Grounding of Objects from Natural Language Queriescited by 79
  38. 2024State Space Models for Event Camerascited by 75

50 most cited of 98 All 98 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learningcited by 21
  2. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  3. 2025Learning to Remove Lens Flare in Event Cameracited by 2
  4. 2026EventDrive: Event Cameras for Vision-Language Driving Intelligencecited by 2
  5. 2025See No Evil: Adversarial Attacks Against Linguistic-Visual Association in Referring Multi-Object Tracking Systemscited by 1
  6. 2025EventBench: Towards Comprehensive Benchmarking of Event-based MLLMscited by 1