Reconstruction as a Bridge for Event-Based Visual Question Answering
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
Abstract (arXiv)
Integrating event cameras with Multimodal Large Language Models (MLLMs) promises general scene understanding in challenging visual conditions, yet requires navigating a trade-off between preserving the unique advantages of event data and ensuring compatibility with frame-based models. We address this challenge by using reconstruction as a bridge, proposing a straightforward Frame-based Reconstruction and Tokenization (FRT) method and designing an efficient Adaptive Reconstruction and Tokenization (ART) method that leverages event sparsity. For robust evaluation, we introduce EvQA, the first objective, real-world benchmark for event-based MLLMs, comprising 1,000 event-Q&A pairs from 22 public datasets. Our experiments demonstrate that our methods achieve state-of-the-art performance on EvQA, highlighting the significant potential of MLLMs in event-based vision.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 53 references that are in the map. First posted on arXiv 12 Dec 2025.
Builds on
43 of 53 references.
Closest ancestors · 4
Same direction, newest first
- 2025EventVL: Understand Event Streams via Multimodal Large Language Model23 Jan
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark28 Nov
- 2023EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding6 Aug
- 2023EventCLIP: Adapting CLIP for Event-based Object Recognition10 Jun
Other references · 39
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2025Qwen3 Technical Reportcited by 8,195
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
- 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
- 2023Large Language Models are not Fair Evaluatorscited by 1,050
27 more
- 2019Event-based Vision: A Surveycited by 1,010
- 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
- 2015Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccadescited by 308
- 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 297
- 2016The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAMcited by 203
- 2019Events-to-Video: Bringing Modern Computer Vision to Event Camerascited by 191
- 2021N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Camerascited by 91
- 2023Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baselinecited by 73
- 2022HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensorscited by 46
- 20233ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Networkcited by 30
- 2023A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wildcited by 27
- 2024eTraM: Event-based Traffic Monitoring Datasetcited by 24
- 2023EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms with Real-captured Hybrid Datasetcited by 19
- 2024Event-Based Eye Tracking. AIS 2024 Challenge Surveycited by 19
- 2023EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervisioncited by 17
- 2023Bullying10K: A Large-Scale Neuromorphic Dataset towards Privacy-Preserving Bullying Recognitioncited by 17
- 2024DailyDVS-200: A Comprehensive Benchmark Dataset for Event-Based Action Recognitioncited by 16
- 2025FRED: The Florence RGB-Event Drone Datasetcited by 16
- 2025Event-based Tiny Object Detection: A Benchmark Dataset and Baselinecited by 14
- 2023Low-power, Continuous Remote Behavioral Localization with Event Camerascited by 9
- 2025RGB-Event based Pedestrian Attribute Recognition: A Benchmark Dataset and An Asymmetric RWKV Fusion Frameworkcited by 7
- 2025MTevent: A Multi-Task Event Camera Dataset for 6D Pose Estimation and Moving Object Detectioncited by 6
- 2024MouseSIS: A Frames-and-Events Dataset for Space-Time Instance Segmentation of Micecited by 5
- 2025Event-Based Eye Tracking. 2025 Event-based Vision Workshopcited by 5
- 2025EventSTR: A Benchmark Dataset and Baselines for Event Stream based Scene Text Recognitioncited by 4
- 2025V2V: Scaling Event-Based Vision through Efficient Video-to-Voxel Simulationcited by 4
- 2025LET-US: Long Event-Text Understanding of Scenescited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025EventSTU: Event-Guided Efficient Spatio-Temporal Understanding for Video Large Language Models
- 2026EventFlash: Towards Efficient MLLMs for Event-Based Vision
- 2024EventGPT: Event Stream Understanding with Multimodal Large Language Models
- 2025Enhancing Long Video Question Answering with Scene-Localized Frame Grouping
- 2026Semantic Event Graphs for Long-Form Video Question Answering
- 2024Recoverable Compression: A Multimodal Vision Token Recovery Mechanism Guided by Text Information
- 2025An Empirical Study for Representations of Videos in Video Question Answering via MLLMs
- 2024PAR: Prompt-Aware Token Reduction Method for Efficient Large Multimodal Models