mlsanityData to 14 Oct 2026

Label-Free Event-based Object Recognition via Joint Learning with Image Reconstruction from Events

arXiv 2308.09383 ↗First posted 18 Aug 2023cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

Recognizing objects from sparse and noisy events becomes extremely difficult when paired images and category labels do not exist. In this paper, we study label-free event-based object recognition where category labels and paired images are not available. To this end, we propose a joint formulation of object recognition and image reconstruction in a complementary manner. Our method first reconstructs images from events and performs object recognition through Contrastive Language-Image Pre-training (CLIP), enabling better recognition through a rich context of images. Since the category information is essential in reconstructing images, we propose category-guided attraction loss and category-agnostic repulsion loss to bridge the textual features of predicted categories and the visual features of reconstructed images using CLIP. Moreover, we introduce a reliable data sampling strategy and local-global reconstruction consistency to boost joint learning of two tasks. To enhance the accuracy of prediction and quality of reconstruction, we also propose a prototype-based approach using unpaired images. Extensive experiments demonstrate the superiority of our method and its extensibility for zero-shot object recognition. Our project code is available at urlhttps://github.com/Chohoonhee/Ev-LaFOR.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 38 of its 74 references that are in the map. First posted on arXiv 18 Aug 2023.

Builds on

38 of 74 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2016Improved Techniques for Training GANscited by 2,970
  9. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  10. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 674
  11. 2019High Speed and High Dynamic Range Video with an Event Cameracited by 345
  12. 2015Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccadescited by 314
26 more
  1. 2018Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotioncited by 307
  2. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  3. 2018HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classificationcited by 218
  4. 2019End-to-End Learning of Representations for Asynchronous Event-Based Datacited by 217
  5. 2016Training Deep Spiking Neural Networks using Backpropagationcited by 205
  6. 2018Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Carscited by 200
  7. 2019Events-to-Video: Bringing Modern Computer Vision to Event Camerascited by 193
  8. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 181
  9. 2021ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmeticcited by 150
  10. 2020Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancycited by 125
  11. 2021Event-Based Fusion for Motion Deblurring with Cross-modal Attentioncited by 113
  12. 2020Event-based Asynchronous Sparse Convolutional Networkscited by 104
  13. 2016Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequencescited by 99
  14. 2018Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networkscited by 97
  15. 2021N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Camerascited by 92
  16. 2022Secrets of Event-Based Optical Flowcited by 92
  17. 2019Graph-Based Object Classification for Neuromorphic Vision Sensingcited by 85
  18. 2018Continuous-time Intensity Estimation Using Event Camerascited by 68
  19. 2015HFirst: A Temporal Approach to Object Recognitioncited by 50
  20. 2016Real-Time Intensity-Image Reconstruction for Event Cameras Using Manifold Regularisationcited by 49
  21. 2021EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillationcited by 47
  22. 2022Masked Event Modeling: Self-Supervised Pretraining for Event Camerascited by 41
  23. 2021Image-Based CLIP-Guided Essence Transfercited by 39
  24. 2021Dual Transfer Learning for Event-based End-task Prediction via Pluggable Event to Image Translationcited by 32
  25. 2018PRED18: Dataset and Further Experiments with DAVIS Event Camera in Predator-Prey Robot Chasingcited by 7
  26. 2018Learn to See by Events: Color Frame Synthesis from Event and RGB Camerascited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 11; 2025: 9; 2026: 6.

28 papers.

Sort:

By area:

  1. 2024Recent Event Camera Innovations: A Surveycited by 68
  2. 2024OpenESS: Event-based Semantic Scene Understanding with Open Vocabulariescited by 38
  3. 2024ExACT: Language-guided Conceptual Reasoning and Uncertainty Estimation for Event-based Action Recognition and Morecited by 33
  4. 2023LEOD: Label-Efficient Object Detection for Event Camerascited by 24
  5. 2024EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognitioncited by 23
  6. 2025EventVL: Understand Event Streams via Multimodal Large Language Modelcited by 13
  7. 2024Temporal Event Stereo via Joint Learning with Stereoscopic Flowcited by 12
  8. 2024Towards Robust Event-based Networks for Nighttime via Unpaired Day-to-Night Event Translationcited by 9
  9. 2024EZSR: Event-based Zero-Shot Recognitioncited by 8
  10. 2024Finding Meaning in Points: Weakly Supervised Semantic Segmentation for Event Camerascited by 6
  11. 2025Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Applicationcited by 6
  12. 2025Talk2Event: Grounded Understanding of Dynamic Scenes from Event Camerascited by 6
16 more
  1. 2023Unleashing the Power of CNN and Transformer for Balanced RGB-Event Video Recognitioncited by 5
  2. 2025TESPEC: Temporally-Enhanced Self-Supervised Pretraining for Event Camerascited by 4
  3. 2025From Sharp to Blur: Unsupervised Domain Adaptation for 2D Human Pose Estimation Under Extreme Motion Blur Using Event Camerascited by 3
  4. 2026Segment Any Events with Languagecited by 3
  5. 2025Revealing Latent Information: A Physics-inspired Self-supervised Pre-training Framework for Noisy and Sparse Eventscited by 2
  6. 2026Scaling Dense Event-Stream Pretraining from Visual Foundation Modelscited by 2
  7. 2026EventDrive: Event Cameras for Vision-Language Driving Intelligencecited by 2
  8. 2024EventDance++: Language-guided Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognitioncited by 1
  9. 2024Expanding Event Modality Applications through a Robust CLIP-Based Encodercited by 1
  10. 2025Rethinking High-speed Image Reconstruction Framework with Spike Cameracited by 1
  11. 2025Visual Grounding from Event Camerascited by 1
  12. 2025Event-based Facial Keypoint Alignment via Cross-Modal Fusion Attention and Self-Supervised Multi-Event Representation Learningcited by 1
  13. 2026LAND: A Longitudinal Analysis of Neuromorphic Datasetscited by 1
  14. 2024OVOSE: Open-Vocabulary Semantic Segmentation in Event-Based Camerascited by 0
  15. 2026GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Trackingcited by 0
  16. 2026The Shape of Events: Edge-Based Inductive Biases via Cross-Domain Distillationcited by 0