# Label-Free Event-based Object Recognition via Joint Learning with Image Reconstruction from Events

arXiv: https://arxiv.org/abs/2308.09383 · HTML: https://mlsanity.com/paper/2308.09383/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 74 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Large Batch Optimization for Deep Learning: Training BERT in 76 minutes](https://mlsanity.com/paper/1904.00962/) (2019)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation](https://mlsanity.com/paper/2110.02624/) (2021)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Training Deep Spiking Neural Networks using Backpropagation](https://mlsanity.com/paper/1608.08782/) (2016)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Events-to-Video: Bringing Modern Computer Vision to Event Cameras](https://mlsanity.com/paper/1904.08298/) (2019)
- [CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP](https://mlsanity.com/paper/2301.04926/) (2023)
- [ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic](https://mlsanity.com/paper/2111.14447/) (2021)
- [Back to Event Basics: Self-Supervised Learning of Image Reconstruction for Event Cameras via Photometric Constancy](https://mlsanity.com/paper/2009.08283/) (2020)
- [Event-Based Fusion for Motion Deblurring with Cross-modal Attention](https://mlsanity.com/paper/2112.00167/) (2021)
- [Event-based Asynchronous Sparse Convolutional Networks](https://mlsanity.com/paper/2003.09148/) (2020)
- [Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences](https://mlsanity.com/paper/1610.09513/) (2016)
- [Event-based High Dynamic Range Image and Very High Frame Rate Video Generation using Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1811.08230/) (2018)
- [N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras](https://mlsanity.com/paper/2112.01041/) (2021)
- [Secrets of Event-Based Optical Flow](https://mlsanity.com/paper/2207.10022/) (2022)
- [Graph-Based Object Classification for Neuromorphic Vision Sensing](https://mlsanity.com/paper/1908.06648/) (2019)
- [Continuous-time Intensity Estimation Using Event Cameras](https://mlsanity.com/paper/1811.00386/) (2018)

## Built on by (most cited first, 28 of 28)

- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2405.05259/) (2024)
- [ExACT: Language-guided Conceptual Reasoning and Uncertainty Estimation for Event-based Action Recognition and More](https://mlsanity.com/paper/2403.12534/) (2024)
- [LEOD: Label-Efficient Object Detection for Event Cameras](https://mlsanity.com/paper/2311.17286/) (2023)
- [EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition](https://mlsanity.com/paper/2403.14082/) (2024)
- [EventVL: Understand Event Streams via Multimodal Large Language Model](https://mlsanity.com/paper/2501.13707/) (2025)
- [Temporal Event Stereo via Joint Learning with Stereoscopic Flow](https://mlsanity.com/paper/2407.10831/) (2024)
- [Towards Robust Event-based Networks for Nighttime via Unpaired Day-to-Night Event Translation](https://mlsanity.com/paper/2407.10703/) (2024)
- [EZSR: Event-based Zero-Shot Recognition](https://mlsanity.com/paper/2407.21616/) (2024)
- [Finding Meaning in Points: Weakly Supervised Semantic Segmentation for Event Cameras](https://mlsanity.com/paper/2407.11216/) (2024)
- [Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application](https://mlsanity.com/paper/2503.22943/) (2025)
- [Talk2Event: Grounded Understanding of Dynamic Scenes from Event Cameras](https://mlsanity.com/paper/2507.17664/) (2025)
- [Unleashing the Power of CNN and Transformer for Balanced RGB-Event Video Recognition](https://mlsanity.com/paper/2312.11128/) (2023)
- [TESPEC: Temporally-Enhanced Self-Supervised Pretraining for Event Cameras](https://mlsanity.com/paper/2508.00913/) (2025)
- [From Sharp to Blur: Unsupervised Domain Adaptation for 2D Human Pose Estimation Under Extreme Motion Blur Using Event Cameras](https://mlsanity.com/paper/2507.22438/) (2025)
- [Segment Any Events with Language](https://mlsanity.com/paper/2601.23159/) (2026)
- [Revealing Latent Information: A Physics-inspired Self-supervised Pre-training Framework for Noisy and Sparse Events](https://mlsanity.com/paper/2508.05507/) (2025)
- [Scaling Dense Event-Stream Pretraining from Visual Foundation Models](https://mlsanity.com/paper/2603.03969/) (2026)
- [EventDrive: Event Cameras for Vision-Language Driving Intelligence](https://mlsanity.com/paper/2606.18242/) (2026)
- [EventDance++: Language-guided Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition](https://mlsanity.com/paper/2409.12778/) (2024)
- [Expanding Event Modality Applications through a Robust CLIP-Based Encoder](https://mlsanity.com/paper/2412.03093/) (2024)
- [Rethinking High-speed Image Reconstruction Framework with Spike Camera](https://mlsanity.com/paper/2501.04477/) (2025)
- [Visual Grounding from Event Cameras](https://mlsanity.com/paper/2509.09584/) (2025)
- [Event-based Facial Keypoint Alignment via Cross-Modal Fusion Attention and Self-Supervised Multi-Event Representation Learning](https://mlsanity.com/paper/2509.24968/) (2025)
- [LAND: A Longitudinal Analysis of Neuromorphic Datasets](https://mlsanity.com/paper/2602.15973/) (2026)
- [OVOSE: Open-Vocabulary Semantic Segmentation in Event-Based Cameras](https://mlsanity.com/paper/2408.09424/) (2024)
- [GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking](https://mlsanity.com/paper/2607.15699/) (2026)
- [The Shape of Events: Edge-Based Inductive Biases via Cross-Domain Distillation](https://mlsanity.com/paper/2609.30478/) (2026)

## Abstract (arXiv)

Recognizing objects from sparse and noisy events becomes extremely difficult when paired images and category labels do not exist. In this paper, we study label-free event-based object recognition where category labels and paired images are not available. To this end, we propose a joint formulation of object recognition and image reconstruction in a complementary manner. Our method first reconstructs images from events and performs object recognition through Contrastive Language-Image Pre-training (CLIP), enabling better recognition through a rich context of images. Since the category information is essential in reconstructing images, we propose category-guided attraction loss and category-agnostic repulsion loss to bridge the textual features of predicted categories and the visual features of reconstructed images using CLIP. Moreover, we introduce a reliable data sampling strategy and local-global reconstruction consistency to boost joint learning of two tasks. To enhance the accuracy of prediction and quality of reconstruction, we also propose a prototype-based approach using unpaired images. Extensive experiments demonstrate the superiority of our method and its extensibility for zero-shot object recognition. Our project code is available at urlhttps://github.com/Chohoonhee/Ev-LaFOR.