# EASE: Embodied Active Event Perception via Self-Supervised Energy Minimization

arXiv: https://arxiv.org/abs/2506.17516 · HTML: https://mlsanity.com/paper/2506.17516/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 28 of 28)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Colorization as a Proxy Task for Visual Understanding](https://mlsanity.com/paper/1703.04044/) (2017)
- [Self-supervised Video Representation Learning by Pace Prediction](https://mlsanity.com/paper/2008.05861/) (2020)
- [TCLR: Temporal Contrastive Learning for Video Representation](https://mlsanity.com/paper/2101.07974/) (2021)
- [Weakly Supervised Action Localization by Sparse Temporal Pooling Network](https://mlsanity.com/paper/1712.05080/) (2017)
- [Learning Image Representations by Completing Damaged Jigsaw Puzzles](https://mlsanity.com/paper/1802.01880/) (2018)
- [Generic Event Boundary Detection: A Benchmark for Event Segmentation](https://mlsanity.com/paper/2101.10511/) (2021)
- [Unifying Short and Long-Term Tracking with Graph Hierarchies](https://mlsanity.com/paper/2212.03038/) (2022)
- [Weakly-Supervised Action Localization by Generative Attention Modeling](https://mlsanity.com/paper/2003.12424/) (2020)
- [Video Representation Learning with Visual Tempo Consistency](https://mlsanity.com/paper/2006.15489/) (2020)
- [Weakly-supervised Action Localization with Background Modeling](https://mlsanity.com/paper/1908.06552/) (2019)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [Video Contrastive Learning with Global Context](https://mlsanity.com/paper/2108.02722/) (2021)
- [A Perceptual Prediction Framework for Self Supervised Event Segmentation](https://mlsanity.com/paper/1811.04869/) (2018)
- [Masked Motion Encoding for Self-Supervised Video Representation Learning](https://mlsanity.com/paper/2210.06096/) (2022)
- [End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learning](https://mlsanity.com/paper/1808.03405/) (2018)
- [Towards Distraction-Robust Active Visual Tracking](https://mlsanity.com/paper/2106.10110/) (2021)
- [End-to-end Active Object Tracking via Reinforcement Learning](https://mlsanity.com/paper/1705.10561/) (2017)
- [The Free Energy Principle for Perception and Action: A Deep Learning Perspective](https://mlsanity.com/paper/2207.06415/) (2022)
- [Pose-Assisted Multi-Camera Collaboration for Active Object Tracking](https://mlsanity.com/paper/2001.05161/) (2020)
- [Pointly-Supervised Action Localization](https://mlsanity.com/paper/1805.11333/) (2018)
- [Action Localization through Continual Predictive Learning](https://mlsanity.com/paper/2003.12185/) (2020)
- [Coordinate-Aligned Multi-Camera Collaboration for Active Multi-Object Tracking](https://mlsanity.com/paper/2202.10881/) (2022)
- [Unsupervised Gaze Prediction in Egocentric Videos by Energy-based Surprise Modeling](https://mlsanity.com/paper/2001.11580/) (2020)
- [Towards Active Vision for Action Localization with Reactive Control and Predictive Learning](https://mlsanity.com/paper/2111.05448/) (2021)
- [Self-supervised Multi-actor Social Activity Understanding in Streaming Videos](https://mlsanity.com/paper/2406.14472/) (2024)
- [Emergence of sensory attenuation based upon the free-energy principle](https://mlsanity.com/paper/2111.02666/) (2021)

## Built on by (most cited first, 2 of 2)

- [Generalized Event Partonomy Inference with Structured Hierarchical Predictive Learning](https://mlsanity.com/paper/2512.04219/) (2025)
- [FUSE: Active Functional Affordance Grounding through Adaptive Semantic-Geometric Evidence Acquisition](https://mlsanity.com/paper/2608.12683/) (2026)

## Abstract (arXiv)

Active event perception, the ability to dynamically detect, track, and summarize events in real time, is essential for embodied intelligence in tasks such as human-AI collaboration, assistive robotics, and autonomous navigation. However, existing approaches often depend on predefined action spaces, annotated datasets, and extrinsic rewards, limiting their adaptability and scalability in dynamic, real-world scenarios. Inspired by cognitive theories of event perception and predictive coding, we propose EASE, a self-supervised framework that unifies spatiotemporal representation learning and embodied control through free energy minimization. EASE leverages prediction errors and entropy as intrinsic signals to segment events, summarize observations, and actively track salient actors, operating without explicit annotations or external rewards. By coupling a generative perception model with an action-driven control policy, EASE dynamically aligns predictions with observations, enabling emergent behaviors such as implicit memory, target continuity, and adaptability to novel environments. Extensive evaluations in simulation and real-world settings demonstrate EASE's ability to achieve privacy-preserving and scalable event perception, providing a robust foundation for embodied systems in unscripted, dynamic tasks.