# HippoMM: Hippocampal-inspired Multimodal Memory for Long Audiovisual Event Understanding

arXiv: https://arxiv.org/abs/2504.10739 · HTML: https://mlsanity.com/paper/2504.10739/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2025.

## Built on by (most cited first, 19 of 19)

- [WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning](https://mlsanity.com/paper/2512.02425/) (2025)
- [KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems](https://mlsanity.com/paper/2510.12872/) (2025)
- [From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms](https://mlsanity.com/paper/2605.06716/) (2026)
- [TeleMem: Building Long-Term and Multimodal Memory for Agentic AI](https://mlsanity.com/paper/2601.06037/) (2025)
- [Angles Don't Lie: Unlocking Training-Efficient RL Through the Model's Own Signals](https://mlsanity.com/paper/2506.02281/) (2025)
- [CoreMatching: A Co-adaptive Sparse Inference Framework with Token and Neuron Pruning for Comprehensive Acceleration of Vision-Language Models](https://mlsanity.com/paper/2505.19235/) (2025)
- [FileGram: Grounding Agent Personalization in File-System Behavioral Traces](https://mlsanity.com/paper/2604.04901/) (2026)
- [Keep It Simple: Multi-Key Episodic Memory Retrieval for Ultra-Long Video Understanding](https://mlsanity.com/paper/2608.07663/) (2026)
- [Bridging the Perception Gap: A Lightweight Coarse-to-Fine Architecture for Edge Audio Systems](https://mlsanity.com/paper/2601.15676/) (2026)
- [CraniMem: Cranial Inspired Gated and Bounded Memory for Agentic Systems](https://mlsanity.com/paper/2603.15642/) (2026)
- [Task-Focused Memorization for Multimodal Agents](https://mlsanity.com/paper/2605.31075/) (2026)
- [GuideMe: Multi-Domain Task Guidance and Intervention in Streaming Video](https://mlsanity.com/paper/2607.02991/) (2026)
- [Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory](https://mlsanity.com/paper/2607.05511/) (2026)
- [EM^2Mem: Event-Centric Multimodal Memory for Large Language Models](https://mlsanity.com/paper/2609.00551/) (2026)
- [Statistical Adversaries: Natural Backdoor-like Adversarial Features in Clean Vision Datasets](https://mlsanity.com/paper/2607.05516/) (2026)
- [ReflectWorld-MM: An Entity-Oriented Multimodal Memory System for Open-Ended Video Streams](https://mlsanity.com/paper/2607.09759/) (2026)
- [CAM: Question Answering on Entity-Centric Videos with Continuous Extraction and Adaptive Querying](https://mlsanity.com/paper/2609.06504/) (2026)
- [Vortex: Bridging Extreme Compression and Efficient LLM Inference](https://mlsanity.com/paper/2609.12208/) (2026)
- [Beyond Dyadic Memory: Interaction-Aware Multimodal Memory with Adaptive Agentic Retrieval for Multi-Party Spoken Conversations](https://mlsanity.com/paper/2609.32522/) (2026)

## Abstract (arXiv)

Comprehending extended audiovisual experiences remains challenging for computational systems, particularly temporal integration and cross-modal associations fundamental to human episodic memory. We introduce HippoMM, a computational cognitive architecture that maps hippocampal mechanisms to solve these challenges. Rather than relying on scaling or architectural sophistication, HippoMM implements three integrated components: (i) Episodic Segmentation detects audiovisual input changes to split videos into discrete episodes, mirroring dentate gyrus pattern separation; (ii) Memory Consolidation compresses episodes into summaries with key features preserved, analogous to hippocampal memory formation; and (iii) Hierarchical Memory Retrieval first searches semantic summaries, then escalates via temporal window expansion around seed segments for cross-modal queries, mimicking CA3 pattern completion. These components jointly create an integrated system exceeding the sum of its parts. On our HippoVlog benchmark testing associative memory, HippoMM achieves state-of-the-art 78.2% accuracy while operating 5x faster than retrieval-augmented baselines. Our results demonstrate that cognitive architectures provide blueprints for next-generation multimodal understanding. The code and benchmark dataset are publicly available at https://github.com/linyueqian/HippoMM.