# Revisiting Your Memory: Reconstruction of Affect-Contextualized Memory via EEG-guided Audiovisual Generation

arXiv: https://arxiv.org/abs/2412.05296 · HTML: https://mlsanity.com/paper/2412.05296/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 77 references that are in the map.
- First posted on arXiv 24 Nov 2024.

## Builds on (most cited first, 30 of 34)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation](https://mlsanity.com/paper/2212.09478/) (2022)
- [Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders](https://mlsanity.com/paper/1704.01279/) (2017)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [CLIPstyler: Image Style Transfer with a Single Text Condition](https://mlsanity.com/paper/2112.00374/) (2021)
- [Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfer](https://mlsanity.com/paper/2312.09008/) (2023)
- [Texture Networks: Feed-forward Synthesis of Textures and Stylized Images](https://mlsanity.com/paper/1603.03417/) (2016)
- [Learnable latent embeddings for joint behavioral and neural analysis](https://mlsanity.com/paper/2204.00673/) (2022)
- [Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration](https://mlsanity.com/paper/1805.03857/) (2018)
- [Controlling Perceptual Factors in Neural Style Transfer](https://mlsanity.com/paper/1611.07865/) (2016)
- [Fast Patch-based Style Transfer of Arbitrary Style](https://mlsanity.com/paper/1612.04337/) (2016)
- [Exploring the structure of a real-time, arbitrary neural artistic stylization network](https://mlsanity.com/paper/1705.06830/) (2017)
- [Dynamic Instance Normalization for Arbitrary Style Transfer](https://mlsanity.com/paper/1911.06953/) (2019)
- [Directed Diffusion: Direct Control of Object Placement through Attention Guidance](https://mlsanity.com/paper/2302.13153/) (2023)
- [Rethinking and Improving the Robustness of Image Style Transfer](https://mlsanity.com/paper/2104.05623/) (2021)
- [Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling](https://mlsanity.com/paper/2007.15474/) (2020)
- [MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer](https://mlsanity.com/paper/1809.07600/) (2018)
- [TimbreTron: A WaveNet(CycleGAN(CQT(Audio))) Pipeline for Musical Timbre Transfer](https://mlsanity.com/paper/1811.09620/) (2018)
- [MicroAST: Towards Super-Fast Ultra-Resolution Arbitrary Style Transfer](https://mlsanity.com/paper/2211.15313/) (2022)
- [MM-LDM: Multi-Modal Latent Diffusion Model for Sounding Video Generation](https://mlsanity.com/paper/2410.01594/) (2024)
- [Music Style Transfer: A Position Paper](https://mlsanity.com/paper/1803.06841/) (2018)
- [Audio style transfer](https://mlsanity.com/paper/1710.11385/) (2017)
- [Musical Composition Style Transfer via Disentangled Timbre Representations](https://mlsanity.com/paper/1905.13567/) (2019)

## Built on by (most cited first, 1 of 1)

- [Foundation Models for Cross-Domain EEG Analysis Application: A Survey](https://mlsanity.com/paper/2508.15716/) (2025)

## Abstract (arXiv)

In this paper, we introduce RevisitAffectiveMemory, a novel task designed to reconstruct autobiographical memories through audio-visual generation guided by affect extracted from electroencephalogram (EEG) signals. To support this pioneering task, we present the EEG-AffectiveMemory dataset, which encompasses textual descriptions, visuals, music, and EEG recordings collected during memory recall from nine participants. Furthermore, we propose RYM (Revisit Your Memory), a three-stage framework for generating synchronized audio-visual contents while maintaining dynamic personal memory affect trajectories. Experimental results demonstrate our method successfully decodes individual affect dynamics trajectories from neural signals during memory recall (F1=0.9). Also, our approach faithfully reconstructs affect-contextualized audio-visual memory across all subjects, both qualitatively and quantitatively, with participants reporting strong affective concordance between their recalled memories and the generated content. Especially, contents generated from subject-reported affect dynamics showed higher correlation with participants' reported affect dynamics trajectories (r=0.265, p<.05) and received stronger user preference (preference=56%) compared to those generated from randomly reordered affect dynamics. Our approaches advance affect decoding research and its practical applications in personalized media creation via neural-based affect comprehension. Codes and the dataset are available at https://github.com/ioahKwon/Revisiting-Your-Memory.