mlsanityData to 7 Oct 2026

Learnable latent embeddings for joint behavioral and neural analysis

arXiv 2204.00673 ↗First posted 1 Apr 2022cs.LG q-bio.NC q-bio.QMcited by 113

Not on the map. Why?

Abstract (arXiv)

Mapping behavioral actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioral data increases, there is growing interest in modeling neural dynamics during adaptive behaviors to probe neural representations. In particular, neural latent embeddings can reveal underlying correlates of behavior, yet, we lack non-linear techniques that can explicitly and flexibly leverage joint behavior and neural data. Here, we fill this gap with a novel method, CEBRA, that jointly uses behavioral and neural data in a hypothesis- or discovery-driven manner to produce consistent, high-performance latent spaces. We validate its accuracy and demonstrate our tool's utility for both calcium and electrophysiology datasets, across sensory and motor tasks, and in simple or complex behaviors across species. It allows for single and multi-session datasets to be leveraged for hypothesis testing or can be used label-free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, and rapid, high-accuracy decoding of natural movies from visual cortex.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 22 of its 74 references that are in the map. First posted on arXiv 1 Apr 2022.

Builds on

22 of 74 references.

References · 22

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  7. 2020Supervised Contrastive Learningcited by 2,803
  8. 2016Density estimation using Real NVPcited by 2,029
  9. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  10. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  11. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  12. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 121
10 more
  1. 2020On Linear Identifiability of Learned Representationscited by 103
  2. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  3. 2021Neural Latents Benchmark '21: Evaluating latent variable models of neural population activitycited by 57
  4. 2020Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAEcited by 48
  5. 2021Large-scale neural recordings call for new insights to link brain and behaviorcited by 28
  6. 2021Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activitycited by 24
  7. 2014Demixed principal component analysis of population activity in higher cortical areas reveals independent representation of task parameterscited by 15
  8. 2016What can topology tell us about the neural code?cited by 6
  9. 2020Strong and weak principles of neural dimension reductioncited by 2
  10. 2019Decoding hand kinematics from population responses in sensorimotor cortex during graspingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 19; 2024: 27; 2025: 35; 2026: 28.

113 papers.

Sort:

By area:

  1. 2023Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priorscited by 156
  2. 2024MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Datacited by 105
  3. 2023Decoding Natural Images from EEG for Object Recognitioncited by 79
  4. 2022Rank-N-Contrast: Learning Continuous Representations for Regressioncited by 74
  5. 2023Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activitycited by 69
  6. 2023A Unified, Scalable Framework for Neural Population Decodingcited by 59
  7. 2023Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learningcited by 48
  8. 2023Brain Captioning: Decoding human brain activity into images and textcited by 33
  9. 2024Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structurescited by 20
  10. 2024Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked Autoencodercited by 15
  11. 2022Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamicscited by 13
  12. 2024NeuroBind: Towards Unified Multimodal Representations for Neural Signalscited by 13
38 more
  1. 2023Interpretable statistical representations of neural population dynamics and geometrycited by 12
  2. 2023AmadeusGPT: a natural language interface for interactive animal behavioral analysiscited by 12
  3. 2026A foundation model of vision, audition, and language for in-silico neurosciencecited by 12
  4. 2026Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Modelscited by 11
  5. 2023Decoding visual brain representations from electroencephalography through Knowledge Distillation and latent diffusion modelscited by 10
  6. 2023Neuroformer: Multimodal and Multitask Generative Pretraining for Brain Datacited by 10
  7. 2023Multi-modal Gaussian Process Variational Autoencoders for Neural and Behavioral Datacited by 9
  8. 2024A Cross-Modal Approach to Silent Speech with LLM-Enhanced Recognitioncited by 9
  9. 2024Words in Motion: Extracting Interpretable Control Vectors for Motion Transformerscited by 9
  10. 2024Exploring Behavior-Relevant and Disentangled Neural Dynamics with Generative Diffusion Modelscited by 9
  11. 2024Self-supervised contrastive learning performs non-linear system identificationcited by 9
  12. 2025Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Researchcited by 9
  13. 2023Learning Time-Invariant Representations for Individual Neurons from Population Dynamicscited by 8
  14. 2025Generalizable, real-time neural decoding with hybrid state-space modelscited by 8
  15. 2025BRAID: Input-Driven Nonlinear Dynamical Modeling of Neural-Behavioral Datacited by 8
  16. 2024Meta-Dynamical State Space Models for Integrative Neural Data Analysiscited by 7
  17. 2025BaRISTA: Brain Scale Informed Spatiotemporal Representation of Human Intracranial Neural Activitycited by 7
  18. 2022Expressive architectures enhance interpretability of dynamics-based neural population modelscited by 6
  19. 2024Separating common from salient patterns with Contrastive Representation Learningcited by 6
  20. 2024Probabilistic Decomposed Linear Dynamical Systems for Robust Discovery of Latent Neural Dynamicscited by 6
  21. 2025Dynamical Modeling of Behaviorally Relevant Spatiotemporal Patterns in Neural Imaging Datacited by 6
  22. 2023V1T: large-scale mouse V1 response prediction using a Vision Transformercited by 5
  23. 2023SiBBlInGS: Similarity-driven Building-Block Inference using Graphs across Statescited by 5
  24. 2024Adaptive Intelligence: leveraging insights from adaptive behavior in animals to build flexible AI systemscited by 5
  25. 2025A Tale of Two Classes: Adapting Supervised Contrastive Learning to Binary Imbalanced Datasetscited by 5
  26. 2025Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRIcited by 5
  27. 2025AI shares emotion with humans across languages and culturescited by 5
  28. 2025SPINT: Spatial Permutation-Invariant Neural Transformer for Consistent Intracortical Motor Decodingcited by 5
  29. 2025Animal behavioral analysis and neural encoding with transformer-based self-supervised pretrainingcited by 5
  30. 2024Animal Behavior Analysis Methods Using Deep Learning: A Surveycited by 4
  31. 2024BLEND: Behavior-guided Neural Population Dynamics Modeling via Privileged Knowledge Distillationcited by 4
  32. 2025Random Forest Autoencoders for Guided Representation Learningcited by 4
  33. 2025Meta-Learning an In-Context Transformer Model of Human Higher Visual Cortexcited by 4
  34. 2025Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environmentscited by 4
  35. 2026Understanding Self-Supervised Learning via Latent Distribution Matchingcited by 4
  36. 2022Understanding Neural Coding on Latent Manifolds by Sharing Features and Dividing Ensemblescited by 3
  37. 2023SupReMix: Supervised Contrastive Learning for Medical Imaging Regression with Mixupcited by 3
  38. 2024Review of Deep Representation Learning Techniques for Brain-Computer Interfaces and Recommendationscited by 3

50 most cited of 113 All 113 (JSON)