mlsanityData to 14 Oct 2026

Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity

arXiv 2111.02338 ↗First posted 3 Nov 2021cs.LG stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

Meaningful and simplified representations of neural activity can yield insights into how and what information is being processed within a neural circuit. However, without labels, finding representations that reveal the link between the brain and behavior can be challenging. Here, we introduce a novel unsupervised approach for learning disentangled representations of neural activity called Swap-VAE. Our approach combines a generative modeling framework with an instance-specific alignment loss that tries to maximize the representational similarity between transformed views of the input (brain state). These transformed (or augmented) views are created by dropping out neurons and jittering samples in time, which intuitively should lead the network to a representation that maintains both temporal consistency and invariance to the specific neurons used to represent the neural state. Through evaluations on both synthetic data and neural recordings from hundreds of neurons in different primate brains, we show that it is possible to build representations that disentangle neural datasets along relevant latent dimensions linked to behavior.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 30 of its 54 references that are in the map. First posted on arXiv 3 Nov 2021.

Builds on

30 of 54 references.

References · 30

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  7. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  8. 2016Density estimation using Real NVPcited by 2,042
  9. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  10. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  11. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  12. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
18 more
  1. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  2. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  3. 2018Understanding disentangling in β-VAEcited by 335
  4. 2018Towards a Definition of Disentangled Representationscited by 296
  5. 2019On the Fairness of Disentangled Representationscited by 144
  6. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 102
  7. 2019Are Disentangled Representations Helpful for Abstract Visual Reasoning?cited by 100
  8. 2016Disentangling factors of variation in deep representations using adversarial trainingcited by 87
  9. 2017Multi-style Generative Network for Real-time Transfercited by 61
  10. 2014Variational Recurrent Auto-Encoderscited by 53
  11. 2021Representation learning for neural population activity with Neural Data Transformerscited by 50
  12. 2020Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAEcited by 48
  13. 2016Linear dynamical neural population models through nonlinear embeddingscited by 36
  14. 2021Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Predictioncited by 36
  15. 2018Adversarial Domain Adaptation for Stable Brain-Machine Interfacescited by 27
  16. 2019Self-supervised Label Augmentation via Input Transformationscited by 15
  17. 2019Deep Random Splines for Point Process Intensity Estimation of Neural Population Datacited by 8
  18. 2018Style Separation and Synthesis via Generative Adversarial Networkscited by 5

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 5; 2024: 4; 2025: 4.

18 papers.

Sort:

By area:

  1. 2023A Unified, Scalable Framework for Neural Population Decodingcited by 61
  2. 2022The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspectivecited by 24
  3. 2022STNDT: Modeling Neural Population Activity with a Spatiotemporal Transformercited by 23
  4. 2022Seeing the forest and the tree: Building representations of both individual and collective dynamics with transformerscited by 18
  5. 2024Exploring Behavior-Relevant and Disentangled Neural Dynamics with Generative Diffusion Modelscited by 9
  6. 2023Extraction and Recovery of Spatio-Temporal Structure in Latent Dynamics Alignment with Diffusion Modelscited by 7
  7. 2025Your contrastive learning problem is secretly a distribution alignment problemcited by 6
  8. 2023Neural Latent Aligner: Cross-trial Alignment for Learning Representations of Complex, Naturalistic Neural Datacited by 5
  9. 2023Temporal Conditioning Spiking Latent Variable Models of the Neural Response to Natural Visual Scenescited by 4
  10. 2025Cross-Modal Representational Knowledge Distillation for Enhanced Spike-Informed LFP Modelingcited by 3
  11. 2025Know Thyself by Knowing Others: Learning Neuron Identity from Population Contextcited by 2
  12. 2022Capturing cross-session neural population variability through self-supervised identification of consistent neuron ensemblescited by 1
6 more
  1. 2022Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypescited by 1
  2. 2023Predictive variational autoencoder for learning robust representations of time-series datacited by 1
  3. 2024Diffusion-Based Generation of Neural Activity from Disentangled Latent Codescited by 1
  4. 2024Unsupervised discovery of the shared and private geometry in multi-view datacited by 1
  5. 2024Generalizable autoregressive modeling of time series through functional narrativescited by 1
  6. 2025A Scalable, Causal, and Energy Efficient Framework for Neural Decoding with Spiking Neural Networkscited by 0