mlsanityData to 7 Oct 2026

Large-scale Foundation Models and Generative AI for BigData Neuroscience

arXiv 2310.18377 ↗First posted 27 Oct 2023q-bio.NC cs.AI cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in machine learning have made revolutionary breakthroughs in computer games, image and natural language understanding, and scientific discovery. Foundation models and large-scale language models (LLMs) have recently achieved human-like intelligence thanks to BigData. With the help of self-supervised learning (SSL) and transfer learning, these models may potentially reshape the landscapes of neuroscience research and make a significant impact on the future. Here we present a mini-review on recent advances in foundation models and generative AI models as well as their applications in neuroscience, including natural language and speech, semantic memory, brain-machine interfaces (BMIs), and data augmentation. We argue that this paradigm-shift framework will open new avenues for many neuroscience research directions and discuss the accompanying challenges and opportunities.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 49 of its 139 references that are in the map. First posted on arXiv 27 Oct 2023.

Builds on

49 of 139 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2013Auto-Encoding Variational Bayescited by 3,980
  12. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
37 more
  1. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  2. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  3. 2014How transferable are features in deep neural networks?cited by 1,216
  4. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  5. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  6. 2023A Comprehensive Overview of Large Language Modelscited by 639
  7. 2021Score-based Generative Modeling in Latent Spacecited by 634
  8. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  9. 2020A Review on Generative Adversarial Networks: Algorithms, Theory, and Applicationscited by 209
  10. 2021BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG datacited by 194
  11. 2022Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decodingcited by 142
  12. 2022Decoding speech perception from non-invasive brain recordingscited by 129
  13. 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 128
  14. 2022Learnable latent embeddings for joint behavioral and neural analysiscited by 113
  15. 2021Towards artificial general intelligence via a multimodal foundation modelcited by 112
  16. 2020Uncovering the structure of clinical EEG signals with self-supervised learningcited by 109
  17. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 108
  18. 2023Natural scene reconstruction from fMRI signals using generative latent diffusioncited by 97
  19. 2021Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysiscited by 75
  20. 2023Brain decoding: toward real-time reconstruction of visual perceptioncited by 64
  21. 2023A Unified, Scalable Framework for Neural Population Decodingcited by 59
  22. 2020Neural Dynamic Policies for End-to-End Sensorimotor Learningcited by 56
  23. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  24. 2021Representation learning for neural population activity with Neural Data Transformerscited by 48
  25. 2015Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortexcited by 44
  26. 2018Reconstructing Faces from fMRI Patterns using Deep Generative Neural Networkscited by 43
  27. 2022Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Datacited by 39
  28. 2023Brain Captioning: Decoding human brain activity into images and textcited by 33
  29. 2021Self-supervised EEG Representation Learning for Automatic Sleep Stagingcited by 30
  30. 2019FMRI data augmentation via synthesiscited by 19
  31. 2022Self-supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluationcited by 14
  32. 2021Decoding EEG Brain Activity for Multi-Modal Natural Language Processingcited by 13
  33. 2022Semantic Brain Decoding: from fMRI to conceptually similar image reconstruction of visual stimulicited by 12
  34. 2022Multimodal foundation models are better simulators of the human braincited by 10
  35. 2023Eeg2vec: Self-Supervised Electroencephalographic Representation Learningcited by 8
  36. 2022Modern Views of Machine Learning for Precision Psychiatrycited by 5
  37. 2022Transfer learning to decode brain states reflecting the relationship between cognitive taskscited by 5

Built on by

Not cited yet.