Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challenges
Not on the map. Why?
Abstract (arXiv)
Machine Learning algorithms have had a profound impact on the field of computer science over the past few decades. These algorithms performance is greatly influenced by the representations that are derived from the data in the learning process. The representations learned in a successful learning process should be concise, discrete, meaningful, and able to be applied across a variety of tasks. A recent effort has been directed toward developing Deep Learning models, which have proven to be particularly effective at capturing high-dimensional, non-linear, and multi-modal characteristics. In this work, we discuss the principles and developments that have been made in the process of learning representations, and converting them into desirable applications. In addition, for each framework or model, the key issues and open challenges, as well as the advantages, are examined.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 258 of its 483 references that are in the map. First posted on arXiv 27 Nov 2022.
Builds on
258 of 483 references.
References · 258
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2017Graph Attention Networkscited by 8,854
38 more
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2022Generative Adversarial Networkscited by 5,603
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2013Intriguing properties of neural networkscited by 5,114
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2018How Powerful are Graph Neural Networks?cited by 4,355
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
- 2021Zero-Shot Text-to-Image Generationcited by 4,276
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2013Auto-Encoding Variational Bayescited by 3,980
- 2022A ConvNet for the 2020scited by 3,545
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2017Neural Message Passing for Quantum Chemistrycited by 3,460
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2018Deep contextualized word representationscited by 2,285
- 2013Visualizing and Understanding Convolutional Networkscited by 2,278
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
- 2014Conditional Generative Adversarial Netscited by 2,152
- 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
- 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
- 2020Big Bird: Transformers for Longer Sequencescited by 1,673
- 2016Enriching Word Vectors with Subword Informationcited by 1,578
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
50 most cited of 258 All 258 (JSON)
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 1; 2025: 1.
3 papers.