{"list":"other","total":76,"rows":[{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2103.00020","title":"Learning Transferable Visual Models From Natural Language Supervision","created":"2021-02-26","cited_by_count":33744},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2002.05709","title":"A Simple Framework for Contrastive Learning of Visual Representations","created":"2020-02-13","cited_by_count":12049},{"id":"1807.03748","title":"Representation Learning with Contrastive Predictive Coding","created":"2018-07-10","cited_by_count":7777},{"id":"1911.05722","title":"Momentum Contrast for Unsupervised Visual Representation Learning","created":"2019-11-13","cited_by_count":7513},{"id":"2111.06377","title":"Masked Autoencoders Are Scalable Vision Learners","created":"2021-11-11","cited_by_count":7403},{"id":"2104.14294","title":"Emerging Properties in Self-Supervised Vision Transformers","created":"2021-04-29","cited_by_count":6968},{"id":"2204.06125","title":"Hierarchical Text-Conditional Image Generation with CLIP Latents","created":"2022-04-13","cited_by_count":6585},{"id":"2006.07733","title":"Bootstrap your own latent: A new approach to self-supervised Learning","created":"2020-06-13","cited_by_count":4750},{"id":"2102.12092","title":"Zero-Shot Text-to-Image Generation","created":"2021-02-24","cited_by_count":4288},{"id":"2011.10566","title":"Exploring Simple Siamese Representation Learning","created":"2020-11-20","cited_by_count":2866},{"id":"2006.09882","title":"Unsupervised Learning of Visual Features by Contrasting Cluster Assignments","created":"2020-06-17","cited_by_count":2857},{"id":"1906.08237","title":"XLNet: Generalized Autoregressive Pretraining for Language Understanding","created":"2019-06-19","cited_by_count":2811},{"id":"2106.08254","title":"BEiT: BERT Pre-Training of Image Transformers","created":"2021-06-15","cited_by_count":2375},{"id":"2003.04297","title":"Improved Baselines with Momentum Contrastive Learning","created":"2020-03-09","cited_by_count":2287},{"id":"2104.08821","title":"SimCSE: Simple Contrastive Learning of Sentence Embeddings","created":"2021-04-18","cited_by_count":2173},{"id":"2103.03230","title":"Barlow Twins: Self-Supervised Learning via Redundancy Reduction","created":"2021-03-04","cited_by_count":1838},{"id":"2104.02057","title":"An Empirical Study of Training Self-Supervised Vision Transformers","created":"2021-04-05","cited_by_count":1616},{"id":"1906.05849","title":"Contrastive Multiview Coding","created":"2019-06-13","cited_by_count":1301},{"id":"1808.06670","title":"Learning deep representations by mutual information estimation and maximization","created":"2018-08-20","cited_by_count":1273},{"id":"2105.04906","title":"VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning","created":"2021-05-11","cited_by_count":1057},{"id":"2005.10243","title":"What Makes for Good Views for Contrastive Learning?","created":"2020-05-20","cited_by_count":846},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"1901.08584","title":"Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks","created":"2019-01-24","cited_by_count":566},{"id":"1902.09229","title":"A Theoretical Analysis of Contrastive Unsupervised Representation Learning","created":"2019-02-25","cited_by_count":530},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1811.04918","title":"Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers","created":"2018-11-12","cited_by_count":340},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"2106.04156","title":"Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss","created":"2021-06-08","cited_by_count":313},{"id":"2106.04619","title":"Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style","created":"2021-06-08","cited_by_count":312},{"id":"2012.09816","title":"Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning","created":"2020-12-17","cited_by_count":297},{"id":"2007.06346","title":"Whitening for Self-Supervised Representation Learning","created":"2020-07-13","cited_by_count":279},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"2102.06810","title":"Understanding self-supervised Learning Dynamics without Contrastive Pairs","created":"2021-02-12","cited_by_count":256},{"id":"1806.00468","title":"Implicit Bias of Gradient Descent on Linear Convolutional Networks","created":"2018-06-01","cited_by_count":251},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"2105.00470","title":"On Feature Decorrelation in Self-Supervised Learning","created":"2021-05-02","cited_by_count":160},{"id":"2008.01064","title":"Predicting What You Already Know Helps: Provable Self-Supervised Learning","created":"2020-08-03","cited_by_count":152},{"id":"2008.10150","title":"Contrastive learning, multi-view redundancy, and linear models","created":"2020-08-24","cited_by_count":146},{"id":"1705.09886","title":"Convergence Analysis of Two-layer Neural Networks with ReLU Activation","created":"2017-05-28","cited_by_count":137},{"id":"2011.02803","title":"Intriguing Properties of Contrastive Losses","created":"2020-11-05","cited_by_count":130},{"id":"2005.06398","title":"Implicit Regularization in Deep Learning May Not Be Explainable by Norms","created":"2020-05-13","cited_by_count":119},{"id":"2105.15134","title":"Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning","created":"2021-05-31","cited_by_count":119},{"id":"2106.11230","title":"Can contrastive learning avoid shortcut solutions?","created":"2021-06-21","cited_by_count":114},{"id":"2005.10190","title":"Feature Purification: How Adversarial Training Performs Robust Deep Learning","created":"2020-05-20","cited_by_count":107},{"id":"2103.06695","title":"BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation","created":"2021-03-11","cited_by_count":106},{"id":"2004.14646","title":"Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning","created":"2020-04-30","cited_by_count":102},{"id":"2111.00743","title":"Towards the Generalization of Contrastive Self-Supervised Learning","created":"2021-11-01","cited_by_count":99},{"id":"2202.14037","title":"Understanding Contrastive Learning Requires Incorporating Inductive Biases","created":"2022-02-28","cited_by_count":96},{"id":"2010.10241","title":"BYOL works even without batch statistics","created":"2020-10-20","cited_by_count":95},{"id":"1706.03175","title":"Recovery Guarantees for One-hidden-layer Neural Networks","created":"2017-06-10","cited_by_count":94},{"id":"1905.10337","title":"What Can ResNet Learn Efficiently, Going Beyond Kernels?","created":"2019-05-24","cited_by_count":91},{"id":"1810.12065","title":"On the Convergence Rate of Training Recurrent Neural Networks","created":"2018-10-29","cited_by_count":82},{"id":"2106.09226","title":"Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning","created":"2021-06-17","cited_by_count":82},{"id":"2010.03648","title":"A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks","created":"2020-10-07","cited_by_count":72},{"id":"1905.12207","title":"On the Expressive Power of Deep Polynomial Neural Networks","created":"2019-05-29","cited_by_count":66},{"id":"2203.16262","title":"How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning","created":"2022-03-30","cited_by_count":65},{"id":"1702.07966","title":"Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs","created":"2017-02-26","cited_by_count":61},{"id":"1705.04591","title":"Learning ReLUs via Gradient Descent","created":"2017-05-10","cited_by_count":54},{"id":"2003.02234","title":"Contrastive estimation reveals topic posterior information to linear models","created":"2020-03-04","cited_by_count":54},{"id":"2106.09943","title":"Investigating the Role of Negatives in Contrastive Representation Learning","created":"2021-06-18","cited_by_count":54},{"id":"2110.02473","title":"The Power of Contrast for Feature Learning: A Theoretical Analysis","created":"2021-10-06","cited_by_count":53},{"id":"1703.00560","title":"An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis","created":"2017-03-02","cited_by_count":52},{"id":"2108.11371","title":"Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization","created":"2021-08-25","cited_by_count":49},{"id":"2203.15702","title":"Contrasting the landscape of contrastive and non-contrastive learning","created":"2022-03-29","cited_by_count":24},{"id":"2110.04947","title":"Towards Demystifying Representation Learning with Non-contrastive Self-supervision","created":"2021-10-11","cited_by_count":23},{"id":"2010.08508","title":"For self-supervised learning, Rationality implies generalization, provably","created":"2020-10-16","cited_by_count":22},{"id":"2007.04596","title":"Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK","created":"2020-07-09","cited_by_count":18},{"id":"2110.11271","title":"Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation","created":"2021-10-21","cited_by_count":15},{"id":"2103.03568","title":"Can Pretext-Based Self-Supervised Learning Be Boosted by Downstream Data? A Theoretical Analysis","created":"2021-03-05","cited_by_count":9},{"id":"2106.02619","title":"Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions","created":"2021-06-04","cited_by_count":9},{"id":"2202.09305","title":"Masked prediction tasks: a parameter identifiability view","created":"2022-02-18","cited_by_count":4},{"id":"2204.04209","title":"Learning Polynomial Transformations","created":"2022-04-08","cited_by_count":3},{"id":"1610.09300","title":"Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods","created":"2016-10-28","cited_by_count":2}],"next_cursor":null}