{"list":"other","total":67,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2201.11903","title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","created":"2022-01-28","cited_by_count":12459},{"id":"1503.02531","title":"Distilling the Knowledge in a Neural Network","created":"2015-03-09","cited_by_count":10324},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"1703.03400","title":"Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks","created":"2017-03-09","cited_by_count":4897},{"id":"1409.0473","title":"Neural Machine Translation by Jointly Learning to Align and Translate","created":"2014-09-01","cited_by_count":4548},{"id":"2101.00190","title":"Prefix-Tuning: Optimizing Continuous Prompts for Generation","created":"2021-01-01","cited_by_count":3915},{"id":"2104.08691","title":"The Power of Scale for Parameter-Efficient Prompt Tuning","created":"2021-04-18","cited_by_count":3795},{"id":"1606.08415","title":"Gaussian Error Linear Units (GELUs)","created":"2016-06-27","cited_by_count":3502},{"id":"1703.05175","title":"Prototypical Networks for Few-shot Learning","created":"2017-03-15","cited_by_count":3015},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"2006.16236","title":"Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention","created":"2020-06-29","cited_by_count":2022},{"id":"2009.14794","title":"Rethinking Attention with Performers","created":"2020-09-30","cited_by_count":1601},{"id":"1803.10122","title":"World Models","created":"2018-03-27","cited_by_count":1513},{"id":"2006.04768","title":"Linformer: Self-Attention with Linear Complexity","created":"2020-06-08","cited_by_count":1474},{"id":"2202.12837","title":"Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?","created":"2022-02-25","cited_by_count":1434},{"id":"1610.01644","title":"Understanding intermediate layers using linear classifier probes","created":"2016-10-05","cited_by_count":1353},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"1703.00443","title":"OptNet: Differentiable Optimization as a Layer in Neural Networks","created":"2017-03-01","cited_by_count":599},{"id":"1611.02779","title":"RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning","created":"2016-11-09","cited_by_count":503},{"id":"2102.11174","title":"Linear Transformers Are Secretly Fast Weight Programmers","created":"2021-02-22","cited_by_count":502},{"id":"2212.14052","title":"Hungry Hungry Hippos: Towards Language Modeling with State Space Models","created":"2022-12-28","cited_by_count":499},{"id":"2008.02217","title":"Hopfield Networks is All You Need","created":"2020-07-16","cited_by_count":475},{"id":"2302.10866","title":"Hyena Hierarchy: Towards Larger Convolutional Language Models","created":"2023-02-21","cited_by_count":393},{"id":"1611.05763","title":"Learning to reinforcement learn","created":"2016-11-17","cited_by_count":377},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"2309.00941","title":"Emergent Linear Representations in World Models of Self-Supervised Sequence Models","created":"2023-09-02","cited_by_count":319},{"id":"2205.05055","title":"Data Distributional Properties Drive Emergent In-Context Learning in Transformers","created":"2022-04-22","cited_by_count":309},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268},{"id":"2212.13345","title":"The Forward-Forward Algorithm: Some Preliminary Investigations","created":"2022-12-27","cited_by_count":266},{"id":"1906.01820","title":"Risks from Learned Optimization in Advanced Machine Learning Systems","created":"2019-06-05","cited_by_count":258},{"id":"1710.03667","title":"High-dimensional dynamics of generalization error in neural networks","created":"2017-10-10","cited_by_count":241},{"id":"2301.13196","title":"Looped Transformers as Programmable Computers","created":"2023-01-30","cited_by_count":237},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"1610.06258","title":"Using Fast Weights to Attend to the Recent Past","created":"2016-10-20","cited_by_count":170},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170},{"id":"1412.7525","title":"Difference Target Propagation","created":"2014-12-23","cited_by_count":148},{"id":"2306.15063","title":"Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression","created":"2023-06-26","cited_by_count":146},{"id":"1901.06656","title":"Training Neural Networks with Local Error Signals","created":"2019-01-20","cited_by_count":136},{"id":"1812.11446","title":"Greedy Layerwise Learning Can Scale to ImageNet","created":"2018-12-29","cited_by_count":115},{"id":"2212.04458","title":"General-Purpose In-Context Learning by Meta-Learning Transformers","created":"2022-12-08","cited_by_count":105},{"id":"1905.11786","title":"Putting An End to End-to-End: Gradient-Isolated Learning of Representations","created":"2019-05-28","cited_by_count":86},{"id":"1909.04866","title":"Deep Declarative Networks: A New Hope","created":"2019-09-11","cited_by_count":78},{"id":"2209.00099","title":"Efficient Methods for Natural Language Processing: A Survey","created":"2022-08-31","cited_by_count":78},{"id":"2103.14636","title":"A Practical Survey on Faster and Lighter Transformers","created":"2021-03-26","cited_by_count":73},{"id":"2102.13249","title":"Chess as a Testbed for Language Model State Tracking","created":"2021-02-26","cited_by_count":72},{"id":"2302.07253","title":"Energy Transformer","created":"2023-02-14","cited_by_count":71},{"id":"2210.05675","title":"Transformers generalize differently from information stored in context vs in weights","created":"2022-10-11","cited_by_count":49},{"id":"1903.06842","title":"Formulas for Data-driven Control: Stabilization, Optimality and Robustness","created":"2019-03-15","cited_by_count":47},{"id":"2006.07214","title":"Sparse and Continuous Attention Mechanisms","created":"2020-06-12","cited_by_count":39},{"id":"2205.03076","title":"Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation","created":"2022-05-06","cited_by_count":26},{"id":"2308.06912","title":"CausalLM is not optimal for in-context learning","created":"2023-08-14","cited_by_count":26},{"id":"2305.18869","title":"Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning","created":"2023-05-30","cited_by_count":19},{"id":"2207.01332","title":"The least-control principle for local learning at equilibrium","created":"2022-07-04","cited_by_count":16},{"id":"1811.05090","title":"A General Method for Amortizing Variational Filtering","created":"2018-11-13","cited_by_count":14},{"id":"2305.10203","title":"Exploring the Space of Key-Value-Query Models with Intention","created":"2023-05-17","cited_by_count":7},{"id":"1905.01320","title":"Meta-learners' learning dynamics are unlike learners'","created":"2019-05-03","cited_by_count":3}],"next_cursor":null}