{"list":"other","total":94,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2203.02155","title":"Training language models to follow instructions with human feedback","created":"2022-03-04","cited_by_count":15971},{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2201.11903","title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","created":"2022-01-28","cited_by_count":12459},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2108.07258","title":"On the Opportunities and Risks of Foundation Models","created":"2021-08-16","cited_by_count":4033},{"id":"2101.00190","title":"Prefix-Tuning: Optimizing Continuous Prompts for Generation","created":"2021-01-01","cited_by_count":3915},{"id":"2104.08691","title":"The Power of Scale for Parameter-Efficient Prompt Tuning","created":"2021-04-18","cited_by_count":3795},{"id":"2109.01652","title":"Finetuned Language Models Are Zero-Shot Learners","created":"2021-09-03","cited_by_count":3780},{"id":"2305.10601","title":"Tree of Thoughts: Deliberate Problem Solving with Large Language Models","created":"2023-05-17","cited_by_count":3211},{"id":"2210.11416","title":"Scaling Instruction-Finetuned Language Models","created":"2022-10-20","cited_by_count":3024},{"id":"2303.12712","title":"Sparks of Artificial General Intelligence: Early experiments with GPT-4","created":"2023-03-22","cited_by_count":2592},{"id":"2206.07682","title":"Emergent Abilities of Large Language Models","created":"2022-06-15","cited_by_count":2512},{"id":"2102.09690","title":"Calibrate Before Use: Improving Few-Shot Performance of Language Models","created":"2021-02-19","cited_by_count":1392},{"id":"2012.14913","title":"Transformer Feed-Forward Layers Are Key-Value Memories","created":"2020-12-29","cited_by_count":1336},{"id":"2012.15723","title":"Making Pre-trained Language Models Better Few-shot Learners","created":"2020-12-31","cited_by_count":1256},{"id":"2305.11206","title":"LIMA: Less Is More for Alignment","created":"2023-05-18","cited_by_count":1100},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2112.00114","title":"Show Your Work: Scratchpads for Intermediate Computation with Language Models","created":"2021-11-30","cited_by_count":886},{"id":"2204.07705","title":"Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks","created":"2022-04-16","cited_by_count":868},{"id":"2303.16199","title":"LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention","created":"2023-03-28","cited_by_count":781},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2104.08773","title":"Cross-Task Generalization via Natural Language Crowdsourcing Instructions","created":"2021-04-18","cited_by_count":699},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2304.15010","title":"LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model","created":"2023-04-28","cited_by_count":602},{"id":"2304.11015","title":"DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction","created":"2023-04-21","cited_by_count":477},{"id":"2110.15943","title":"MetaICL: Learning to Learn In Context","created":"2021-10-29","cited_by_count":454},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"2212.14024","title":"Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP","created":"2022-12-28","cited_by_count":318},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"2201.03514","title":"Black-Box Tuning for Language-Model-as-a-Service","created":"2022-01-10","cited_by_count":256},{"id":"2212.12017","title":"OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization","created":"2022-12-22","cited_by_count":236},{"id":"2310.16028","title":"What Algorithms can Transformers Learn? A Study in Length Generalization","created":"2023-10-24","cited_by_count":199},{"id":"2212.02437","title":"In-context Examples Selection for Machine Translation","created":"2022-12-05","cited_by_count":172},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170},{"id":"2301.11235","title":"Handbook of Convergence Theorems for (Stochastic) Gradient Methods","created":"2023-01-26","cited_by_count":163},{"id":"2306.00802","title":"Birth of a Transformer: A Memory Viewpoint","created":"2023-06-01","cited_by_count":161},{"id":"2305.09731","title":"What In-Context Learning \"Learns\" In-Context: Disentangling Task Recognition and Task Learning","created":"2023-05-16","cited_by_count":152},{"id":"2311.12424","title":"Looped Transformers are Better at Learning Learning Algorithms","created":"2023-11-21","cited_by_count":149},{"id":"2306.15063","title":"Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression","created":"2023-06-26","cited_by_count":146},{"id":"2312.03002","title":"The mechanistic basis of data dependence and abrupt learning in an in-context classification task","created":"2023-12-03","cited_by_count":118},{"id":"2310.05249","title":"In-Context Convergence of Transformers","created":"2023-10-08","cited_by_count":115},{"id":"2307.15936","title":"A Theory for Emergence of Complex Skills in Language Models","created":"2023-07-29","cited_by_count":113},{"id":"2211.09066","title":"Teaching Algorithmic Reasoning via In-context Learning","created":"2022-11-15","cited_by_count":112},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102},{"id":"2310.08391","title":"How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?","created":"2023-10-12","cited_by_count":96},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93},{"id":"2303.04245","title":"How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding","created":"2023-03-07","cited_by_count":86},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77},{"id":"2305.08298","title":"Symbol tuning improves in-context learning in language models","created":"2023-05-15","cited_by_count":75},{"id":"2310.10616","title":"How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations","created":"2023-10-16","cited_by_count":75},{"id":"2410.08292","title":"Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?","created":"2024-10-10","cited_by_count":65},{"id":"2312.06528","title":"Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context","created":"2023-12-11","cited_by_count":63},{"id":"2005.08575","title":"Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representation","created":"2020-05-18","cited_by_count":59},{"id":"2310.00535","title":"JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention","created":"2023-10-01","cited_by_count":52},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45},{"id":"2405.19592","title":"Why Larger Language Models Do In-context Learning Differently?","created":"2024-05-30","cited_by_count":45},{"id":"2309.12673","title":"On Sparse Modern Hopfield Model","created":"2023-09-22","cited_by_count":42},{"id":"2402.04520","title":"On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity Analysis","created":"2024-02-07","cited_by_count":41},{"id":"2407.01079","title":"On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs)","created":"2024-07-01","cited_by_count":40},{"id":"2407.15720","title":"Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability","created":"2024-07-22","cited_by_count":40},{"id":"2409.17422","title":"Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction","created":"2024-09-25","cited_by_count":39},{"id":"2404.03827","title":"Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models","created":"2024-04-04","cited_by_count":38},{"id":"2404.03828","title":"Outlier-Efficient Hopfield Layers for Large Transformer-Based Models","created":"2024-04-04","cited_by_count":38},{"id":"2304.04397","title":"Randomized and Deterministic Attention Sparsification Algorithms for Over-parameterized Feature Dimension","created":"2023-04-10","cited_by_count":37},{"id":"2312.17346","title":"STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction","created":"2023-12-28","cited_by_count":35},{"id":"2405.05219","title":"Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers","created":"2024-05-08","cited_by_count":32},{"id":"2402.15017","title":"Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning","created":"2024-02-22","cited_by_count":31},{"id":"2402.01107","title":"Simulation of Graph Algorithms with Looped Transformers","created":"2024-02-02","cited_by_count":30},{"id":"2405.16411","title":"Training Tensor Attention Efficiently: From Cubic to Almost Linear Time","created":"2024-05-26","cited_by_count":30},{"id":"2303.00106","title":"The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning","created":"2023-02-28","cited_by_count":27},{"id":"2403.03183","title":"How Well Can Transformers Emulate In-context Newton's Method?","created":"2024-03-05","cited_by_count":25},{"id":"2405.16418","title":"Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective","created":"2024-05-26","cited_by_count":24},{"id":"2312.04021","title":"A Study on the Calibration of In-context Learning","created":"2023-12-07","cited_by_count":22},{"id":"2408.13233","title":"Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Time","created":"2024-08-23","cited_by_count":22},{"id":"2404.03830","title":"BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model","created":"2024-04-04","cited_by_count":21},{"id":"2408.12151","title":"A Tighter Complexity Analysis of SparseGPT","created":"2024-08-22","cited_by_count":19},{"id":"2410.09375","title":"Looped ReLU MLPs May Be All You Need as Practical Programmable Computers","created":"2024-10-12","cited_by_count":19},{"id":"2410.10165","title":"HSR-Enhanced Sparse Attention Acceleration","created":"2024-10-14","cited_by_count":19},{"id":"2404.03900","title":"Nonparametric Modern Hopfield Models","created":"2024-04-05","cited_by_count":16},{"id":"2405.03251","title":"Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyond","created":"2024-05-06","cited_by_count":15},{"id":"2210.08361","title":"A Nearly Optimal Size Coreset Algorithm with Nearly Linear Time","created":"2022-10-15","cited_by_count":13},{"id":"2410.09397","title":"On Fine-Grained I/O Complexity of Attention Backward Passes","created":"2024-10-12","cited_by_count":11},{"id":"2303.06210","title":"A Theoretical Analysis Of Nearest Neighbor Search On Approximate Near Neighbor Graph","created":"2023-03-10","cited_by_count":10},{"id":"2406.14036","title":"Towards Infinite-Long Prefix in Transformer","created":"2024-06-20","cited_by_count":10},{"id":"2309.07452","title":"Is Solving Graph Neural Tangent Kernel Equivalent to Training Graph Neural Network?","created":"2023-09-14","cited_by_count":8},{"id":"2309.05135","title":"Streaming Semidefinite Programs: $O(\\sqrt{n})$ Passes, Small Space and Fast Runtime","created":"2023-09-10","cited_by_count":5},{"id":"2409.00055","title":"SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models","created":"2024-08-21","cited_by_count":5},{"id":"2203.03539","title":"Understanding The Robustness of Self-supervised Learning Through Topic Modeling","created":"2022-02-02","cited_by_count":3}],"next_cursor":null}