{"list":"other","total":73,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"2305.18290","title":"Direct Preference Optimization: Your Language Model is Secretly a Reward Model","created":"2023-05-29","cited_by_count":8065},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2205.01068","title":"OPT: Open Pre-trained Transformer Language Models","created":"2022-05-02","cited_by_count":3512},{"id":"2001.04451","title":"Reformer: The Efficient Transformer","created":"2020-01-13","cited_by_count":1668},{"id":"2303.17564","title":"BloombergGPT: A Large Language Model for Finance","created":"2023-03-30","cited_by_count":939},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"2301.10226","title":"A Watermark for Large Language Models","created":"2023-01-24","cited_by_count":725},{"id":"2011.04006","title":"Long Range Arena: A Benchmark for Efficient Transformers","created":"2020-11-08","cited_by_count":700},{"id":"1810.02054","title":"Gradient Descent Provably Optimizes Over-parameterized Neural Networks","created":"2018-10-04","cited_by_count":659},{"id":"1901.08584","title":"Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks","created":"2019-01-24","cited_by_count":566},{"id":"2310.06694","title":"Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning","created":"2023-10-10","cited_by_count":415},{"id":"2001.02610","title":"iDLG: Improved Deep Leakage from Gradients","created":"2020-01-08","cited_by_count":382},{"id":"1911.03894","title":"CamemBERT: a Tasty French Language Model","created":"2019-11-10","cited_by_count":368},{"id":"2305.17333","title":"Fine-Tuning Language Models with Just Forward Passes","created":"2023-05-27","cited_by_count":345},{"id":"1702.07464","title":"Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning","created":"2017-02-24","cited_by_count":334},{"id":"2202.06539","title":"Deduplicating Training Data Mitigates Privacy Risks in Language Models","created":"2022-02-14","cited_by_count":314},{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265},{"id":"2112.00059","title":"Evaluating Gradient Inversion Attacks and Defenses in Federated Learning","created":"2021-11-30","cited_by_count":196},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133},{"id":"1909.12292","title":"Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks","created":"2019-09-26","cited_by_count":121},{"id":"2210.05643","title":"A Kernel-Based View of Language Model Fine-Tuning","created":"2022-10-11","cited_by_count":121},{"id":"1601.01073","title":"Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism","created":"2016-01-06","cited_by_count":117},{"id":"2307.15936","title":"A Theory for Emergence of Complex Skills in Language Models","created":"2023-07-29","cited_by_count":113},{"id":"2302.06600","title":"Task-Specific Skill Localization in Fine-tuned Language Models","created":"2023-02-13","cited_by_count":105},{"id":"2302.13214","title":"Fast Attention Requires Bounded Entries","created":"2023-02-26","cited_by_count":102},{"id":"2302.10870","title":"On Provable Copyright Protection for Generative Models","created":"2023-02-21","cited_by_count":98},{"id":"2010.02772","title":"InstaHide: Instance-hiding Schemes for Private Distributed Learning","created":"2020-10-06","cited_by_count":90},{"id":"2303.04245","title":"How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding","created":"2023-03-07","cited_by_count":86},{"id":"2209.08773","title":"CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks","created":"2022-09-19","cited_by_count":76},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74},{"id":"2305.18465","title":"Federated Learning of Gboard Language Models with Differential Privacy","created":"2023-05-29","cited_by_count":66},{"id":"2004.10397","title":"A Framework for Evaluating Gradient Leakage Attacks in Federated Learning","created":"2020-04-22","cited_by_count":64},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62},{"id":"1906.03593","title":"Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound","created":"2019-06-09","cited_by_count":59},{"id":"2006.11648","title":"Training (Overparametrized) Neural Networks in Near-Linear Time","created":"2020-06-20","cited_by_count":58},{"id":"2302.02451","title":"KDEformer: Accelerating Transformers via Kernel Density Estimation","created":"2023-02-05","cited_by_count":58},{"id":"2304.10411","title":"Attention Scheme Inspired Softmax Regression","created":"2023-04-20","cited_by_count":49},{"id":"2102.10707","title":"A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization","created":"2021-02-21","cited_by_count":46},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45},{"id":"2004.07470","title":"Faster Dynamic Matrix Inverse for Faster LPs","created":"2020-04-16","cited_by_count":44},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44},{"id":"1609.00408","title":"Defeating Image Obfuscation with Deep Learning","created":"2016-09-01","cited_by_count":43},{"id":"2304.13276","title":"The Closeness of In-Context Learning and Weight Shifting for Softmax Regression","created":"2023-04-26","cited_by_count":41},{"id":"1905.11675","title":"Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems","created":"2019-05-28","cited_by_count":40},{"id":"2103.07601","title":"Approximating How Single Head Attention Learns","created":"2021-03-13","cited_by_count":38},{"id":"2304.02207","title":"Algorithm and Hardness for Dynamic Attention Maintenance in Large Language Models","created":"2023-04-05","cited_by_count":37},{"id":"2211.06033","title":"A Faster Small Treewidth SDP Solver","created":"2022-11-11","cited_by_count":36},{"id":"2310.01655","title":"PolySketchFormer: Fast Transformers via Sketching Polynomial Kernels","created":"2023-10-02","cited_by_count":36},{"id":"1912.03194","title":"Why are Adaptive Methods Good for Attention Models?","created":"2019-12-06","cited_by_count":35},{"id":"2307.02419","title":"In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trick","created":"2023-07-05","cited_by_count":32},{"id":"2211.14227","title":"Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing","created":"2022-11-25","cited_by_count":29},{"id":"2306.02295","title":"A Mathematical Abstraction for Balancing the Trade-off Between Creativity and Reality in Large Language Models","created":"2023-06-04","cited_by_count":29},{"id":"2212.03714","title":"Reconstructing Training Data from Model Gradient, Provably","created":"2022-12-07","cited_by_count":27},{"id":"2009.09829","title":"Generalized Leverage Score Sampling for Neural Networks","created":"2020-09-21","cited_by_count":26},{"id":"2308.12247","title":"How to Protect Copyright Data in Optimization of Large Language Models?","created":"2023-08-23","cited_by_count":26},{"id":"2305.04701","title":"Differentially Private Attention Computation","created":"2023-05-08","cited_by_count":25},{"id":"2308.10502","title":"GradientCoin: A Peer-to-Peer Decentralized Large Language Models","created":"2023-08-21","cited_by_count":23},{"id":"2210.12468","title":"Discrepancy Minimization in Input-Sparsity Time","created":"2022-10-22","cited_by_count":22},{"id":"2308.14304","title":"Solving Attention Kernel Regression Problem via Pre-conditioner","created":"2023-08-28","cited_by_count":20},{"id":"2307.08045","title":"Beyond Classical Attention: Quantum Attention for Scalable Computation","created":"2023-07-16","cited_by_count":18},{"id":"2310.08540","title":"Do pretrained Transformers Learn In-Context by Gradient Descent?","created":"2023-10-12","cited_by_count":18},{"id":"2307.01189","title":"Trainable Transformer in Transformer","created":"2023-07-03","cited_by_count":17},{"id":"2307.08352","title":"Zero-th Order Algorithm for Softmax Attention Optimization","created":"2023-07-17","cited_by_count":16},{"id":"2307.07735","title":"Faster Algorithms for Structured Linear and Kernel Support Vector Machines","created":"2023-07-15","cited_by_count":13},{"id":"2304.03426","title":"Convex Minimization with Integer Minima in $\\widetilde O(n^4)$ Time","created":"2023-04-07","cited_by_count":12},{"id":"2310.03331","title":"Fine-tune Language Models to Approximate Unbiased In-context Learning","created":"2023-10-05","cited_by_count":12},{"id":"2307.06565","title":"Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification","created":"2023-07-13","cited_by_count":11},{"id":"2310.08425","title":"Differentially Private Non-convex Learning for Multi-layer Neural Networks","created":"2023-10-12","cited_by_count":10},{"id":"2308.08358","title":"Convergence of Two-Layer Regression with Nonlinear Units","created":"2023-08-16","cited_by_count":6}],"next_cursor":null}