{"list":"other","total":68,"rows":[{"id":"1602.01783","title":"Asynchronous Methods for Deep Reinforcement Learning","created":"2016-02-04","cited_by_count":2258},{"id":"2309.17453","title":"Efficient Streaming Language Models with Attention Sinks","created":"2023-09-29","cited_by_count":2186},{"id":"1802.05957","title":"Spectral Normalization for Generative Adversarial Networks","created":"2018-02-16","cited_by_count":1636},{"id":"2211.10438","title":"SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models","created":"2022-11-18","cited_by_count":1386},{"id":"2002.04745","title":"On Layer Normalization in the Transformer Architecture","created":"2020-02-12","cited_by_count":922},{"id":"1905.09418","title":"Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned","created":"2019-05-23","cited_by_count":856},{"id":"1905.10650","title":"Are Sixteen Heads Really Better than One?","created":"2019-05-25","cited_by_count":816},{"id":"2305.02301","title":"Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes","created":"2023-05-03","cited_by_count":737},{"id":"2302.05442","title":"Scaling Vision Transformers to 22 Billion Parameters","created":"2023-02-10","cited_by_count":694},{"id":"1602.07868","title":"Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks","created":"2016-02-25","cited_by_count":543},{"id":"1701.06548","title":"Regularizing Neural Networks by Penalizing Confident Output Distributions","created":"2017-01-23","cited_by_count":464},{"id":"2305.01610","title":"Finding Neurons in a Haystack: Case Studies with Sparse Probing","created":"2023-05-02","cited_by_count":334},{"id":"2409.02060","title":"OLMoE: Open Mixture-of-Experts Language Models","created":"2024-09-03","cited_by_count":301},{"id":"2212.09720","title":"The case for 4-bit precision: k-bit Inference Scaling Laws","created":"2022-12-19","cited_by_count":251},{"id":"1906.04284","title":"Analyzing the Structure of Attention in a Transformer Language Model","created":"2019-06-07","cited_by_count":243},{"id":"2310.17884","title":"Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory","created":"2023-10-27","cited_by_count":230},{"id":"2410.10781","title":"When Attention Sink Emerges in Language Models: An Empirical View","created":"2024-10-14","cited_by_count":208},{"id":"2309.14322","title":"Small-scale proxies for large-scale Transformer training instabilities","created":"2023-09-25","cited_by_count":183},{"id":"1705.07798","title":"A unified view of entropy-regularized Markov decision processes","created":"2017-05-22","cited_by_count":182},{"id":"2209.13325","title":"Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models","created":"2022-09-27","cited_by_count":181},{"id":"1811.11214","title":"Understanding the impact of entropy on policy optimization","created":"2018-11-27","cited_by_count":176},{"id":"2310.07298","title":"Beyond Memorization: Violating Privacy Via Inference with Large Language Models","created":"2023-10-11","cited_by_count":175},{"id":"2006.14032","title":"Compositional Explanations of Neurons","created":"2020-06-24","cited_by_count":172},{"id":"2402.03898","title":"DistiLLM: Towards Streamlined Distillation for Large Language Models","created":"2024-02-06","cited_by_count":163},{"id":"2303.06296","title":"Stabilizing Transformer Training by Preventing Attention Entropy Collapse","created":"2023-03-11","cited_by_count":161},{"id":"2403.06634","title":"Stealing Part of a Production Language Model","created":"2024-03-11","cited_by_count":159},{"id":"2306.12929","title":"Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing","created":"2023-06-22","cited_by_count":151},{"id":"2306.14050","title":"Symbolic Chain-of-Thought Distillation: Small Models Can Also \"Think\" Step-by-Step","created":"2023-06-24","cited_by_count":148},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141},{"id":"2109.00984","title":"CrypTen: Secure Multi-Party Computation Meets Machine Learning","created":"2021-09-02","cited_by_count":131},{"id":"2402.04347","title":"The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry","created":"2024-02-06","cited_by_count":120},{"id":"2105.06990","title":"BERT Busters: Outlier Dimensions that Disrupt Transformers","created":"2021-05-14","cited_by_count":112},{"id":"2203.07852","title":"Block-Recurrent Transformers","created":"2022-03-11","cited_by_count":110},{"id":"2310.04564","title":"ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models","created":"2023-10-06","cited_by_count":94},{"id":"2405.15177","title":"Diffusion Actor-Critic with Entropy Regulator","created":"2024-05-24","cited_by_count":94},{"id":"2210.01351","title":"Less is More: Task-aware Layer-wise Distillation for Language Model Compression","created":"2022-10-04","cited_by_count":93},{"id":"2212.14034","title":"Cramming: Training a Language Model on a Single GPU in One Day","created":"2022-12-28","cited_by_count":81},{"id":"2406.15765","title":"Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration","created":"2024-06-22","cited_by_count":74},{"id":"2211.01452","title":"MPCFormer: fast, performant and private Transformer inference with MPC","created":"2022-11-02","cited_by_count":69},{"id":"2312.11420","title":"Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning","created":"2023-12-18","cited_by_count":64},{"id":"2312.06528","title":"Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context","created":"2023-12-11","cited_by_count":63},{"id":"2402.09221","title":"Spectral Filters, Dark Signals, and Attention Sinks","created":"2024-02-14","cited_by_count":56},{"id":"1911.09724","title":"Information-Theoretic Confidence Bounds for Reinforcement Learning","created":"2019-11-21","cited_by_count":51},{"id":"2405.18781","title":"On the Role of Attention Masks and LayerNorm in Transformers","created":"2024-05-29","cited_by_count":48},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47},{"id":"2410.01104","title":"Softmax is not Enough (for Sharp Size Generalisation)","created":"2024-10-01","cited_by_count":46},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45},{"id":"2405.06105","title":"Can Perplexity Reflect Large Language Model's Ability in Long Text Understanding?","created":"2024-05-09","cited_by_count":45},{"id":"2404.03828","title":"Outlier-Efficient Hopfield Layers for Large Transformer-Based Models","created":"2024-04-04","cited_by_count":38},{"id":"2311.01906","title":"Simplifying Transformer Blocks","created":"2023-11-03","cited_by_count":37},{"id":"2402.15607","title":"How Do Nonlinear Transformers Learn and Generalize in In-Context Learning?","created":"2024-02-23","cited_by_count":35},{"id":"2305.02582","title":"On the Expressivity Role of LayerNorm in Transformers' Attention","created":"2023-05-04","cited_by_count":33},{"id":"1710.03348","title":"What does Attention in Neural Machine Translation Pay Attention to?","created":"2017-10-09","cited_by_count":30},{"id":"2301.12017","title":"Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases","created":"2023-01-27","cited_by_count":23},{"id":"2402.04779","title":"StableMask: Refining Causal Masking in Decoder-only Transformer","created":"2024-02-07","cited_by_count":20},{"id":"2311.08610","title":"Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption","created":"2023-11-15","cited_by_count":19},{"id":"2402.02098","title":"Self-attention Networks Localize When QK-eigenspectrum Concentrates","created":"2024-02-03","cited_by_count":19},{"id":"2305.15008","title":"Are Chatbots Ready for Privacy-Sensitive Applications? An Investigation into Input Regurgitation and Prompt-Induced Sanitization","created":"2023-05-24","cited_by_count":17},{"id":"2108.08375","title":"Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks","created":"2021-08-18","cited_by_count":16},{"id":"2305.18399","title":"On the impact of activation and normalization in obtaining isometric embeddings at initialization","created":"2023-05-28","cited_by_count":13},{"id":"2311.13541","title":"Linear Log-Normal Attention with Unbiased Concentration","created":"2023-11-22","cited_by_count":11},{"id":"2309.11197","title":"The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute","created":"2023-09-20","cited_by_count":10},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10},{"id":"2008.12338","title":"Adversarially Robust Learning via Entropic Regularization","created":"2020-08-27","cited_by_count":9},{"id":"2406.01255","title":"On the Nonlinearity of Layer Normalization","created":"2024-06-03","cited_by_count":9},{"id":"2208.01134","title":"Improving the Trainability of Deep Neural Networks through Layerwise Batch-Entropy Regularization","created":"2022-08-01","cited_by_count":7},{"id":"2310.04604","title":"PriViT: Vision Transformers for Fast Private Inference","created":"2023-10-06","cited_by_count":4},{"id":"2406.03947","title":"Weight-based Decomposition: A Case for Bilinear MLPs","created":"2024-06-06","cited_by_count":2}],"next_cursor":null}